As far as it is my understanding, the only way I can set a proxy for a request is to assign it to a client and then use that for a request. However, I also understand I should reuse the clients for my requests. Is there any way to individually assign proxies for each request whilst using the same client? Or any way to efficiently do a collection of requests with a different proxy for each request?
- What have you tried to set proxies?Eugene Lisitsky– Eugene Lisitsky2017-11-15 08:46:39 +00:00Commented Nov 15, 2017 at 8:46
- tried assigning a proxy within a transport and then including that when I initialise a client. Then the proxy will be used within the requests issued by that client. I'm not aware of any other ways to achieve itJames Morgan– James Morgan2017-11-15 08:49:20 +00:00Commented Nov 15, 2017 at 8:49
1 Answer
Creating of http.Client is not so expensive - its just a struct allocation that can be done for each request.
You should create http.Client and transport objects yourself and set proxy directly there.
proxyUrl, err := url.Parse("http://127.0.0.1:123") if err != nil { panic(err) } myClient := &http.Client{Transport: &http.Transport{Proxy: http.ProxyURL(proxyUrl)}} myClient.Get("http://www.google.com/") Tere is the other posibility to use always the same client with different proxies:
http.ProxyURL call from the first example is function:
// ProxyURL returns a proxy function (for use in a Transport) // that always returns the same URL. func ProxyURL(fixedURL *url.URL) func(*Request) (*url.URL, error) { return func(*Request) (*url.URL, error) { return fixedURL, nil } } You can extend it and chose proxy based on the information from request or implement some other logic.
UPD: here is implementation how to use ProxyUrl with only one Client. But overehead of changing context is comparable with overhead on creating a client. I would use the first scenario.
// We must write our own Get (or some other) request and add somewere the info that this is a proxy request. You can do it via context of request.
func GetWithProxy(c *http.Client,proxyUrl *url.URL, url string) (resp *http.Response, err error) { req, err := http.NewRequest("GET", url, nil) if err != nil { return nil, err } if proxyUrl != nil { req = req.WithContext(context.WithValue(context.Background(),"proxy",proxyUrl)) } return c.Do(req) } // Organize client that can use this information:
proxyUrl, err := url.Parse("http://127.0.0.1:123") if err != nil { panic(err) } myClient := &http.Client{Transport: &http.Transport{Proxy: func(req *http.Request) (*url.URL, error) { currentProxy,_ := req.Context().Value("proxy").(*url.URL) return currentProxy, nil }}} Use it:
GetWithProxy(myClient,nil,"http://www.google.com/") GetWithProxy(myClient,proxyUrl,"http://www.google.com/") 6 Comments
http.Transport for each request (though I can't tell if that's what you're recommending or not). Not reusing the transport will leak open connections.