2

As far as it is my understanding, the only way I can set a proxy for a request is to assign it to a client and then use that for a request. However, I also understand I should reuse the clients for my requests. Is there any way to individually assign proxies for each request whilst using the same client? Or any way to efficiently do a collection of requests with a different proxy for each request?

2
  • What have you tried to set proxies? Commented Nov 15, 2017 at 8:46
  • tried assigning a proxy within a transport and then including that when I initialise a client. Then the proxy will be used within the requests issued by that client. I'm not aware of any other ways to achieve it Commented Nov 15, 2017 at 8:49

1 Answer 1

2

Creating of http.Client is not so expensive - its just a struct allocation that can be done for each request.

You should create http.Client and transport objects yourself and set proxy directly there.

proxyUrl, err := url.Parse("http://127.0.0.1:123") if err != nil { panic(err) } myClient := &http.Client{Transport: &http.Transport{Proxy: http.ProxyURL(proxyUrl)}} myClient.Get("http://www.google.com/") 

Tere is the other posibility to use always the same client with different proxies:

http.ProxyURL call from the first example is function:

// ProxyURL returns a proxy function (for use in a Transport) // that always returns the same URL. func ProxyURL(fixedURL *url.URL) func(*Request) (*url.URL, error) { return func(*Request) (*url.URL, error) { return fixedURL, nil } } 

You can extend it and chose proxy based on the information from request or implement some other logic.

UPD: here is implementation how to use ProxyUrl with only one Client. But overehead of changing context is comparable with overhead on creating a client. I would use the first scenario.

// We must write our own Get (or some other) request and add somewere the info that this is a proxy request. You can do it via context of request.

func GetWithProxy(c *http.Client,proxyUrl *url.URL, url string) (resp *http.Response, err error) { req, err := http.NewRequest("GET", url, nil) if err != nil { return nil, err } if proxyUrl != nil { req = req.WithContext(context.WithValue(context.Background(),"proxy",proxyUrl)) } return c.Do(req) } 

// Organize client that can use this information:

proxyUrl, err := url.Parse("http://127.0.0.1:123") if err != nil { panic(err) } myClient := &http.Client{Transport: &http.Transport{Proxy: func(req *http.Request) (*url.URL, error) { currentProxy,_ := req.Context().Value("proxy").(*url.URL) return currentProxy, nil }}} 

Use it:

GetWithProxy(myClient,nil,"http://www.google.com/") GetWithProxy(myClient,proxyUrl,"http://www.google.com/") 
Sign up to request clarification or add additional context in comments.

6 Comments

So you're saying that it wouldn't actually be too costly to create a new client for each request I make, correct? That's good to hear. Will look into extending the method you've given as well, thanks :)
@JordanAllen: Yes - client its just struct with 4 fields . Transport is the interface - http.Transport is abit more than client itself - if you want you can preapre it for each proxy but actually it does not worth .
Amazing answer. Thank you, given me a lot of options and a lot of help understanding my options. Appreciate it!
Never create an http.Transport for each request (though I can't tell if that's what you're recommending or not). Not reusing the transport will leak open connections.
@JimB: You are right - Transport can keep connection open.
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.