How to rotate between multiple forwarding proxies for outgoing requests with golang-CodePudding

I will like to pass a list of forwarding proxy servers for POST request

Currently i am able to do it with just single forwarding proxy

    serverProxy := "http://user:[email protected]:3128"

    request, error := http.NewRequest("POST", httpposturl, bytes.NewBuffer(requestJSON))
    request.Header.Set("Content-Type", "application/json; charset=UTF-8")

    proxyURL, _ := url.Parse(serverProxy)
    proxy := http.ProxyURL(proxyURL)
    transport := &http.Transport{Proxy: proxy}
    client := &http.Client{Transport: transport}

what i will like to do is pass a list to url.Parse and want it to use them using round robin balancing

so something like this

serverProxy := "http://user:[email protected]:3128, http://user:[email protected]:3128"

and then it will select which of the proxy servers to use and rotate them within requests

Is this possible?

CodePudding user response：

Create a proxy function that round-robins through your proxy URLs. Use that function in your transport:

func roundRobin(urls []*url.URL) func(*http.Request) (*url.URL, error) {
    var mu sync.Mutex
    var i int
    return func(r *http.Request) (*url.URL, error) {
        mu.Lock()
        i := (i   1) % len(urls)
        u := urls[i]
        mu.Unlock()
        return u, nil
    }
}


transport := &http.Transport{Proxy: roundRobin(yourProxyURLs)}
client := &http.Client{Transport: transport}

CodePudding user response：

Here's the Montage's answer with code to parse a string.

func roundRobin(serverProxy string) func(*http.Request) (*url.URL, error) {
    parts := strings.Split(serverProxy, ",")
    var urls []*url.URL
    for _, part := range parts {
        u, err := url.Parse(strings.TrimSpace(part))
        if err != nil {
            log.Fatal(err)
        }
        urls = append(urls, u)
    }
    var mu sync.Mutex
    var i int
    return func(r *http.Request) (*url.URL, error) {
        mu.Lock()
        i := (i   1) % len(urls)
        u := urls[i]
        mu.Unlock()
        return u, nil
    }
}

serverProxy := "http://user:[email protected]:3128, http://user:[email protected]:3128"
transport := &http.Transport{Proxy: roundRobin(serverProxy)}
client := &http.Client{Transport: transport}