I am trying to implement a Reverse Proxy in Go that proxies traffic to different hosts based on some tenant embedded in the URL. The implementation looks like this:
type Offloader struct {
tenantHostMap map[string]string // Map a tenant to its host:port
tenantProxyMap map[string](*httputil.ReverseProxy) // Map a tenant to its reverse proxy
}
func (o *Offloader) OnCreate() {
// Tenants Map
o.tenantHostMap = make(map[string]string)
o.tenantProxyMap = make(map[string]*httputil.ReverseProxy)
o.PopulateTenantHostMap()
// Rx
http.HandleFunc("/", o.ServeHTTP)
go http.ListenAndServe(":5555", nil)
}
// ServeHTTP is the callback that is called each time a Http Request is received.
func (o *Offloader) ServeHTTP(w http.ResponseWriter, req *http.Request) {
incomingUrl := req.URL.RequestURI()
tenant := o.GetTenantFromUrl(incomingUrl)
if proxy, ok := o.tenantProxyMap[tenant]; ok {
proxy.ServeHTTP(w, req)
}
if remoteHostAddr, ok := o.tenantHostMap[tenant]; ok {
remoteUrl, err := url.Parse(fmt.Sprintf("http://%s", remoteHostAddr))
if err != nil {
return
}
proxy := httputil.NewSingleHostReverseProxy(remoteUrl)
o.tenantProxyMap[tenant] = proxy
proxy.ServeHTTP(w, req) // non blocking
} else {
panic("Unknown Tenant")
}
}
When receiving a new HTTP request, I get the tenant from the URL. If this is the first time I am seeing this tenant I create a new ReverseProxy, otherwise I try to use the one I created before and stored in the tenantProxyMap
.
When I test this, I get the following error:
2022/04/05 12:31:01 http: proxy error: readfrom tcp ****: http: invalid Read on closed Body
2022/04/05 12:31:01 http: superfluous response.WriteHeader call from net/http/httputil.(*ReverseProxy).defaultErrorHandler (reverseproxy.go:190)
If I create a new Reverse Proxy for each request rather than reusing the same proxy, the error doesn't happen.
I thought the proxy is per host and not per request (as the name suggests), so I am wondering why this error happens?
I know I need to protect the maps from concurrent reads/writes however that is irrelevant at the moment.
Thanks,
CodePudding user response:
The problem is that in the scenario where a previous proxy already existed, you first pass the request on to that - and then still recreate the proxy, and again pass the request. In other words: you are making two proxied requests for each incoming request, when the tentantProxyMap is already populated for that tenant.
The ReverseProxy
implementation closes the req.Body
, so the second time you pass the request on to the proxy, it attempts reading from an already closed body. You're seeing the http: invalid Read on closed Body
error as a result.
What you should try is to return after proxying the request, e.g. by adding a return
:
if proxy, ok := o.tenantProxyMap[tenant]; ok {
proxy.ServeHTTP(w, req)
return
}