nginx tcp stream (k8s) - keep client connection open when upstream closes-CodePudding

I have an application that accepts TCP traffic (not HTTP) and I'd like the ability to have the traffic load balanced to it. However, one requirement is that when a client makes a connection, we do not close that connection under any circumstances (ideally) since we are dealing with some clients with older technology.

I've set up the kubernetes nginx ingress controller, but it isn't behaving how I'm hoping/expecting. What I would like is: If the connection to one of the upstream servers closes, then the client connection remains open for some amount of time while nginx picks a new upstream server and starts sending data to it. I am not concerned about the stream's data being split across different upstream servers, I just need the connection to stay open from the client's perspective during something like a redeploy.

What is actually happening is that from my client's perspective, currently when the upstream server app closes the connection, my connection is closed and I have to reconnect.

The ingress controller has this configuration, which I thought would accomplish what I want, but it doesn't seem to be working as expected:


       server {
               preread_by_lua_block {
                       ngx.var.proxy_upstream_name="tcp-my-namespace-my-service-7550";
               }

               listen                  7550;

               proxy_timeout           600s;
               proxy_next_upstream     on;
               proxy_next_upstream_timeout 600s;
               proxy_next_upstream_tries   3;

               proxy_pass              upstream_balancer;

       }

Any help at all is greatly appreciated and I'm happy to provide more info.

CodePudding user response：

Unfortunately I haven't been able to get this functionality working with nginx. What I've ended up doing is writing my own basic TCP reverse-proxy that does what I need - if a connection to a backend instance is lost, it attempts to get a new one without interrupting the frontend connection. The traffic that we receive is fairly predictable in that I don't expect that moving the connection will interrupt any of the "logical" messages on the stream 99% of the time.

I'd still love to hear if anyone knows of an existing tool that has this functionality, but at the moment I'm convinced that there isn't one readily available.

CodePudding user response：

I think you need to configure your Nginx Ingress to enable the keepalive options as listed in the documentation here. For instance in your nginx configuration as:

...
keepalive 32;
...

This will activate the keepalive functionality with a cache of upto 32 connections active at a time.