Why is the Recv-Q value in netstat equal to socket backlog 1?-CodePudding

When I execute netstat -tulnp, the output is as follows:

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.11:43043        0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.0.1:8005          0.0.0.0:*               LISTEN      1/java
tcp        0      0 0.0.0.0:2021            0.0.0.0:*               LISTEN      1/java
tcp        0      0 0.0.0.0:22222           0.0.0.0:*               LISTEN      1/java
tcp        0      0 0.0.0.0:8719            0.0.0.0:*               LISTEN      1/java
tcp      101      0 0.0.0.0:80              0.0.0.0:*               LISTEN      1/java
tcp       51      0 0.0.0.0:1234            0.0.0.0:*               LISTEN      1/java
tcp        0      0 0.0.0.0:20891           0.0.0.0:*               LISTEN      1/java
udp        0      0 127.0.0.11:55285        0.0.0.0:*                           -

The value of Recv-Q caught my attention. After my investigation, I found that OOM occurred in the JVM application, and it can be found in the log that the http-nio-80-Acceptor-0 thread responsible for monitoring port 80 has exited, and the thread responsible for dispatching port 1234 requests has exited. The relevant logs are as follows:

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "http-nio-80-Acceptor-0"
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Thread-5"

The default configuration used by tomcat, that is, the backlog is 100, and the source code is located at tomcat/AbstractEndpoint.java at 8.5.59 · apache/tomcat · GitHub:

/**
  * Allows the server developer to specify the acceptCount (backlog) that
  * should be used for server sockets. By default, this value
  * is 100.
  */
private int acceptCount = 100;
public void setAcceptCount(int acceptCount) { if (acceptCount > 0) this.acceptCount = acceptCount; }
public int getAcceptCount() { return acceptCount; }

The listening of port 1234 is triggered and created by HTTPServer, and the created code is HttpServer.create(new InetSocketAddress(PROMETHEUS_SERVER_PORT), 0);, backlog is corrected to 50 in ServerSocket.java, the source code is located at jdk/ServerSocket.java at jdk8-b120 · openjdk/jdk · GitHub:

public void bind(SocketAddress endpoint, int backlog) throws IOException {
    if (isClosed())
        throw new SocketException("Socket is closed");
    if (!oldImpl && isBound())
        throw new SocketException("Already bound");
    if (endpoint == null)
        endpoint = new InetSocketAddress(0);
    if (!(endpoint instanceof InetSocketAddress))
        throw new IllegalArgumentException("Unsupported address type");
    InetSocketAddress epoint = (InetSocketAddress) endpoint;
    if (epoint.isUnresolved())
        throw new SocketException("Unresolved address");
    if (backlog < 1)
      backlog = 50;
    try {
        SecurityManager security = System.getSecurityManager();
        if (security != null)
            security.checkListen(epoint.getPort());
        getImpl().bind(epoint.getAddress(), epoint.getPort());
        getImpl().listen(backlog);
        bound = true;
    } catch(SecurityException e) {
        bound = false;
        throw e;
    } catch(IOException e) {
        bound = false;
        throw e;
    }
}

From netstat(8) - Linux manual page we know, Recv-Q indicates current syn backlog when socket is in Listening state, what confuses me is why Recv-Q is one more than the value of backlog we set?

CodePudding user response：

TL;DR netstat/ss is reporting complete and incomplete connections while the backlog is only concerned about completed connections

we know, Recv-Q indicates current syn backlog when socket is in Listening state,

Yes this is confirmed in the sock_diag man page. It looks like netlink takes this value from the following kernel structure:

rql.udiag_rqueue = sk->sk_receive_queue.qlen;

Looking into qlen more, it looks like it is not the same as backlog. The qlen includes complete and incomplete connection, while the backlog is only concerned with completed connections. The listen man page makes note of this:

The behavior of the backlog argument on TCP sockets changed with Linux 2.2. Now it specifies the queue length for completely established sockets waiting to be accepted, instead of the number of incomplete connection requests.

As for why the receive queue cares about incomplete connections, I can only speculate that it is trying to handle the edge case of a the backlog freeing up while the TCP handshake is still occurring.

CodePudding user response：

A backlog value of N really does mean allow "N 1" connections to queue to a listening socket. This allows one to specify "0" as the backlog and still get 1 connection.

Reference: NET: Revert incorrect accept queue backlog changes. · torvalds/linux@64a1465 · GitHub