I read the docs, concerning the .listen() method, used in express. I can USE the method and setup a server that is listening to HTTP requests.

However, since I am fairly new to coding, I find it difficult to grasp whats really happening when using the .listen() method. The high level explanation "listening for connections" didn't help me.

I think, this could be made easier if I could actually see the function instead of only calling it.

Any help is very much appreciated

CodePudding user response：

In a nutshell, the Express app.listen() method creates an http server object and then configures it to receive incoming TCP connections on a specific port and IP address so that when clients request a connection to that port and send an http request, the server can receive that http request and process it, sending a response. The code in app.listen() is shown below later in the answer - though all it does is call down to one further layer down in the http server object.

Here are the lower level details for how that works.

When a server wishes to start listening for incoming connections, it informs the local TCP stack by creating a socket and binding to a particular port and IP address. That essentially reserves that incoming port for this particular server (no other server will be allowed to also bind to that port). So, for example, on a regular http server on the default port, you would bind to port 80. This type of bound socket is used for incoming connections only, not for two-way communications with a client.

Then, the server informs the TCP stack that it is ready for incoming connections. At the TCP level, this is referred to as listen. Within nodejs, the bind and listen steps are combined into the one step called listen.

From then on, whenever the local TCP stack receives an incoming connecting request whose destination is the IP address and port that the server bound to, then that incoming connection will be accepted and inserted into a queue for the server that is configured for that IP address and port. There will typically be a maximum number of incoming connections that can be queued in this way and, if that number is exceeded, then the connection will be refused. This manages load and protects the host if the server gets "backed up" and is behind on processing incoming connections.

The server will then be informed by the TCP stack for each new incoming connection. Once the server accepts that connection, then it can start reading any data that the client has sent over the socket. In the case of an HTTP server working with the HTTP protocol, this would be the initial request protocol, method, version, headers and any body data. For different types of servers, the data would be in a different format.

Here's a useful diagram of the server:

Source: https://medium.com/javarevisited/fundamentals-of-socket-programming-in-java-bc9acc30eaf4

1. The server creates a socket used for the server to accept new connections..
2. It binds that socket to a specific IP address and port so it will only be informed about incoming connections targeted to that IP address and port.
3. It listens on that port to inform the TCP stack it is ready to accept incoming connections.
4. When it is notified of an incoming connection, it accepts that incoming connection.
5. Then it can read and write to that new connection over the new socket.
6. Then, sometime later, the incoming socket is closed to complete the client transaction.

The app.listen() method in Express encapsulates these steps and a few others. Internally (within Express), the code looks like this:

app.listen = function listen() {
  var server = http.createServer(this);
  return server.listen.apply(server, arguments);
};

You can see that method here in the open source repository.

To get an http server ready for steps 1-6 above, this creates the http server object within nodejs and then registers the app as the request listener for that server object (so it will be notified of incoming http requests).

Then, the call to server.listen() encapsulates steps 1-3 above.

Step 4 happens inside the http server object implementation and the app object is called when a new connection has been established and a new HTTP request is available. The http server reads the initial request and parses the http protocol and that initial request is already made available to the app for routing to the appropriate handler.

Then, subsequent calls such as res.send() or res.json() write a response back on the http socket and close the socket or res.end() will close it directly (steps 5 and 6 above).

Some other useful references:

Why is bind() used in TCP? Why is it used only on server side and not in client side? - Helps explain how a port and IP address define the TCP endpoint represented by a server. This port has to be known by the client so it can specifically request to connect to that port. The client end of the socket also has an IP address and a port, but its port can be dynamically assigned, thus the client does not have to bind to a specific port itself. The four pieces of data [server IP, server port, client IP, client port] define a specific TCP connection.

How TCP sockets work - has a good section about how new connections to a server work.

Understanding socket and port in TCP - talks about active and passive sockets. Passive sockets are sockets in "listen" mode used to accept incoming connections. Active sockets are two-way communications channels between two TCP endpoints.

Transmission Control Protocol (TCP) - more details on the various aspects of TCP from initiating a listening server, initiating a client connection to that server, through packet transmission to closing the socket.

There are a gazillion other references on the topic on the web. You can probably find 1000 articles on any single aspect of TCP that you might want more info about.

I think, this could be made easier if I could actually see the function instead of only calling it.

The underlying code for listen is inside the operating system's TCP stack and is not part of nodejs or Express. Express relies on the nodejs http server object as its interface to that and the nodejs http server object uses native code (built into nodejs) to call libuv (which is a cross platform C library that nodejs uses for networking and other things). Then, libuv talks to the underlying operating system APIs to reach the actual TCP stack on that target host. All of this is to put the server socket into listen mode so it can be notified of new incoming client connections to that target IP address and port.

Here's some doc on the related portions of the Linux TCP API if you want to see what the underlying TCP interface and description of that interface is:

socket() - https://linux.die.net/man/7/socket

bind() - https://linux.die.net/man/2/bind

listen() - https://linux.die.net/man/2/listen

And, portions of the libuv library that nodejs uses for networking:

TCP handles - http://docs.libuv.org/en/v1.x/tcp.html

Server listen() and accept() - http://docs.libuv.org/en/v1.x/stream.html#c.uv_listen