I have a tricky problem, as per the title I want to run a tcp server on the host machine that communicates to clients inside docker containers. I know you can do this with the server in a container, using the -p option to expose ports in the container and match them to the host. Or you could even setup a custom network and connect to the containers by name. My issue though is that I need the sever to be running on the host machine. This is because in our workflow we have various different containers that run different tools and we want to be able to coordinate them.

My grand plan therefore is to have a manager container that reads an input file that list all the tools needed for the workflow. The manger then sends a request to a tcp server to spawn various containers (which thus needs to be running on the host). The server then spawns the requested containers and sends the ip and port of the new container to the manger. The manger and container then communicate what needs to be done between themselves.

Now I already have this working in Apptainer and now we plan to use docker for wider support (mostly mac/windows versions). However docker's network isolation is causing issues.

If I start the server with ip "0.0.0.0" (i.e. listening on all ip's) bond to port 9000 then docker run -p 9000:9000 manger_container (i.e. start the manager container and expose port 9000 in the container to the host). Docker throws an error saying the port is in use. If i try the other way round (i.e. start the container then start server, i get the same issue only this time the server complains the port is "in use". I've also had no joy with --network=host (which incidentally wouldn't help if it did work because as far as i know it's Linux only). So at this stage I just have no idea how to set this up or if it's even possible with docker.

CodePudding user response：

First way of doing so would be mounting the docker.socket into you Manager container.

As the error message says, your trying to start to 2 services on the same port. If I get your use-case right you should start the server on one side and connect as client to the service, instead of spinning up a second service.

CodePudding user response：

A container is, at its core, just a process on the host operating system. And, by publishing a port you have exposed that process on the host system. So you essentially have 2 processes listening on :9000 - the client, and the server.

You have several options:

1. If the code is your's: clarify the role of your client and server - servers listen, clients connect - only one of the processes should need to publish :9000 if the names "client" and "server" are meaningful.
2. Assuming that's not an option, then put the server inside a container too. Then there's no need to publish ports on the host, and the client and server can connect to each other as server:9000 and client:9000 respectively.
3. Otherwise, one of them needs to listen / connect on a port other than 9000.