How to properly configure HAProxy in Docker Swarm to automatically route traffic to replicated servi-CodePudding

I'm trying to deploy a Docker Swarm of three host nodes with a single replicated service and put an HAProxy in front of it. I want the clients to be able to connect via SSL.

My docker-compose.yml:

version: '3.9'

services:
  proxy:
    image: haproxy
    ports:
      - 443:8080
    volumes:
      - haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg
    deploy:
      placement:
        constraints: [node.role == manager]
    networks:
      - servers-network
  node-server:
    image: glusk/hackathon-2021:latest
    ports:
      - 8080:8080
    command: npm run server
    deploy:
      mode: replicated
      replicas: 2
    networks:
      - servers-network
networks:
  servers-network:
    driver: overlay

My haproxy.cfg (based on the official example):

# Simple configuration for an HTTP proxy listening on port 80 on all
# interfaces and forwarding requests to a single backend "servers" with a
# single server "server1" listening on 127.0.0.1:8000
global
    daemon
    maxconn 256

defaults
    mode http
    timeout connect 5000ms
    timeout client 50000ms
    timeout server 50000ms

frontend http-in
    bind *:80
    default_backend servers

backend servers
    server server1 127.0.0.1:8000 maxconn 32

My hosts are Lightsail VPS Ubuntu instances and share the same private network.

node-service runs each https server task inside its own container on: 0.0.0.0:8080.

The way I'm trying to make this work at the moment is to ssh into the manager node (which also has a static and public IP), copy over my configuration files from above, and run:

docker stack deploy --compose-file=docker-compose.yml hackathon-2021

but it doesn't work.

CodePudding user response：

Well, first of all and regarding SSL (since it's the first thing that you mention) you need to configure it using the certificate and listen on the port 443, not port 80.

With that modification, your Proxy configuration would already change to:

global
    daemon
    maxconn 256

defaults
    mode http
    timeout connect 5000ms
    timeout client 50000ms
    timeout server 50000ms

frontend http-in
    bind *:80
    default_backend servers

frontend https-in
    bind *:443 ssl crt /etc/ssl/certs/hackaton2021.pem
    default_backend servers

That would be a really simplified configuration for allowing SSL connection.

Now, let's go for the access to the different services.

First of all, you cannot access to the service on localhost, actually you shouldn't even expose the ports of the services you have to the host. The reason? That you already have those applications in the same network than the haproxy, so the ideal would be to take advantage of the Docker DNS to access directly to them

In order to do this, first we need to be able to resolve the service names. For that you need to add the following section to your configuration:

resolvers docker
    nameserver dns1 127.0.0.11:53
    resolve_retries 3
    timeout resolve 1s
    timeout retry   1s
    hold other      10s
    hold refused    10s
    hold nx         10s
    hold timeout    10s
    hold valid      10s
    hold obsolete   10s

The Docker Swarm DNS service is always available at 127.0.0.11.

Now to your previous existent configuration, we would have to add the server but using the service-name discovery:

backend servers
    balance roundrobin
    server-template node- 2 node-server:8080 check resolvers docker init-addr libc,none

If you check what we are doing, we are creating a server for each one of the discovered containers in the Swarm within the node-server service (so the replicas) and we will create those adding the prefix node- to each one of them.

Basically, that would be the equivalent to get the actual IPs of each of the replicas and add them stacked as a basic server configuration.

For deployment, you also have some errors, since we aren't interested into actually expose the node-server ports to the host, but to create the two replicas and use HAProxy for the networking.

For that, we should use the following Docker Compose:

version: '3.9'

services:
  proxy:
    image: haproxy
    ports:
      - 80:80
      - 443:443
    volumes:
      - hackaton2021.pem:/etc/ssl/certs/hackaton2021.pem
      - haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg
    deploy:
      placement:
        constraints: [node.role == manager]

  node-server:
    image: glusk/hackathon-2021:latest
    command: npm run server
    deploy:
      mode: replicated
      replicas: 2

Remember to copy your haproxy.cfg and the self-signed (or real) certificate for your application to the instance before deploying the Stack.

Also, when you create that stack it will automatically create a network with the name <STACK_NAME>-default, so you don't need to define a network just for connecting both services.