How to set up alertmanager.service for running in docker container-CodePudding

I am running prometheus in a docker container, and I want to configure an AlertManager for making it send me an email when the service is down. I created the alert_rules.yml and the prometheus.yml, and I run everything with the following command, mounting both the yml files onto the docker container at the path /etc/prometheus:

docker run -d -p 9090:9090 --add-host host.docker.internal:host-gateway -v "$PWD/prometheus.yml":/etc/prometheus/prometheus.yml -v "$PWD/alert_rules.yml":/etc/prometheus/alert_rules.yml prom/prometheus

Now, I also want prometheus to send me an email when an alert comes up, and that's where I encounter some problems. I configured my alertmanager.yml as follows:

route:
  group_by: ['alertname']
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 1h
  receiver: email-me
receivers:
- name: 'gmail'
  email_configs:
  - to: '[email protected]'
    from: '[email protected]'
    smarthost: smtp.gmail.com:587
    auth_username: '[email protected]'
    auth_identity: '[email protected]'
    auth_password: 'the_password'

I actually don't know if the smarthost parameter is configured correctly since I can't find any documentation about it and I don't know which values it should contain I also created an alertmanager.service file:

[Unit]
Description=AlertManager Server Service
Wants=network-online.target
After=network-online.target

[Service]
User=root
Group=root
Type=Simple
ExecStart=/usr/local/bin/alertmanager \
    --config.file /etc/alertmanager.yml

[Install]
WantedBy=multi-user.target

I think something here is messed up: I think the first parameter I pass to ExecStart is a path that doesn't exist in the container, but I have no idea on how I should replace it. I tried mounting the last two files into the docker container in the same directory where I mount the first two yml files by using the following command:

docker run -d -p 9090:9090 --add-host host.docker.internal:host-gateway -v "$PWD/prometheus.yml":/etc/prometheus/prometheus.yml -v "$PWD/alert_rules.yml":/etc/prometheus/alert_rules.yml -v "$PWD/alertmanager.yml":/etc/prometheus/alertmanager.yml -v "$PWD/alertmanager.service":/etc/prometheus/alertmanager.service prom/prometheus

But the mailing alert is not working and I don't know how to fix the configuration for smoothly running all of this into a docker container. As I said, I suppose the main problem resides in the ExecStart command present in alertmanager.service, but maybe I'm wrong. I can't find anything helpful online, hence I would really appreciate some help

CodePudding user response：

The best practice with containers is to aim to run a single process per container.

In your container, this suggests one container for prom/prometheus and another for prom/alertmanager.

You can run these using docker as:

docker run \
--detach \
--name=prometheus \
--volume=${PWD}:/prometheus.yml:/etc/prometheus/prometheus.yml \
--volume=${PWD}:/rules.yml:/etc/alertmanager/rules.yml \
--publish=9090:9090 \
prom/prometheus:v2.26.0 \
--config.file=/etc/prometheus/promtheus.yml

docker run \
--detach \
--name=alertmanager \
--volume=${PWD}:/rules.yml:/etc/alertmanager/rules.yml \
--publish=9093:9093 \
prom/alertmanager:v0.21.0

A good tool when you run multiple container is Docker Compose in which case, your docker-compose.yml could be:

version: "3"

services:
  prometheus:
    restart: always
    image: prom/prometheus:v2.26.0
    container_name: prometheus
    command:
      - --config.file=/etc/prometheus/prometheus.yml
    volumes:
      - ${PWD}/prometheus.yml:/etc/prometheus/prometheus.yml
      - ${PWD}/rules.yml:/etc/alertmanager/rules.yml
    expose:
      - "9090"
    ports:
      - 9090:9090

  alertmanager:
    restart: always
    depends_on:
      - prometheus
    image: prom/alertmanager:v0.21.0
    container_name: alertmanager
    volumes:
      - ${PWD}/alertmanager.yml:/etc/alertmanager/alertmanager.yml
    expose:
      - "9093"
    ports:
      - 9093:9093

and you could:

docker-compose up

In either case, you can then browse:

Prometheus on the host's port 9090 i.e. localhost:9090
Alert Manager on the host's port 9093, i.e. localhost:9093