My problem:

Docker is pre-pending my_container_name | to every log entry. (You can see how the logs look further down, where the entries I'm interested in are the ones including JSON.)

I've been researching configurations for both the unified CloudWatch Agent (the "Some background info" below explains how that's being used) and Docker logging drivers but haven't found a way to modify the Docker logs output. I think I understand why Docker is pre-pending the container name to the logs (if multiple containers log to the same file, this would add a way to parse which entries are for which container), but doing this breaks how I'm expecting my logs to look.

In my opinion, it makes sense for my project code to be in complete control of the log data, with Docker being in control of the log location.

Question:

How can I configure Docker to do one of the following:

1. Prevent pre-pended text for each log entry?

   (so each entry is the raw data my Go project logs out)

2. Apply my own formatting to the output of each log entry?

   (such as applying a regex stripper to the output, removing the my_container_name | sub-string)

Some background info:

I have a Docker/Go project that deploys to AWS Elastic Beanstalk, where I'm needing to route the Docker logs to AWS CloudWatch logs so I can query each log entry there; I've setup a pre-deploy hook that configures the unified CloudWatch Agent to consume the Docker logs (see this SO answer for an example of the hook I've setup).

My Go code is logging data (to Stdout) in a JSON format so I can easily query specific fields/values in CloudWatch logs insights, but since Docker is pre-pending it's own text the entries are no longer in JSON format, meaning I can't easily query them.

The configuration for the unified CloudWatch agent pre-deploy hook doesn't allow me to do post-processing to log entries; I can only route the files I want to watch to go to CloudWatch logs. (I can do some basic filtering, but that will still include the entire log line; there's not a way for me to parse out just the JSON from the entry.)

Example of my Docker logs:

my_container_name  | [GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

my_container_name  | 

my_container_name  | [GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.

my_container_name  |  - using env: export GIN_MODE=release

my_container_name  |  - using code: gin.SetMode(gin.ReleaseMode)

my_container_name  | 

my_container_name  | [GIN-debug] GET    /eb/health_check          --> main.main.func1 (3 handlers)

my_container_name  | [GIN-debug] GET    /routes                   --> main.main.func2 (3 handlers)

my_container_name  | [GIN-debug] Loaded HTML Templates (2): 

my_container_name  |  - 502.html

my_container_name  |  - 

my_container_name  | 

my_container_name  | [GIN-debug] GET    /some/endpoint/clients --> github.com/xxx/xxx.routeClients (3 handlers)

my_container_name  | {"level":"info","msg":"routes setup","tag":"tag1","time":"2022-02-28T23:11:17.58730Z"}

my_container_name  | {"GlobalStatus":200,"Port":"3010","level":"info","msg":"all response status codes will be: 200","tag":"main","time":"2022-02-28T23:11:17.58745Z"}

my_container_name  | [GIN-debug] Listening and serving HTTP on :3010

CodePudding user response：

I don't have enough reputation to comment, but you might be able to accomplish this at runtime by specifying a log driver. See here for more info: https://docs.docker.com/config/containers/logging/log_tags/

CodePudding user response：

Plain Docker doesn't prepend the container name, but docker-compose does. If you're using docker-compose, you can specify the --no-log-prefix option on the docker-compose up command.