Connect python app container, postgres container, and persistent database on external harddrive with-CodePudding

Using docker-compose, I am running a postgres container (db). The data itself is persistently stored on my Windows machine. And this works. I am unable to get another container running a python application to access the database.

My docker-compose file is as follows, where I use ## to denote some options that I've tried that :

version: '0.1'

services
    
  db: #also the server name
    image: postgres:15.0
    restart: always
      
    environment:
       
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: my_password
      ##POSTGRES_HOST_AUTH_METHOD: trust
      ##POSTGRES_HOST: postgres
      ##PGDATA: d:/pg #/var/lib/postgresql/data/pgdata
    ports:
        #port on machine:port on container
      - 1234:5432
    volumes: 
        #path on my windows machine:path to postgres default folder
      - D:/pg:/var/lib/postgresql/data
    privileged:
        true

  app:
    image: simple_python_debug

    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=my_password
      - POSTGRES_PORT=5432
      - POSTGRES_DB_NAME=postgres
      - POSTGRES_HOSTNAME=localhost

    depends_on:
      - db

My dockerfile (simple_python_debug) for the python script is

FROM python:3.9

# Install pip requirements
COPY requirements.txt .
RUN pip3 install -r requirements.txt
WORKDIR /app #this is where the .py file is located
COPY . /app

# Creates a non-root user with an explicit UID and adds permission to access the /app folder
# For more info, please refer to https://aka.ms/vscode-docker-python-configure-containers
RUN adduser -u 5678 --disabled-password --gecos "" appuser && chown -R appuser /app
USER appuser

CMD ["python", "test.py"]

The dockerfile for postgres is simply:

services:

  db:
    image: postgres:15.0
    restart: always
    environment:
      POSTGRES_PASSWORD: my_password

test.py (stored in /app) contains

import os
from sqlalchemy import create_engine

port = os.getenv('POSTGRES_PORT')#'5432'
password = os.getenv('POSTGRES_PASSWORD')#'my_password'
user = os.getenv('POSTGRES_USER')#'postgres'
hostname = os.getenv('POSTGRES_HOSTNAME') #'localhost'
database_name = os.getenv('POSTGRES_DB_NAME') #'postgres'
connection_params = 'postgresql psycopg2://'   user      ':'   password   '@'   hostname   '/'   database_name 
    # \ "connect_args = {'options': '-csearch_path={}'.format("   schema   ')}'
engine = create_engine(connection_params)

With docker desktop running on my local windows machine and postgres service on the local windows machine shutdown for good measure, I run

.../test>docker-compose up from powershell 7; part of the message for the working postgres container is:

test-db-1   |
test-db-1   | PostgreSQL Database directory appears to contain a database; Skipping initialization
test-db-1   |
test-db-1   | 2022-11-12 15:27:14.665 UTC [1] LOG:  starting PostgreSQL 15.0 (Debian 15.0-1.pgdg110 1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
test-db-1   | 2022-11-12 15:27:14.665 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
test-db-1   | 2022-11-12 15:27:14.665 UTC [1] LOG:  listening on IPv6 address "::", port 5432
test-db-1   | 2022-11-12 15:27:14.674 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"

test.py fails on calling create_engine; the tail of the error message is

test-app-1  |     Is the server running on host "localhost" (127.0.0.1) and accepting
test-app-1  |     TCP/IP connections on port 5432?
test-app-1  | could not connect to server: Cannot assign requested address
test-app-1  |     Is the server running on host "localhost" (::1) and accepting
test-app-1  |     TCP/IP connections on port 5432?
test-app-1  |
test-app-1  | (Background on this error at: https://sqlalche.me/e/14/e3q8)

There are several stack overflow related questions and the error message. There's even medium articles on using postgres with python and docker.

https://stefanopassador.medium.com/docker-compose-with-python-and-posgresql-45c4c5174299

Docker - How can run the psql command in the postgres container?

Can't connect python app container to local postgres

From inside of a Docker container, how do I connect to the localhost of the machine?

Perhaps I'm missing something simple, but any suggestions or pointing to the appropriate docker docs could help get me unstuck.

CodePudding user response：

It will be better to define the volumes before using them like here.

Also if you want postgres' data to persist you need to place inside the volume the folder

/var/lib/postgresql/data

something like

volumes:
 - fancy-volume:
 ... fancy definition..

services:
 fancy-app:
   ...
   volumes:
   - fancy-volume:/var/lib/postgresql/data

CodePudding user response：

This log is the key:

Is the server running on host "localhost" (127.0.0.1) and accepting TCP/IP connections on port 5432?

In docker there is no localhost.

To demonstrate this, just replace localhost by the ip in which the postgress is running.

To understand why localhost should kot be used in docker & docker-compose and what are the solutions, check:

If this work for you, add the exact error log in the question to help future generations