Trouble Remotely Connecting Flask App to Selenium Grid using Docker-CodePudding

I am new to both Docker and Selenium grid and am having issues getting my web app to connect to the selenium hub.

docker-compose.yml

version: '3.8'
services:
  db:
    image: postgres
    environment:
      POSTGRES_DB: ${POSTGRES_DB}
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    ports:
      - "${POSTGRES_PORT}:5432"
    volumes:
      - pgdata:/var/lib/postgresql/data
  web:
    build:
      context: ..
      dockerfile: docker/Dockerfile
    environment:
      FLASK_ENV: ${FLASK_ENV}
      FLASK_CONFIG: ${FLASK_CONFIG}
      APPLICATION_DB: ${APPLICATION_DB}
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_HOSTNAME: "db"
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_PORT: ${POSTGRES_PORT}
    command: flask run --host 0.0.0.0
    volumes:
      - ..:/opt/code
    ports:
      - "5000:5000"
  chrome: 
    image: selenium/node-chrome:4.0.0-20211013
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_NODE_GRID_URL=http://localhost:4444
    ports:
      - "6900:5900"

  edge:
    image: selenium/node-edge:4.0.0-20211013
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_NODE_GRID_URL=http://localhost:4444
    ports:
      - "6901:5900"

  firefox:
    image: selenium/node-firefox:4.0.0-20211013
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_NODE_GRID_URL=http://localhost:4444
    ports:
      - "6902:5900"

  selenium-hub:
    image: selenium/hub:4.0.0-20211013
    container_name: selenium-hub
    ports:
      - "4442:4442"
      - "4443:4443"
      - "4444:4444"


volumes:
  pgdata:

When running the stack of containers and checking netstat -a, I can see my desktop listening to port 4444 and when I kill the containers its not.

I can also verify that the hub is running and all of my nodes are connecting fine by visiting https//:localhost/4444, however when I run driver = webdriver.Remote(command_executor="http://localhost:4444") from my python flask app (which is running in the container specified as web above) I get the error:

urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=4444): Max retries
exceeded with url: /session (Caused by NewConnectionError('<urllib3.connection.HTTPConnection
object at 0x7fd4855b7730>: Failed to establish a new connection: [Errno 111] Connection refused'))

I have tried specifying the desired capabilities to match the driver for specific nodes, however I receive this same error regardless.

I am using Selenium's latest build "4.0.0" and as you can see 4.0.0 images for the parts of the grid so I don't think its a compatibility issue.

docker ps


       Name                       Command              State                            Ports                          
------------------------------------------------------------------------------------------------------------------------
development_chrome_1    /opt/bin/entry_point.sh         Up      0.0.0.0:6900->5900/tcp,:::6900->5900/tcp                
development_db_1        docker-entrypoint.sh postgres   Up      0.0.0.0:5432->5432/tcp,:::5432->5432/tcp                
development_edge_1      /opt/bin/entry_point.sh         Up      0.0.0.0:6901->5900/tcp,:::6901->5900/tcp                
development_firefox_1   /opt/bin/entry_point.sh         Up      0.0.0.0:6902->5900/tcp,:::6902->5900/tcp                
development_web_1       flask run --host 0.0.0.0        Up      0.0.0.0:5000->5000/tcp,:::5000->5000/tcp                
selenium-hub            /opt/bin/entry_point.sh         Up      0.0.0.0:4442->4442/tcp,:::4442->4442/tcp,               
                                                                0.0.0.0:4443->4443/tcp,:::4443->4443/tcp,               
                                                                0.0.0.0:4444->4444/tcp,:::4444->4444/tcp

I feel like I'm fundamentally missing something here, Any thoughts?

CodePudding user response：

I see the mistake now. I was mistakenly attempting to connect to http://localhost:4444 with my client, when I was needing to specify the network name deployed by selenium grid.

Fix

Change this line in your flask_app.py

driver = webdriver.Remote(command_executor="http://localhost:4444")

To:

driver = webdriver.Remote(command_executor="http://container-name:4444")

Where container-name is the selenium hub name set in docker-compose.yml

  selenium-hub:
    image: selenium/hub:4.0.0-20211013
    container_name: selenium-hub
    ports:
      - "4442:4442"
      - "4443:4443"
      - "4444:4444"

in my case: "selenium-hub"

Resource on docker networking used: https://docs.docker.com/compose/networking/

Final Thoughts

I guess I got tripped up by the fact that I am still using http://localhost:port to connect to both the grid hub and web container. I guess the difference is where the client request comes from? From outside the docker stack vs within? Anyway, hope this helps someone.