Home > Blockchain >  Python threads run in parallel when in Docker container, but run sequentially when container is run
Python threads run in parallel when in Docker container, but run sequentially when container is run

Time:11-26

first post on SO, let me know how I can improve my question.

1. Summary

Goal: My app is a Craigslist scraper, it finds new posts and sends the feed to the user's email. It has a main thread that always listens to requests, and a worker thread, managed by the main thread, and is either working or sleeping.

Expected vs Actual Results: As mentioned in the title, the app works fine in the container locally, but when it's hosted on Cloud Run, the output and side-effects of the worker thread are deferred until the worker thread is killed.

It seems like the worker thread is only started when the main thread calls to kill it, at which point the worker thread only does one iteration of its work and then kills itself.

No errors messages.

2. What I've tried.

I've tried changing some of the Cloud Run container environment settings, like increasing amount of memory and number of vCPUs.

settings

Please let me know what additional information would help.

CodePudding user response:

Unless you enable No CPU Throttling for Cloud Run, you cannot use background threads.

See this article for the latest new feature that will support your workload:

Run more workloads on Cloud Run with new CPU allocation controls

Documentation

In summary add this command line option: --no-cpu-throttling

gcloud beta run services update SERVICE-NAME --no-cpu-throttling

Also set the number of CPUs greater than on --cpu=N

CodePudding user response:

Setting this solved the issue:

img

Lesson: Always check out all the settings.

  • Related