I am trying to include celery into our Django app and am struggling with the setup. So far all my searching of stackoverflow/google tells me I have a circular dependency, but I can't see it. The docs, https://docs.celeryproject.org/en/stable/getting-started/first-steps-with-celery.html, clearly use from celery import Celery

I have defined: app_name/app_name_celery.py with

from celery import signals, Celery
import os
from django.conf import settings
# disable celery logging so that it inherits from the configured root logger
@signals.setup_logging.connect
def setup_celery_logging(**kwargs):
    pass

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings.local")

# Create default Celery app
app = Celery('app_name')

# namespace='CELERY' means all celery-related configuration keys
# should be uppercased and have a `CELERY_` prefix in Django settings.
# https://docs.celeryproject.org/en/stable/userguide/configuration.html
app.config_from_object("django.conf:settings", namespace="CELERY")

# When we use the following in Django, it loads all the <appname>.tasks
# files and registers any tasks it finds in them. We can import the
# tasks files some other way if we prefer.
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

Additionally, I have defined app_name/my_app_config.py with

from django.apps import AppConfig


class MyAppConfig(AppConfig):
    # ...

    def ready(self):
        # Import celery app now that Django is mostly ready.
        # This initializes Celery and autodiscovers tasks
        import app_name.app_name_celery

Lastly, I have added to my __init__.py:

# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .app_name import app as celery_app

__all__ = ('celery_app',)

Also, the logging setup is unchanged for this pr, but here it is:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'json': {
            '()': 'app_name.utils.logging.JSONFormatter'
        }
    },
    'handlers': {
        'console': {
            'level': 'DEBUG',
            'class': 'logging.StreamHandler',
            'formatter': 'json'
        }
    },
    'loggers': {
        'ddtrace': {
            'level': 'WARNING'
        }
    },
    'root': {
        'level': 'DEBUG',
        'handlers': ['console']
    }
}

Can anyone see the circular dependency or what else I may be missing?

CodePudding user response：

You are trying to import django settings on

app_name/app_name_celery.py

from celery import signals, Celery
import os
from django.conf import settings # This line here

Also you don't need to import celery into your app config.

Auto discover finds tasks in app/tasks.py, app_2/tasks.py etc.