I see in the Airflow 2 SubdagOperator documentation link that using mode = reschedule we can get rid of potential deadlock. To my understanding it is not a param which can be be passed with list of other params. If anyone has used this let me know how to incorporate it in SubdagOperator.

CodePudding user response：

Technically, a SubDagOperator is a sensor, which can take an argument mode="reschedule". The default mode poke keeps a slot open, which can potentially lead to a deadlock situation in case you're using lots of sensors. Mode reschedule instead stops the process and creates a new process on every check, not causing a situation where all slots are occupied by sensors waiting on each other.

SubDagOperator(task_id="foobar", ..., mode="reschedule")

With that said, the SubDagOperator is deprecated since Airflow 2.0 and it is advised to use TaskGroups. TaskGroups are a visual way to group together tasks within a DAG (tutorial here: https://www.astronomer.io/guides/task-groups).

Alternatively, you can use the TriggerDagRunOperator to trigger another DAG (tutorial: https://www.astronomer.io/guides/cross-dag-dependencies).