Pain point · Reliability & bugs · Causes churn
Scheduler stops scheduling DAG runs because the DAG record stays locked
2 source threads · first seen 2020-03 · last seen 2024-01
Summary
On Airflow 2.7.3 on Kubernetes, after a few successful runs the scheduler skips scheduling because the DAG record is locked, even with one scheduler.
- Affects
- Teams running frequent DAGs on Kubernetes
- Workaround
- Revert the change from the related PR
Evidence
Excerpts are copied word for word from the source; follow the link to read it in full.
“after a few successful DagRuns, the scheduler stops scheduling new runs”
“The scheduler gets stuck without a trace or error”