Apache Airflow

On Airflow 2.0.0 with KubernetesExecutor, tasks stay scheduled or queued indefinitely even though nothing is running and the default pool appears full.

Affects: Teams on KubernetesExecutor

Tasks on Airflow 2.1.3 with the Kubernetes executor are intermittently killed with SIGTERM, and suggested config changes did not stop it.

Affects: Teams on KubernetesExecutor

On Airflow 2.0.1 the scheduler grows to several GB over days and tasks get stuck; the old option to restart it periodically is gone.

Affects: Teams on KubernetesExecutor

Thousands of tasks, about 2000 a day for one team, move to retry with an executor reports finished while queued error, causing significant SLA impact.

Affects: Teams using ExternalTaskSensors with Celery

On Airflow 2.7.3 on Kubernetes, after a few successful runs the scheduler skips scheduling because the DAG record is locked, even with one scheduler.

Affects: Teams running frequent DAGs on Kubernetes

Disabling a DAG stops it midway, which is a problem during upgrades; a drain option is requested to let active runs finish first.

Affects: Operators upgrading Airflow environments

TaskGroups replaced SubDags but cannot be retried, so the user cannot repeat a group of tasks without rerunning the whole large DAG.

Affects: DAG authors replacing SubDags

The Docker image fails to start on ARM nodes such as Graviton or Apple M1, and the user requests ARM64 support for released images.

Affects: Teams running ARM Kubernetes nodes

Running Airflow on Windows needs WSL or Docker, which have GPU and networking limitations; a user asks for native support, replacing the Unix daemon library.

Affects: Windows-based companies running GPU ML pipelines

With many teams on one server, permissions must be granted DAG by DAG, so tag-based read and write permissions are requested.

Affects: Platform teams serving multiple teams

At busy times Airflow launches hundreds of worker processes each needing a database connection, so client-side pooling does not help and a server-side pooler is used.

Affects: Teams running Airflow clusters on Postgres

A self-hosted Airflow occasionally had its disk filled by DAG logs; the user found maintenance DAGs that prevent many such issues.

Affects: Self-hosting small team

A user on managed Airflow sees DAG processes killed with no explanation in the logs and is moving to Dagster mainly because it has not crashed on them.

Affects: Data engineers on AWS MWAA

A user calls the documentation badly lacking and says this makes running and using Airflow needlessly frustrating, to the point of not using it again.