Pain point · Reliability & bugs · Blocks work
Tasks intermittently terminated with SIGTERM on KubernetesExecutor
3 source threads · first seen 2020-08 · last seen 2021-09
Summary
Tasks on Airflow 2.1.3 with the Kubernetes executor are intermittently killed with SIGTERM, and suggested config changes did not stop it.
- Affects
- Teams on KubernetesExecutor
- Workaround
- Tried raising killed_task_cleanup_time and disabling schedule_after_task_execution without success
Evidence
Excerpts are copied word for word from the source; follow the link to read it in full.
“Received SIGTERM. Terminating subprocesses”
“the task will sometimes fail while the EMR step sucessfully ran”
“This usually appears when the task consumes a lot of CPU resources”