Prefect
prefect.io · Data pipelines
Workflow orchestration for data pipelines in Python.
46 items · 56 source threads · 2 source types · updated 2026-05
No items match these filters.
Pain points 29
| Item | Area | Severity | Support | Last seen |
|---|---|---|---|---|
| Mapped tasks run multiple times on GKE Autopilot On GKE Autopilot, mapped child tasks sometimes start several times, even after the original has succeeded, leaving flow runs in inconsistent states. Hours-long tasks make it costly. Affects: teams running flows on GKE Autopilot |
Reliability & bugs | Causes churn | 3 sources | 2025-07 |
| ||||
| Completed flow crashes after concurrency lease renewal fails On 3.4.20 a flow whose tasks all finished ends as Crashed after a lease renewal failure terminates execution. |
Reliability & bugs | Blocks work | 3 sources | 2025-10 |
| ||||
| Server start fails because docker-compose file is invalid After the 0.14.7 release, starting the local Prefect server fails because docker-compose rejects the depends_on entries. Affects: users starting the server locally |
Self-hosting & upgrades | Causes churn | 2 sources | 2024-01 |
| ||||
| Frequent Postgres deadlocks after upgrading to Prefect 3.3.1 A Helm-deployed Prefect server on EKS with RDS Postgres started hitting frequent deadlock errors in the message consume loop after the upgrade. Affects: teams running Prefect server on EKS with RDS |
Reliability & bugs | Blocks work | 2 sources | 2026-01 |
| ||||
| Passing large data between tasks eats memory that is never released A user who otherwise likes Prefect's design finds that large task inputs and outputs stay in memory even after deleting them, unless result persistence is disabled on the task. Affects: data engineers passing large data frames |
Performance | Blocks work | 2 sources | 2024-04 |
| ||||
| Agent intermittently fails to fetch runs from Prefect Cloud after upgrade Locked: summary and evidence are part of Sprint Pass and Pro |
Reliability & bugs | Blocks work | 2 sources | 2024-01 |
| Server stops responding during heavy concurrent flows using blocks Locked: summary and evidence are part of Sprint Pass and Pro |
Reliability & bugs | Blocks work | 2 sources | 2022-11 |
| RayTaskRunner flows fail with sqlite database is locked Locked: summary and evidence are part of Sprint Pass and Pro |
Integrations | Blocks work | 2 sources | 2022-10 |
| Task occasionally fails because its serialized result file is missing Locked: summary and evidence are part of Sprint Pass and Pro |
Reliability & bugs | Blocks work | 2 sources | 2022-09 |
| Prefect 3 server memory climbs and feels unstable after migration Locked: summary and evidence are part of Sprint Pass and Pro |
Reliability & bugs | Causes churn | 1 source | 2025-07 |
| No bulk deletion of flows in the UI is a dealbreaker Locked: summary and evidence are part of Sprint Pass and Pro |
UI & usability | Causes churn | 1 source | 2022-07 |
| Windows workers cannot authenticate the events websocket to a Linux server Locked: summary and evidence are part of Sprint Pass and Pro |
Setup & onboarding | Blocks work | 1 source | 2026-02 |
| Hung Prefect workflows leave orphaned pods holding expensive compute Locked: summary and evidence are part of Sprint Pass and Pro |
Reliability & bugs | Blocks work | 1 source | 2025-09 |
| Prefect 3 server loops time out on slow database calls Locked: summary and evidence are part of Sprint Pass and Pro |
Performance | Blocks work | 1 source | 2024-12 |
| Simultaneous ECS flows collide registering task definitions Locked: summary and evidence are part of Sprint Pass and Pro |
Integrations | Blocks work | 1 source | 2024-10 |
| Task runs stop appearing in local server UI after 3.0.0rc19 Locked: summary and evidence are part of Sprint Pass and Pro |
Reliability & bugs | Blocks work | 1 source | 2024-08 |
| Python multiprocessing inside a flow deadlocks in Docker Locked: summary and evidence are part of Sprint Pass and Pro |
Reliability & bugs | Blocks work | 1 source | 2023-09 |
| Unpinned anyio 4.0.0 release broke Prefect import Locked: summary and evidence are part of Sprint Pass and Pro |
Reliability & bugs | Blocks work | 1 source | 2023-08 |
| Self-hosted UI shows a blank screen over a corporate VPN Locked: summary and evidence are part of Sprint Pass and Pro |
Self-hosting & upgrades | Blocks work | 1 source | 2023-08 |
| Second concurrent flow using prefect-dbt fails with ModuleNotFoundError Locked: summary and evidence are part of Sprint Pass and Pro |
Reliability & bugs | Blocks work | 1 source | 2023-01 |
| Flow shows Running after the agent is stopped Locked: summary and evidence are part of Sprint Pass and Pro |
Reliability & bugs | Blocks work | 1 source | 2022-10 |
| A few Docker deployment runs per day crash within seconds Locked: summary and evidence are part of Sprint Pass and Pro |
Reliability & bugs | Blocks work | 1 source | 2022-08 |
| Mapping about 1500 tasks hits database QueuePool errors Locked: summary and evidence are part of Sprint Pass and Pro |
Performance | Blocks work | 1 source | 2022-08 |
| UI on another machine ignores the configured API URL Locked: summary and evidence are part of Sprint Pass and Pro |
Self-hosting & upgrades | Blocks work | 1 source | 2022-04 |
| Cancelling a flow leaves Kubernetes or Dask compute running Locked: summary and evidence are part of Sprint Pass and Pro |
Reliability & bugs | Blocks work | 1 source | 2021-09 |
| Docker agent runs stuck in Submitted on local server Locked: summary and evidence are part of Sprint Pass and Pro |
Setup & onboarding | Blocks work | 1 source | 2021-09 |
| Kubernetes cluster autoscaler can kill long-running flow runs Locked: summary and evidence are part of Sprint Pass and Pro |
Reliability & bugs | Blocks work | 1 source | 2020-07 |
| Cron schedules effectively require the Prefect UI Locked: summary and evidence are part of Sprint Pass and Pro |
Documentation | Annoyance | 1 source | 2022-09 |
| Unclear which deployment files belong in source control Locked: summary and evidence are part of Sprint Pass and Pro |
Documentation | Annoyance | 1 source | 2022-09 |
Feature requests 12
| Item | Area | Severity | Support | Last seen |
|---|---|---|---|---|
| Run flows on remote infrastructure without creating a deployment for each Multi-infrastructure workflows force a separate deployment per flow plus run_deployment, causing deployment sprawl and brittle coupling to server-side names. The author wants direct submission to remote infrastructure. Affects: teams running workflows across several infrastructures |
API & developer experience | Blocks work | 1 source | 2025-02 |
| ||||
| Flow retries should also apply to crashed runs An automation that reruns crashed flows loops forever on a deterministic crash, and the flow retries setting does not apply to crashes. The user wants retries respected on Crashed state with a maximum count. |
Reliability & bugs | Blocks work | 1 source | 2023-07 |
| ||||
| Deploying should not require every flow dependency to be installed Deploying flows fails unless all dependencies of every flow are present, which is a problem with per-flow virtual environments and makes CI/CD duplicative of a Docker build. Affects: teams deploying multiple flows from CI/CD |
API & developer experience | Blocks work | 1 source | 2023-05 |
| ||||
| Clear late runs in bulk so a restarted agent does not run them all When an agent goes down and restarts, all missed late runs are picked up immediately. Users want to filter runs by state, tag or queue and bulk cancel or delete them. Affects: users with hourly scheduled deployments |
UI & usability | Blocks work | 1 source | 2022-09 |
| ||||
| Bring back module storage to package flows with their dependencies A team that bakes flow code into its container image wants module-path storage like in Prefect 1.0, rather than needing external S3 or Git storage that adds failure points. Affects: teams packaging flows in Docker images on Kubernetes |
Setup & onboarding | Blocks work | 1 source | 2022-06 |
| ||||
| Orchestrators should resume a partially failed DAG without redoing work Locked: summary and evidence are part of Sprint Pass and Pro |
Reliability & bugs | Blocks work | 1 source | 2021-10 |
| Docs needed for using Prefect with Slurm and other HPC schedulers Locked: summary and evidence are part of Sprint Pass and Pro |
Documentation | Annoyance | 1 source | 2023-07 |
| Support a tags argument on the flow decorator Locked: summary and evidence are part of Sprint Pass and Pro |
API & developer experience | Annoyance | 1 source | 2023-04 |
| Show Pydantic model parameter fields in defined order in the UI Locked: summary and evidence are part of Sprint Pass and Pro |
UI & usability | Annoyance | 1 source | 2023-02 |
| Add caching at the flow level, not only per task Locked: summary and evidence are part of Sprint Pass and Pro |
API & developer experience | Annoyance | 1 source | 2022-10 |
| Allow subflows to be submitted to run in the background Locked: summary and evidence are part of Sprint Pass and Pro |
API & developer experience | Annoyance | 1 source | 2022-09 |
| Publish arm64 Docker images for Prefect Locked: summary and evidence are part of Sprint Pass and Pro |
Self-hosting & upgrades | Annoyance | 1 source | 2022-01 |
Workarounds 4
| Item | Area | Severity | Support | Last seen |
|---|---|---|---|---|
| Schedules intermittently create duplicate runs at the same time Deployments sometimes get two scheduled runs with identical start times, apparently after maintenance or redeployments. The team runs a cleanup script that toggles schedules to fix it. Affects: teams on Kubernetes with Helm |
Reliability & bugs | Blocks work | 2 sources | 2025-09 |
| ||||
| Per-flow ECS task launch adds minutes of startup delay Each Prefect flow starts in its own ECS task, so slow container startup on Fargate added up. The team moved to an EC2 autoscaling group with a custom AMI that has images pre-baked, cutting startup from about 3 minutes to 30 seconds. Affects: teams running flows on AWS ECS |
Performance | Blocks work | 1 source | 2025-11 |
| ||||
| Prefect server OOMs fixed only by doubling resources The server was seen running out of memory and restarting on Kubernetes. Doubling CPU and memory limits eased the problem. Affects: team running Prefect server on Kubernetes |
Self-hosting & upgrades | Blocks work | 1 source | 2024-12 |
| ||||
| Running the same subflow concurrently with different parameters fails Launching two concurrent calls to the same subflow raises an error that the task runner is already started. A staff member suggested using a task instead. Affects: users running parallel subflows |
Reliability & bugs | Blocks work | 1 source | 2022-05 |
| ||||
Switching reasons 1
| Item | Area | Severity | Support | Last seen |
|---|---|---|---|---|
| Team prefers Temporal over Prefect and Airflow for orchestration A team that built a large business on Temporal says it feels much better than the orchestrators they used before, Prefect and Airflow, and they would adopt it again. Affects: engineering team building business-critical applications |
Other | Causes churn | 1 source | 2026-05 |
| ||||