Friction

Prefect

prefect.io · Data pipelines

Workflow orchestration for data pipelines in Python.

46 items · 56 source threads · 2 source types · updated 2026-05

Compare

Pain points 29

ItemAreaSeveritySupportLast seen
Mapped tasks run multiple times on GKE Autopilot

On GKE Autopilot, mapped child tasks sometimes start several times, even after the original has succeeded, leaving flow runs in inconsistent states. Hours-long tasks make it costly.

Affects: teams running flows on GKE Autopilot

Reliability & bugsCauses churn3 sources2025-07

“Data skew, because downstream tasks re-process the same data input and causes issue”

“Occasionally, the Prefect agent submits duplicate flow runs to the infrastructure”

Completed flow crashes after concurrency lease renewal fails

On 3.4.20 a flow whose tasks all finished ends as Crashed after a lease renewal failure terminates execution.

Reliability & bugsBlocks work3 sources2025-10

“Concurrency lease renewal failed - slots are no longer reserved. Terminating execution to prevent over-allocation”

“It never happened when we were running on prefect agent and started appearing after migration”

Server start fails because docker-compose file is invalid

After the 0.14.7 release, starting the local Prefect server fails because docker-compose rejects the depends_on entries.

Affects: users starting the server locally

Self-hosting & upgradesCauses churn2 sources2024-01

“is not writeable in scenarios where the Docker user is non-root”

“services.ui.depends_on contains an invalid type, it should be an array”

Frequent Postgres deadlocks after upgrading to Prefect 3.3.1

A Helm-deployed Prefect server on EKS with RDS Postgres started hitting frequent deadlock errors in the message consume loop after the upgrade.

Affects: teams running Prefect server on EKS with RDS

Reliability & bugsBlocks work2 sources2026-01

“when I run multiple background services version 3.6.9 I get this exact err”

“Lately, we've been experiencing frequent deadlocks after upgrading to Prefect 3.3.1”

Passing large data between tasks eats memory that is never released

A user who otherwise likes Prefect's design finds that large task inputs and outputs stay in memory even after deleting them, unless result persistence is disabled on the task.

Affects: data engineers passing large data frames

PerformanceBlocks work2 sources2024-04

“Memory consumption continues to increase with the number of task runs”

“This eaten up memory cannot be released, or at least, I could not find any option to do so”

Agent intermittently fails to fetch runs from Prefect Cloud after upgrade

Locked: summary and evidence are part of Sprint Pass and Pro

Reliability & bugsBlocks work2 sources2024-01
Server stops responding during heavy concurrent flows using blocks

Locked: summary and evidence are part of Sprint Pass and Pro

Reliability & bugsBlocks work2 sources2022-11
RayTaskRunner flows fail with sqlite database is locked

Locked: summary and evidence are part of Sprint Pass and Pro

IntegrationsBlocks work2 sources2022-10
Task occasionally fails because its serialized result file is missing

Locked: summary and evidence are part of Sprint Pass and Pro

Reliability & bugsBlocks work2 sources2022-09
Prefect 3 server memory climbs and feels unstable after migration

Locked: summary and evidence are part of Sprint Pass and Pro

Reliability & bugsCauses churn1 source2025-07
No bulk deletion of flows in the UI is a dealbreaker

Locked: summary and evidence are part of Sprint Pass and Pro

UI & usabilityCauses churn1 source2022-07
Windows workers cannot authenticate the events websocket to a Linux server

Locked: summary and evidence are part of Sprint Pass and Pro

Setup & onboardingBlocks work1 source2026-02
Hung Prefect workflows leave orphaned pods holding expensive compute

Locked: summary and evidence are part of Sprint Pass and Pro

Reliability & bugsBlocks work1 source2025-09
Prefect 3 server loops time out on slow database calls

Locked: summary and evidence are part of Sprint Pass and Pro

PerformanceBlocks work1 source2024-12
Simultaneous ECS flows collide registering task definitions

Locked: summary and evidence are part of Sprint Pass and Pro

IntegrationsBlocks work1 source2024-10
Task runs stop appearing in local server UI after 3.0.0rc19

Locked: summary and evidence are part of Sprint Pass and Pro

Reliability & bugsBlocks work1 source2024-08
Python multiprocessing inside a flow deadlocks in Docker

Locked: summary and evidence are part of Sprint Pass and Pro

Reliability & bugsBlocks work1 source2023-09
Unpinned anyio 4.0.0 release broke Prefect import

Locked: summary and evidence are part of Sprint Pass and Pro

Reliability & bugsBlocks work1 source2023-08
Self-hosted UI shows a blank screen over a corporate VPN

Locked: summary and evidence are part of Sprint Pass and Pro

Self-hosting & upgradesBlocks work1 source2023-08
Second concurrent flow using prefect-dbt fails with ModuleNotFoundError

Locked: summary and evidence are part of Sprint Pass and Pro

Reliability & bugsBlocks work1 source2023-01
Flow shows Running after the agent is stopped

Locked: summary and evidence are part of Sprint Pass and Pro

Reliability & bugsBlocks work1 source2022-10
A few Docker deployment runs per day crash within seconds

Locked: summary and evidence are part of Sprint Pass and Pro

Reliability & bugsBlocks work1 source2022-08
Mapping about 1500 tasks hits database QueuePool errors

Locked: summary and evidence are part of Sprint Pass and Pro

PerformanceBlocks work1 source2022-08
UI on another machine ignores the configured API URL

Locked: summary and evidence are part of Sprint Pass and Pro

Self-hosting & upgradesBlocks work1 source2022-04
Cancelling a flow leaves Kubernetes or Dask compute running

Locked: summary and evidence are part of Sprint Pass and Pro

Reliability & bugsBlocks work1 source2021-09
Docker agent runs stuck in Submitted on local server

Locked: summary and evidence are part of Sprint Pass and Pro

Setup & onboardingBlocks work1 source2021-09
Kubernetes cluster autoscaler can kill long-running flow runs

Locked: summary and evidence are part of Sprint Pass and Pro

Reliability & bugsBlocks work1 source2020-07
Cron schedules effectively require the Prefect UI

Locked: summary and evidence are part of Sprint Pass and Pro

DocumentationAnnoyance1 source2022-09
Unclear which deployment files belong in source control

Locked: summary and evidence are part of Sprint Pass and Pro

DocumentationAnnoyance1 source2022-09

Feature requests 12

ItemAreaSeveritySupportLast seen
Run flows on remote infrastructure without creating a deployment for each

Multi-infrastructure workflows force a separate deployment per flow plus run_deployment, causing deployment sprawl and brittle coupling to server-side names. The author wants direct submission to remote infrastructure.

Affects: teams running workflows across several infrastructures

API & developer experienceBlocks work1 source2025-02

“they are forced to create a deployment for each workflow and use run_deployment to invoke the created deployment”

Flow retries should also apply to crashed runs

An automation that reruns crashed flows loops forever on a deterministic crash, and the flow retries setting does not apply to crashes. The user wants retries respected on Crashed state with a maximum count.

Reliability & bugsBlocks work1 source2023-07

“This nicely catches random crashes but goes haywire when an error inherent to the flow causes crashes”

Deploying should not require every flow dependency to be installed

Deploying flows fails unless all dependencies of every flow are present, which is a problem with per-flow virtual environments and makes CI/CD duplicative of a Docker build.

Affects: teams deploying multiple flows from CI/CD

API & developer experienceBlocks work1 source2023-05

“This also makes deploying via cicd cumbersome because, again, all dependencies need to be present”

Clear late runs in bulk so a restarted agent does not run them all

When an agent goes down and restarts, all missed late runs are picked up immediately. Users want to filter runs by state, tag or queue and bulk cancel or delete them.

Affects: users with hourly scheduled deployments

UI & usabilityBlocks work1 source2022-09

“Ability to clear late runs scheduled for the past so that they are not picked up when the agent gets restarted”

Bring back module storage to package flows with their dependencies

A team that bakes flow code into its container image wants module-path storage like in Prefect 1.0, rather than needing external S3 or Git storage that adds failure points.

Affects: teams packaging flows in Docker images on Kubernetes

Setup & onboardingBlocks work1 source2022-06

“is there any plans to bring back 'module' storage as in prefect 1.0?”

Orchestrators should resume a partially failed DAG without redoing work

Locked: summary and evidence are part of Sprint Pass and Pro

Reliability & bugsBlocks work1 source2021-10
Docs needed for using Prefect with Slurm and other HPC schedulers

Locked: summary and evidence are part of Sprint Pass and Pro

DocumentationAnnoyance1 source2023-07
Support a tags argument on the flow decorator

Locked: summary and evidence are part of Sprint Pass and Pro

API & developer experienceAnnoyance1 source2023-04
Show Pydantic model parameter fields in defined order in the UI

Locked: summary and evidence are part of Sprint Pass and Pro

UI & usabilityAnnoyance1 source2023-02
Add caching at the flow level, not only per task

Locked: summary and evidence are part of Sprint Pass and Pro

API & developer experienceAnnoyance1 source2022-10
Allow subflows to be submitted to run in the background

Locked: summary and evidence are part of Sprint Pass and Pro

API & developer experienceAnnoyance1 source2022-09
Publish arm64 Docker images for Prefect

Locked: summary and evidence are part of Sprint Pass and Pro

Self-hosting & upgradesAnnoyance1 source2022-01

Workarounds 4

ItemAreaSeveritySupportLast seen
Schedules intermittently create duplicate runs at the same time

Deployments sometimes get two scheduled runs with identical start times, apparently after maintenance or redeployments. The team runs a cleanup script that toggles schedules to fix it.

Affects: teams on Kubernetes with Helm

Reliability & bugsBlocks work2 sources2025-09

“Scheduled runs are duplicated with the same expected_start_time”

“Occasionally a specific flow will get double-scheduled in the latest version of Prefect server”

Per-flow ECS task launch adds minutes of startup delay

Each Prefect flow starts in its own ECS task, so slow container startup on Fargate added up. The team moved to an EC2 autoscaling group with a custom AMI that has images pre-baked, cutting startup from about 3 minutes to 30 seconds.

Affects: teams running flows on AWS ECS

PerformanceBlocks work1 source2025-11

“each prefect flow launches in a discrete ECS task so the waiting added up”

Prefect server OOMs fixed only by doubling resources

The server was seen running out of memory and restarting on Kubernetes. Doubling CPU and memory limits eased the problem.

Affects: team running Prefect server on Kubernetes

Self-hosting & upgradesBlocks work1 source2024-12

“I've noticed the perfect server experiencing OOMs and restarting, doubling its CPU and memory limits alleviated this”

Running the same subflow concurrently with different parameters fails

Launching two concurrent calls to the same subflow raises an error that the task runner is already started. A staff member suggested using a task instead.

Affects: users running parallel subflows

Reliability & bugsBlocks work1 source2022-05

“i would like to run the same flow with different parameters”

Switching reasons 1

ItemAreaSeveritySupportLast seen
Team prefers Temporal over Prefect and Airflow for orchestration

A team that built a large business on Temporal says it feels much better than the orchestrators they used before, Prefect and Airflow, and they would adopt it again.

Affects: engineering team building business-critical applications

OtherCauses churn1 source2026-05

“Temporal is amazing, it feels much better than previous orchestrators I used like Prefect or Airflow”