Forum Discussion
Inexplicable DataflowsStagingWarehouse Load on Capacity
Hi sviesa,
Your detail about activity every 30-60 seconds, 24/7, while your three dataflows only run for about 5 minutes, is the key clue. Dataflow refreshes can't produce that pattern, so something other than the refreshes is talking to the staging warehouse. While your support case runs, I would narrow it down with a few elimination tests.
1. Prove it is (or isn't) the dataflows. Pause all three dataflow schedules for a few hours and watch the staging warehouse item in the Capacity Metrics app:
schedules paused, activity continues -> not the refreshes:
something else is polling
schedules paused, activity stops -> tied to the dataflows:
go to step 22. Turn off staging where you don't need it. For simple copy-and-drop-a-column queries landing in a Lakehouse, staging adds a warehouse round trip without benefit. In each dataflow, right-click each query and clear Enable staging, then republish and compare the next day's CU on the staging items.
3. Look for anything querying through the staging artifacts. The "High Scale Dataflow Compute - Warehouse Query" activity every minute looks like a consumer rather than a writer. Things worth checking:
semantic models / reports connected to a dataflow
(Dataflows connector) with frequent refresh or DirectQuery
Power Automate / Logic Apps flows polling dataflow output
external tools (Excel, notebooks) connected to the dataflow
monitoring scripts calling dataflow or workspace APIsConsumers connected through the dataflow connector read via the staging warehouse, so a single report on a short refresh interval, or a DirectQuery model, would produce exactly this "every minute" pattern.
4. Give support the timeline. The fact that it started on a specific Monday with no pipeline change is useful evidence:
staging warehouse item name + workspace ID
exact start date of the extra activity
operation names + CU per hour from Capacity Metrics
result of the "schedules paused" test aboveSo my overall approach would be:
pause dataflow schedules -> does the activity stop?
-> disable staging on simple queries
-> find consumers reading through the dataflow connector
-> share the timeline and test results with supportThe pause test alone will tell you within a few hours whether you are looking for a refresh problem or a hidden consumer.
AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.