Dataflow - timeout after 10 minutes

[ganiu_]

Hi, 

I have some SQL data that takes about 13 minutes to process for 1.66 million rows. However, when I try to pull it into Dataflow, it times out after 10 minutes. Is there a way to increase the timeout limit, or can you recommend an alternative solution?
Thanks

[ganiu_]

hi thanks, but that didnt work 

[v-kpoloju-msft · Accepted solution]

Hi @ganiu_,

Thank you for reaching out to the Microsoft fabric community forum. Also, thanks to @GilbertQ, @Royel, for those inputs on this thread.

The issue you are experiencing is due to a known limitation in Power BI Dataflows, where query execution is limited to a maximum of 10 minutes in the Power Query Online environment. Since your SQL query takes approximately 13 minutes, it exceeds this limit and results in a timeout. Dataflows Limitations, restrictions and supported connectors and features - Power BI | Microsoft Lea...
Power Query Online Usage Limits - Power Query | Microsoft Learn

Optimize Your SQL Query: Try reducing execution time using filtering, indexing, or pre-aggregated views. This might help bring the processing time under the 10-minute cap.
Dataflows best practices - Power BI | Microsoft Learn

Use a Staging Table: Load your 1.66M row query into a staging table via SQL Server Agent or automation. Then, point your Dataflow to the already populated staging table, which loads faster. This approach helps avoid timeout errors by removing heavy query logic from the Dataflow refresh process.

Use Power BI Desktop Instead: If Dataflows aren’t mandatory for your scenario, you can load the full dataset in Power BI Desktop (which doesn’t have a 10-minute timeout). Then publish the report and use scheduled refresh via gateway, if needed. Data refresh in Power BI - Power BI | Microsoft Learn
Using incremental refresh with dataflows - Power Query | Microsoft Learn

Hope this helps clarify things and let me know what you find after giving these steps a try happy to help you investigate this further.

Thank you for using the Microsoft Community Forum.

[GilbertQ]

Hi @ganiu_ 

Have you also looked at using incremental refresh as this will typically take a lot quicker to run as you're not gonna have to load all your rows in a single instance which means you will use less time. Here is how to do the incremental refresh Using incremental refresh with dataflows - Power Query | Microsoft Learn

[Royel]

Hi @ganiu_  lets try this 

Increase SQL Command Timeout

Sql.Database("server", "database", [CommandTimeout = #duration(0, 0, 30, 0)])

 
Split Queries:
After you save/publish your dataflow gen2 we require the validation/publish process to finish within 10 minutes per query. If you exceed this 10 minute limit try to simplify your queries or split your queries in dataflow gen2.

Check out Dataflow Gen2 others limitations: Link

Is it helped? ✔ Give a Kudo • Mark as Solution – help others too!