Loading from multiple sources causing timeouts

[qmestu]

Hi,

I have a dashboard that gets its data from multiple webservice calls. As it contains more and more data, the refresh sometimes fails with a timeout. I was wondering what strategies can i use to prevent this. Right now, i combine several tables into one, and each of those tables is related to a webservice call.

For example, would it be possible to have a .pbix file for each webservice call, and have a master file that just reads from multiple .pbix files and updates the visuals?

Thank you.

[rohit1991]

To prevent timeouts when loading data from multiple web service calls in Power BI:

1. Split Workloads: Use separate .pbix files for each web service call, process them independently, and store the results in a central data source (e.g., Azure SQL, Dataverse, or a shared folder).

2. Master File: Create a master .pbix file that connects to the pre-processed data sources instead of directly hitting the web services.

3. Optimize Refresh:

   • Enable incremental refresh for large datasets.
   • Reduce data pulled by filtering or aggregating at the source.

4. Use Dataflows: Move the logic to Power BI Dataflows for preprocessing, which allows better handling of multiple sources.

This approach improves performance and reduces timeouts during refresh.

[qmestu]

Can you explain how do i setup option 1 ? Thanks.

[Anonymous]

Thanks for the reply from rohit1991 , please allow me to provide another insight:

Hi, @qmestu 
Thanks for reaching out to the Microsoft fabric community forum.

Regarding your need to merge data from multiple data sources into a unified dataset, it reminds me of the Dataflow product. Dataflows allow you to extract data from multiple different data sources and integrate this data into a unified data model. This is very useful for scenarios where you need to gather data from multiple systems for comprehensive analysis.

Here is a screenshot of the relevant documentation:

Dataflows can be configured with an automatic refresh schedule to ensure data is updated regularly. This means you can set the dataflow to automatically extract the latest data from the source systems at specific intervals, perform the necessary transformations, and load it accordingly. By pre-processing and transforming the data, dataflows can reduce report loading times and improve query performance. This is especially important for handling large datasets and complex calculations.

For more details, please refer to:

Configure and consume a dataflow - Power BI | Microsoft Learn
 

However, if each of your visual objects only comes from a single web data source, you might also consider using the Dashboard product. Dashboards support visual objects from multiple reports:

For more details, please refer to: 

Basic concepts for designers in the Power BI service - Power BI | Microsoft Learn

Of course, if you have any new discoveries or questions, please feel free to get in touch with us.
 

Best Regards,

Leroy Lu

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.