Forum Discussion

qmestu's avatar
qmestu
Helper IV
1 year ago

Loading from multiple sources causing timeouts

Hi,   I have a dashboard that gets its data from multiple webservice calls. As it contains more and more data, the refresh sometimes fails with a timeout. I was wondering what strategies can i use ...
Need Help
rohit1991's avatar
rohit1991
Super User
1 year ago

To prevent timeouts when loading data from multiple web service calls in Power BI:

  1. Split Workloads: Use separate .pbix files for each web service call, process them independently, and store the results in a central data source (e.g., Azure SQL, Dataverse, or a shared folder).

  2. Master File: Create a master .pbix file that connects to the pre-processed data sources instead of directly hitting the web services.

  3. Optimize Refresh:

    • Enable incremental refresh for large datasets.
    • Reduce data pulled by filtering or aggregating at the source.

  4. Use Dataflows: Move the logic to Power BI Dataflows for preprocessing, which allows better handling of multiple sources.

This approach improves performance and reduces timeouts during refresh.

 

  • qmestu's avatar
    qmestu
    Helper IV
    1 year ago

    Can you explain how do i setup option 1 ? Thanks.

    • Anonymous's avatar
      Anonymous
      Not applicable
      1 year ago

      Thanks for the reply from rohit1991 , please allow me to provide another insight:

      Hi, qmestu 
      Thanks for reaching out to the Microsoft fabric community forum.

      Regarding your need to merge data from multiple data sources into a unified dataset, it reminds me of the Dataflow product. Dataflows allow you to extract data from multiple different data sources and integrate this data into a unified data model. This is very useful for scenarios where you need to gather data from multiple systems for comprehensive analysis.

       

      Here is a screenshot of the relevant documentation:

      Dataflows can be configured with an automatic refresh schedule to ensure data is updated regularly. This means you can set the dataflow to automatically extract the latest data from the source systems at specific intervals, perform the necessary transformations, and load it accordingly. By pre-processing and transforming the data, dataflows can reduce report loading times and improve query performance. This is especially important for handling large datasets and complex calculations.

      For more details, please refer to:

      Configure and consume a dataflow - Power BI | Microsoft Learn
       

      However, if each of your visual objects only comes from a single web data source, you might also consider using the Dashboard product. Dashboards support visual objects from multiple reports:

      For more details, please refer to: 

      Basic concepts for designers in the Power BI service - Power BI | Microsoft Learn


      Of course, if you have any new discoveries or questions, please feel free to get in touch with us.
       

      Best Regards,

      Leroy Lu

      If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.