Forum Discussion

ganaa0810's avatar
ganaa0810
Regular Visitor
8 months ago
Solved

Large datasets

Hello everyone, Im new on powerbi using pro license and dealing with large dataset for me.  I got error after publishing to service because of initial huge data load.  I configured incremental refre...
Need Help
Tips and Tricks
  • v-sgandrathi's avatar
    v-sgandrathi
    8 months ago

    Hi ganaa0810,

     

    The main challenge isn't the size of your PBIX file on desktop, but rather the full refresh that occurs after every model change in the Power BI Service. Even though your PBIX is only 230 MB locally, publishing a model with incremental refresh enabled causes the Service to rebuild the entire dataset once, including all historical data. This process can exceed Pro capacity limits or result in connection timeouts since the Service tries to load two years of data at once. This is expected behavior for Pro: after any model change, incremental refresh can't reuse previous partitions and must reprocess everything. Premium/PPU offers more capacity, faster refreshes, and advanced features like loading data without reprocessing all history. Your approach is correct, but your model has reached a size where Pro struggles during the initial full refresh. On Power BI Pro, you can temporarily reduce the refresh window (such as loading only 6 months), publish, and then expand the window. For a more stable solution, consider moving to PPU or Premium for better handling of large historical data loads.

     

    Thank you.

grazitti_sapna's avatar
grazitti_sapna
Super User
8 months ago

Hi ganaa0810 

 

You can try the following and let us know if the issue still persist:

  • Double-checked incremental refresh policy (RangeStart/RangeEnd on each big table).
  • Reduced columns and pre-aggregated facts.
  • Temporarily limited refresh to latest 6 months for first publish.
  • Increased source DB timeout settings.
  • Monitored refresh history after publish for slow tables.

Full model refresh after schema change with large historic data can trigger timeouts breaking initial load into smaller windows and publishing from Desktop first is key.

 

🌟 I hope this solution helps you unlock your Power BI potential! If you found it helpful, click 'Mark as Solution' to guide others toward the answers they need.

💡 Love the effort? Drop the kudos! Your appreciation fuels community spirit and innovation.

🎖 As a proud SuperUser and Microsoft Partner, we’re here to empower your data journey and the Power BI Community at large.

🔗 Curious to explore more? [Discover here].

Let’s keep building smarter solutions together!