Forum Discussion

rmcgrath's avatar
rmcgrath
Advocate II
1 year ago
Solved

Too many rows?

Hello - I am trying to pull in a General Ledger Entries table via and OData URL from Business Central.  Unfortunately, it has 10,175,525 rows!  Power BI actually pulled it in, but when I tried to do transformations and then Close & Load, that is where it failed.  But I'm not sure if that is due to the source?

 

Any advice on how to handle a table of this size is GREATLY appreciated!

Need Help
Show and Tell
Tips and Tricks
  • Anonymous's avatar
    Anonymous
    1 year ago

    Hi, rmcgrath 

     

    Here are a few strategies you can consider to manage the data more effectively:

     

    DirectQuery Mode: If possible, consider using DirectQuery mode instead of importing the data. This allows Power BI to query the data directly from the source without loading it into memory, which can be beneficial for very large datasets. DirectQuery in Power BI - Power BI | Microsoft Learn

     

    Incremental Refresh: Instead of loading the entire dataset at once, you can set up incremental refresh in Power BI. This allows you to load only new or changed data, which can significantly reduce the load time and improve performance. Incremental refresh for semantic models in Power BI - Power BI | Microsoft Learn

     

    Dataflows: Use Power BI Dataflows to preprocess and transform the data before loading it into Power BI. This can help in offloading some of the transformation work and improve performance. Introduction to dataflows and self-service data prep - Power BI | Microsoft Learn

     

    Best Regards,

    Community Support Team _Charlotte

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

2 Replies