Forum Discussion

askme1217's avatar
askme1217
Frequent Visitor
1 year ago
Solved

help : dataset refresh issue

i have a dataset around 5 GB size. when i refresh it, it is getting failed with memory usage error. it uses around 50 GB of memory which is maximum in capacity. Why is it using 50 gb of memory even though the dataset size is 5 Gb only? The fact table has around 200 million rows, it does not have any heavy powerquery transformations, just  native sql queries with filters. How can i reduce memory usage and refresh it without error?

Need Help

8 Replies

    • askme1217's avatar
      askme1217
      Frequent Visitor
      1 year ago

      Yes, when i check the 'View Native Query' it is enabled. so i think query folding happening. 

    • v-sshirivolu's avatar
      v-sshirivolu
      Community Support
      1 year ago

      Hi askme1217 ,
      Thanks for reaching out to the Microsoft fabric community forum.

      With large datasets, Power BI can use much more memory than the dataset size during refresh. Setting up incremental refresh and removing unused columns or large tables can help reduce memory use. If issues persist, consider upgrading your capacity.

      Power BI Incremental Refresh Documentation - Incremental refresh and real-time data in Power BI
      Optimization Techniques for Import Models (VertiPaq Engine )-
      Data reduction techniques for Import modeling - Power BI | Microsoft Learn
      Power BI Premium Capacity and Limits - What is Power BI Premium? - Power BI | Microsoft Learn

      If the response has addressed your query, please Accept it as a solution and give a 'Kudos' so other members can easily find it

      Best Regards,
      Sreeteja.
      Community Support Team

      • v-sshirivolu's avatar
        v-sshirivolu
        Community Support
        1 year ago

        Hi askme1217 ,

        Just wanted to check if you had the opportunity to review the suggestion provided?

        If the response has addressed your query, please Accept it as a solution and give a 'Kudos' so other members can easily find it.

         

        Thank you.

         

  • pallavi_r's avatar
    pallavi_r
    Super User
    1 year ago

    Hi askme1217 ,

     

    Are you using calculated columns or calculated dax tables? Can you try refreshing dataset after removing calculated columns and tables. Calculated column takes memory depending on power query transformation.

     

    Hope you have taken care of unused columns, incremental refresh and inefficient relational modelling (exp. many-many).

     

    Thanks,

    Pallavi