Forum Discussion

KhajaMohaddin's avatar
KhajaMohaddin
Regular Visitor
7 years ago

Power BI Aggregation table size limit

 

Need your help on the latest features of Power BI on Aggregations & Dual Storage Mode as we are facing some challenges. Providing the context and more details below here..

 

We are building a Power BI Dashboard sourcing from Databricks Delta tables and using the latest features of Power BI Aggregations by creating the Aggregation table with Dual Storage Mode options etc., We have a source data set table of size which is more than 5 GB and we have used the Direct Query Storage mode to pull the data from this table.  While creating the Aggregation table by eliminating few columns and aggregating the data at few required columns granularity, while applying the query changes we have come across the below error mentioning the Total size of serialized results of tasks (4.1 GB) is bigger than spark.drive.maxResultSize (4.0 GB).  

 

We understand that this is size limit of the Spark Driver to Import the data since the Aggregation table uses the Import Storage Mode.  Need your help & suggestions to resolve this issue ? If there are any alternatives / work arounds to support this type of volume data  please let us know ?

 

Would this issue could be resolved if we opt for the Power BI Premium Nodes (P1) with additional Capacity ?

Need Help

1 Reply

  • v-juanli-msft's avatar
    v-juanli-msft
    Community Support
    7 years ago

    Hi KhajaMohaddin 

    If you connect to Databricks Delta tables with Power BI Desktop and this error occurs when clicking "refresh" button in Power BI Desktop, updateing to Power BI Premium Nodes (P1) with additional Capacity would do nothing.

     

    I suggest you to create a support ticket (free for pro users) to get a more quick help.

    https://powerbi.microsoft.com/en-us/support/

     

    Best Regards
    Maggie