Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
7 years ago

Azure Data Bricks Data Refresh

I have a report utilizing data import from a persisted table in Databricks. Once the dataset size increased I received the following error: Total size of serialized results of 17 tasks (4.1 GB) is b...
Dataflow
Need Help
Anonymous's avatar
Anonymous
Not applicable
7 years ago

Hi Anonymous ,

 

I think this issue should more related to database settings. it sounds like response data amount is greater than default cache size so refresh requests has been blocked/canceled.

 

Maybe you can take a look at following link to know more about this issue:

Total size of serialized results of 16 tasks (1048.5 MB) is bigger than spark.driver.maxResultSize (1024.0 MB)

Spark Configuration

 

For Power BI Architecture, you can refer to below link:

Power BI Security

 

Regards,

Xiaoxin Sheng

Anonymous's avatar
Anonymous
Not applicable
7 years ago

Hi Xiaoxin,

 

Thanks for looking at this issue. As an update we increased teh variable size to 35gb on both the clusters we are running, but still encounter the same 4gb error when attempting a refresh. We have a ticket in with the dev team to ascertain whether the error is being thrown by our Databricks instance (meaning we missed something in the adjustment of the variable) or whether it's occurring in the attempt to write to our premium capacity storage.

 

Julius

  • pbiusrwus's avatar
    pbiusrwus
    Microsoft Employee
    7 years ago

    Did you manage to figure this out? I am getting the same error.

  • Anonymous's avatar
    Anonymous
    Not applicable
    7 years ago

    Hi Anonymous ,

    I'd like to suggest you open a support ticket to get better support from dev team, I think this issue is more related to spark itself.

    submit a support ticket

     

    Regards,
    Xiaoxin Sheng

    • Anonymous's avatar
      Anonymous
      Not applicable
      7 years ago

      Thanks Xiaoxin , a ticket is currently in but I have not heard back and need to follow up on it. The changes have been made in spark so I need to confirm with Microsoft support that the issue is not related to the Hive metastore which holds the uploaded PBIX files. 

       

      I'll make sure to post their resolution/recommendation once I'm able to get that back on a call and get it sorted out.