Forum Discussion

V55's avatar
V55
Icon for Helper I rankHelper I
7 years ago

Dataflow error for huge records - Evaluation resulted in a stack overflow

I have a table with lakhs of records. 'Time' is one of the fields. I have created a function fetchlatest to get the latest record based on 'time'. As you can see in the below code, the function basic...
Need Help
curth's avatar
curth
Icon for Power BI Team rankPower BI Team
7 years ago

Does the data file you shared represent the full data set or just a partial data set? Is it the same as the case which fails in Dataflows?

 

When you successfully load the data in Desktop, what's the maximum value reached for the commit size of the process Microsoft.Mashup.Container.NetFx45.exe? (You can see that in the Details tab of the Task Manager, but you'll probably need to right-click on the column header and select the "Commit size" column.)

  • V55's avatar
    V55
    Icon for Helper I rankHelper I
    7 years ago

    Hi curth,

     

    I tested with much larger records. This is what I noticed in my localmachine. The highlighted value is the commit size.

     

    Since it is not executing in dataflow  and giving stack overflow error, could it be related to dataflow capacity?

    I am using powerBI premium shared capacity. But unfortunately, I dont know the dataflow capacity set.

    • V55's avatar
      V55
      Icon for Helper I rankHelper I
      7 years ago

       

      Actually the issue seems to due to the heavy function (get latest function) on the table having lakhs of records.

      In the above post, I posted the commit size when I loaded single table in powerbi desktop. Now this image shows the commit size when I tried to do the same operation for three tables in powerbi desktop. And my CPU and memory hit the peak.

      I also confirmed the memory thrashing in premium exceeded 80%.

       

      Wondering if there is a better way to optimize get latest function?