Forum Discussion

matkvaid's avatar
matkvaid
Helper III
2 years ago

Dataflow gen2 high CU usage

One of my gen2 dataflows uses relatively too much (i think) CU's for what it does: I take 3 tables from warehouse, check max date and filter each of another 3 tables from on prem sql server source b...
Dataflows
matkvaid's avatar
matkvaid
Helper III
2 years ago

I am amazed, how it can be bad like that. I had another ticket open, in two weeks i found how to solve the problem myself. In that time support just asked same questions in emails and teams calls. I had another issue with UTC times and on prem data gateway. It was a bug, i was ignored in support, here and in official reddit sub, no answers were given... I am happy that the bug was fixed, but - i had made workarounds for my reports to work, and the fix broke them. I got questions from report users, why the reports do not work AGAIN... In my opinion, Fabric is still far from being ready for production...

JustinWesty's avatar
JustinWesty
Frequent Visitor
2 years ago

Hi Matkvaid,

What I have found on random occasions is that when I have a comment line in my M code, the query doesn't fold (look at your process volume in the output; I would assume the volume is unfolded). So, I downloaded a template of my original and removed all comments.

 

example 

#"Filtered rows 1" = Table.SelectRows(#"Removed other columns", each [entryNo] > _maxEntryNo),
//#"Filtered rows 1" = Table.SelectRows(#"Removed other columns", each [entryNo] > 0)
Fix
#"Filtered rows 1" = Table.SelectRows(#"Removed other columns", each [entryNo] > _maxEntryNo),