Dataflow Enhanced Engine is On. Why folding doesn't work?

[olegkazanskyi]

My dataflow enhanced engine is On.

From the documentation and my tests, it should work as a SQL server with folding options.

I download the data to the desktop with filter as a first step, but it still downloads all the data as with regular PowerQuery mode (the total Gb is the same).

Does anyone know the reason and how to fix it?

[bcdobbs · Accepted solution]

Which option are you using in Power Bi desktop? I think I read that it will only fold if you use the newer Dataflows connector rather than the original PowerBI Dataflows connector.

[Anonymous · Accepted solution]

Hi @olegkazanskyi 

Dataflow Enhanced Engine and Query folding are not related .

Dataflow Enhanced Engine : Reduce refresh times for long-running ETLs and facilitates direct queries on entities .

Query folding : Query folding is the ability for a Power Query query to generate a single query statement to retrieve and transform source data.

As you said that you have a simple filter function in the dataset's query to extract the last 3 months data , but the amount of data pulled is as big as the full 5 years of transactions . Maybe you can set the filter in Desktop or data source to see if the returned data is filtered by you .

Best Regard

Community Support Team _ Ailsa Tao

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[bcdobbs · Accepted solution]

Which option are you using in Power Bi desktop? I think I read that it will only fold if you use the newer Dataflows connector rather than the original PowerBI Dataflows connector.

[olegkazanskyi]

Using a simple Dataflow connector helped!

Thank you very much!

[lbendlin]

one has nothing to do with the other.  Query folding depends on your data source and the transforms you conduct in Power Query.  For example a seemingly innocent transform like Text.Upper() will completely prevent Query Folding.

[olegkazanskyi]

@lbendlin, 

I have a simple filter function in the dataset's query.

I try to extract the last 3 months, but the amount of data pulled is as big as the full 5 years of transactions.

[lbendlin]

Before moving this to a dataflow, you can test it in Power BI Desktop (ie in Power Query on the desktop).  There you can run query diagnostics to see if and when query folding breaks, and what the queries are that query folding is producing. Those might be suboptimal too.

You can then do the same in your dataflow - right click on each step and select "View datasource query"  and "view query plan".