Forum Discussion
Query Folding and Access Native Query for Incremental Refresh
That is what Query Folding IS! It pushes data transformation steps (like filtering) and many others down to the source , letting those operations be handled by the server. The result is that the server is probably better equipped to handle the steps natively, and it hopefully results in fewer rows returned by the query, and less processing overhead by the client (Power BI).
Below is a very simple example of query folding on my Customer table for customers in the city of Buffalo:
Let the query folding happen naturally. Don't try to force it. If your source is capable (things like Excel and flat files are NOT capable) then it just happens automagically.
Hope that helps.
Gotcha. Well, some of my queries have some pretty complex transformations. I'd like to keep these transformations as is in the SQL. However, I want to add incremental refreshing to my table which requires query folding to be enabled. I was hoping to just add a filter for the Date range ontop of my original SQL and have this filter be equal to parameters labeled "RangeStart" and "RangeEnd".
>> some of my queries have some pretty complex transformations<<
How complex? I don't know about Teradata, but SQL Server can handle a lot of the transformations listed, even GROUPING and PIVOT/UNPIVOT. I chokes on things like splitting a field based on the instance of a delimiting character, or things like FILL DOWN.
If you absolutely MUST have your custom transformations and they preclude query folding in Power BI, then the best bet is to wrap all that in a view in the source database and simply reference the view instead of a base table.
But Power BI needs A) Query Folding and B) appropriate date column in order to support Incremental Refresh.
Hi, I was reading this and I wonder In case of DB like AS400/DB2 which doesnt support Value.NativeQuery, what if we pass Parameters (Start & End) as strings to native sql (complex) to be used in where clause. For sure the the model will refresh, but would it be able to create proper partition and proper incrmenatal setup?
RS = "'" & DateTime.ToText(RangeStart, "yyyy-MM-dd HH:mm:ss") & "'",
RE = "'" & DateTime.ToText(RangeEnd, "yyyy-MM-dd HH:mm:ss") & "'",
and inside native query we apply:
WHERE T.SNAP_DT >= " & RS & " AND T.SNAP_DT < " & RE & "
