Forum Discussion
Power BI Incremental refresh issue (query folding issue)
Hi
It worked a few month ago. Now my Power BI Incremental refresh got issue with Query folding. the native query is greyed out.
The datasource is from postgres views. Let me know if there is work around? thanks
12 Replies
- ShahRukhSameer
Continued Contributor
Hi yannaingoo,
Yes, there are a few things worth checking before assuming that PostgreSQL views are the problem.
The fact that it worked a few months ago and now the native query/folding option is greyed out makes me suspect something changed in the query or connector rather than incremental refresh itself.
For incremental refresh, the important part is that the RangeStart and RangeEnd filters can be pushed back to PostgreSQL. If Power Query performs some transformation before applying those filters, folding can be broken.
I'd check these first:
- Make sure RangeStart and RangeEnd are still parameters of type Date/DateTime and that the filter on your date column is applied as early as possible.
- In Power Query, check the steps before and after the date filter. Try right-clicking the step immediately after the RangeStart/RangeEnd filter and see whether "View Native Query" is available.
- If folding stops at a particular step, temporarily remove the transformations after that point and test again. Things like custom functions, Table.Buffer, certain merges, or some transformations can break folding.
- Since you're reading from PostgreSQL views, check whether the underlying view itself has changed. A view can still be queried successfully from Power BI while the additional filter isn't being pushed down as expected.
- Also check whether the PostgreSQL connector/version changed. An update to Power BI Desktop or the connector can sometimes change folding behaviour.
One thing I'd avoid is relying only on "View Native Query". Depending on the connector and the particular step, that option can be unavailable even though some folding is still taking place. The better question is whether the RangeStart/RangeEnd filter is actually being sent to PostgreSQL.
- FarhanJeelani
Super User
Hiyannaingoo ,
Have you changed anything??
can you share the m-query if you have applied any new steps? Like changing data type of date column Field??or check if ur range start and end parameter are working fine?
Please check below each steps and let me know:
Click each step in the Applied Steps pane and right-click → View Native Query. The first step where it greys out is the culprit.
Move the RangeStart/RangeEnd filter so it sits immediately after Source/Navigation, before any other transform.
Check the type on your filter column. If the Postgres column is date or timestamptz and the parameters are datetime, Power Query injects a conversion that kills folding. Fix it by casting inside the view (col::timestamp), not in Power Query.
Remove anything non-foldable ahead of the filter: custom columns, Table.Buffer, merges, Date.From / DateTime.Datewrappers, index columns.
If the source uses a hand-written SQL statement, folding is off by default. Switch to Value.NativeQuery(Source, "select ...", null, [EnableFolding=true]).
If nothing changed in the model, update Power BI Desktop and the on-prem gateway — Npgsql driver changes have broken folding on Postgres before, and a gateway/Desktop version mismatch is a common cause of "it worked months ago."
Please mark this as solution if it helps you. Appreciate Kudos.
- yannaingoo
Advocate II
Moved the RangeStart/RangeEnd before data type changes. Then I can see the view native query
but still not working.
Select xxx,xxx,xx from "public"."TD_View" "_" where "_"."upload_date" >= timestamp '2024-12-31 00:00:00' and "_"."upload_date" < timestamp '2026-12-31 00:00:00'
This is the Mquery
Source = PostgreSQL.Database("xx.xx.xx.xx :xxxx", "TD"),
public_TD_View = Source{[Schema="public",Item="TD_View"]}[Data],
#"Filtered Rows" = Table.SelectRows(public_TD_View, each [upload_date] >= RangeStart and [upload_date] < RangeEnd),
#"Changed Type" = Table.TransformColumnTypes(#"Filtered Rows",{{"Date Open", type datetime} etc
- v-saisrao-msft
Community Support
Hi yannaingoo,
The Native Query now shows the RangeStart/RangeEnd predicates, so there is no need to move the filter step again. The next step is to use Query Diagnostics or source tracing to verify that the actual queries sent to PostgreSQL during the Service refresh contain the appropriate RangeStart/RangeEnd filters for each partition. Also verify that upload_date, RangeStart, and RangeEnd use compatible Date/Time data types. If the traced queries contain the expected filters, further investigate the incremental refresh configuration and refresh behavior in the Power BI Service.
Understand Folding by using Query Diagnostics - Power Query | Microsoft Learn
Troubleshoot incremental refresh and real-time data - Power BI | Microsoft Learn
Thank you.
- krishnakanth240
Super User
Hi yannaingoo
You can check where folding breaks from "View Native Query" after each step from source and moving forward and folding stops at non foldable transformation. If it is due to custom column, merged column, changed data type then that logic you can apply in the data source in Postgres view instead of Power Query.
You can verify RangeStart and RangeEnd filter is applied after source step and its data type matches column being filtered, a mismatch will break folding
- ShivekMaharaj
Memorable Member
Hi yannaingoo,
krishnakanth240's step-by-step check is a good place to start.
One thing I would add is that for incremental refresh, the important question is not only whether View Native Query is enabled, but whether the RangeStart and RangeEnd filter is actually being pushed down to PostgreSQL.
Microsoft's incremental refresh guidance recommends filtering the source with RangeStart and RangeEnd, and notes that a slow or process-intensive load after applying that filter usually indicates the query is no longer folding.
Since you are using PostgreSQL, Microsoft's PostgreSQL connector documentation confirms that folding is supported by default for foldable operations.
I would therefore check the applied steps until you find where folding stops, but I would also run Query Diagnostics and inspect the Data Source Query being sent to PostgreSQL. If the generated SQL does not contain the expected date predicate, that confirms the incremental filter is being evaluated outside the source.
Also verify that your filter follows the usual pattern of one inclusive and one exclusive boundary, for example:
[DateColumn] >= RangeStart and [DateColumn] < RangeEndIf this worked previously, I would also check whether the PostgreSQL view definition or any Power Query transformation changed since then before assuming it is a Power BI regression.
AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.
- Praful_Potphode
Super User
Hi yannaingoo
Please try suggestions by ShivekMaharaj and krishnakanth240 .
your query folding can also break because of view complexity.
Please give kudos or mark it as solution once confirmed.
Regards,
Praful
- yannaingoo
Advocate II
but i need to Join physical tables and show it in 1 view.
- Praful_Potphode
Super User
try to analyze view and identify possibility of optimizations.
if its still breaking incremental refresh, then best way is to bring physical tables in PBI and do incremental refresh.
Please give kudos or mark it as solution once confirmed.
Thanks and Regards,
Praful
- v-saisrao-msft
Community Support
Hi yannaingoo,
Checking in to see if your issue has been resolved. let us know if you still need any assistance.
Thank you.
- v-saisrao-msft
Community Support
HI yannaingoo,
Checking in to see if your issue has been resolved. let us know if you still need any assistance.
Thank you.
- yannaingoo
Advocate II
I tried to make work around and its work and range start/end accordingly in the M query editor.
if I removed the "Change Type" which is transformation part such as data type changing in my case, then it worked. However need this to be work. How shall I tackle this to include and query folding work together ?
Source = PostgreSQL.Database("xx.xx.xx.xx:xxxx", "TD"),
public_TD_View = Source{[Schema="public",Item="TD_View"]}[Data],
#"Sorted Rows" = Table.Sort(public_TD_View,{{"upload_date", Order.Descending}}),
#"Filtered Rows" = Table.SelectRows(#"Sorted Rows", each [upload_date] >= RangeStart and [upload_date] < RangeEnd),
#"Changed Type" = xxxxxx