Incremental refresh and query folding with AWS redshift as data source

[rafaelmorenopbi]

Hi

I have a dataset importing data from a Data Warehouse. So far we operate in a full refresh mode, and I was willing to change to incremental refresh. However I read the documentation and it states that query folding is a pre-requisite for incremental refresh (makes sense), but so far I could not achieve query folding with AWS redshift as a data source. 

Has any of you made it? If so, what did you do in order to achieve success? 

I just want to implement incremental refresh for this particular dataset importing data from this specific data warehouse in redshift. 

Thanks in advance, 

[Anonymous]

Hi @rafaelmorenopbi ,

Most data sources that have the concept of a query language support query folding. These data sources can include relational databases, OData feeds (including SharePoint lists), Exchange, and Active Directory. However, data sources like flat files, blobs, and web typically do not.

In addition, in the Power Query Editor window, it is possible to determine when a Power Query query can be folded. In the Query Settings pane, when you right-click the last applied step, if the View Native Query option is enabled (not greyed out), then the query can be folded.

If the option is greyed out, maybe you could change another data source which supports query folding.

I have found a similar post, please refer to the following document to see if it helps you.

https://community.powerbi.com/t5/Service/Incremental-refresh-with-Amazon-Redshift/m-p/1925887/highli...

• use the ODBC connection

https://www.fourmoo.com/2019/08/20/using-azure-analysis-services-to-connect-via-an-odbc-source-redsh...

Best Regards

Community Support Team _ Polly

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[lbendlin]

"query folding is a pre-requisite for incremental refresh"

That is not accurate. Query folding is desired but not required. Incremental refresh can work with any data sources that allow for RangeStart/RangeEnd filters, regardless of query folding.