Forum Discussion
Custom query - Incremental Refresh
- Anonymous3 years ago
Hi Chris.
I finally made it work. Thanks to nexus150 (Ricardo Rincón) and you Chris for all your help.
My issue was that I configured wrong the incremental refresh. I selected an archive data starting longer than rows in my custom table. Let me explain me better:
- I have 9 rows (months intervals) in my custom table but selected 60 months to archive data. So, PollingExpression was looking for months that didn't exist. Changing archive data to 8 rows solved the issue.Also, in my case, the refreshBookmark property was created just after the first reload with applyRefreshPolicy: true, never with false as were the case at your blog.
Finally, I test some different scenarios just to be sure how it work this method and these are my takeaways:
Test #1: using date field with data in just one partition and loading everything from SSMS.
Result: successful, works perfectly.Test #2: Using DetectDataChangesQuery as a scalar, not table.
Result: successful, works perfectly.Test #3: Test with DetectDataChangesQuery as a table loaded in the model.
Result: successful, works perfectly.Test #4: Test without initial reload to applyRefreshPolicy: false.
Result: successful, works perfectly.
Thanks again Chris for your kindness and your patience trying to help me.
Yes, but what SQL queries do you see running on Snowflake when you try to refresh? That will tell you what might be going wrong.
Chris, I pasted the Snowflake query in the previous message.
It's this one:
This one is the query that Snowflake execute when I run the full refresh TSML script. Or do you mean other query? I'm a little bit lost here.