Forum Discussion
Incremental refresh
Hello my_patil ,
If you company use Microsoft products, you can use Azure SQL Database.
And for python script you can use Azure Functions. It runs on a schedule and it is serverless (no virtual machine to manage).
After writing to Azure SQL Database, you can apply incremental refresh policies.
In Azure SQL DB, create table and add updated_at and write_date columns.
In python script, you can use xmlrpc.client library.
Hope it gives you an idea.
Incremental refresh requires a data source that supports query foldingNot entirely accurate. It prefers that the query folds, but it does not strictly require it. Non-folding sources will not provide any benefit to the source and network, but this will not impact the partition management.
If I do not remember wrong, I read something like this 1.5-2 years ago. When I check documentation now, yes, non-folding queries are acceptable but not recommended.
However, query folding for life 🙂