Forum Discussion
Incremental refresh
Hello my_patil ,
If you company use Microsoft products, you can use Azure SQL Database.
And for python script you can use Azure Functions. It runs on a schedule and it is serverless (no virtual machine to manage).
After writing to Azure SQL Database, you can apply incremental refresh policies.
In Azure SQL DB, create table and add updated_at and write_date columns.
In python script, you can use xmlrpc.client library.
Hope it gives you an idea.
Hello monikapatil2612 ,
Incremental refresh requires a data source that supports query folding. However Python data sources never fold, so Power BI cannot push date filters to Odoo. It must re-download the entire dataset and this causes exceeding API limits.
My recommendation is using a staging database. Can you create a python script to load data into any Database of your company use?
Incremental refresh requires a data source that supports query foldingNot entirely accurate. It prefers that the query folds, but it does not strictly require it. Non-folding sources will not provide any benefit to the source and network, but this will not impact the partition management.
If I do not remember wrong, I read something like this 1.5-2 years ago. When I check documentation now, yes, non-folding queries are acceptable but not recommended.
However, query folding for life 🙂
If I do not remember wrong, I read something like this 1.5-2 years ago. When I check documentation now, yes, non-folding queries are acceptable but not recommended.
However, query folding for life 🙂
Yes I can pyhton script to load data to database.Can you guide me on which database should I select and what is best option among different cloud to store python script and databse ?
Hello my_patil ,
If you company use Microsoft products, you can use Azure SQL Database.
And for python script you can use Azure Functions. It runs on a schedule and it is serverless (no virtual machine to manage).
After writing to Azure SQL Database, you can apply incremental refresh policies.
In Azure SQL DB, create table and add updated_at and write_date columns.
In python script, you can use xmlrpc.client library.
Hope it gives you an idea.
Thank you very much for your help! Your guidence is really useful to me.
I have proposed as you said to manager. I am waiting for his response on this.
