Power BI Data Flow Incremental refresh and Full refresh

[venkata-100887]

Hi,

I have unique problem, we have Power BI premium capacity P1, we have very large number of tables in each dataflow. Some records update every day such as new records added every day and some records updated or delete some times after a month or 3 months or six months and there are some records update or delete after a year. As we have large number of tables and some big tables, we want one full refresh once every day or week and incremental refresh every 2 hours for new records added every day. How can we design full load probably once a week and incremetnal refresh every two hours in Power BI dataFlow. As of now we implemeted every two hour incremental refresh on creted dttm field. Ther eson why I am sking for both full load and incremetnal refresh is we dont know when we will update a record (no time frame) and full load is consuming lot of resources is source system. We have source system nearly which have huge Gig memory and even it cannt take full load every two hours. So My question is how can we implement one full load and multiple incremental loads in Power BI data flow

Thank you,

Kartheek

[v-kelly-msft]

Hi @venkata-100887 ,

Full Refresh

By default, extracts are fully refreshed. That means that every time you refresh the extract, all of the rows are replaced with the data in the underlying data source. While this kind of refresh ensures you have an exact copy of what is in the underlying data source, it can sometimes take a long time and be expensive on the database depending on how big the extract is.If the extract is not set up for incremental extract, selecting to refresh the extract will fully refresh the extract. 

Incremental Refresh

Rather than refreshing the entire extract, you can set it up to only add the rows that are new since the last time you extracted data. For example, you may have a data source that is updated daily with new sales transactions. Rather than rebuild the entire extract each day, you can just add the new transactions that occurred that day. Then once a week you may want to do a full refresh just to be sure you have the most up to date data.

If you wanna do incremental refresh every 2 hours,set it as below:

And as for the full refresh,you can set like:

You can refer to the document below to get more information if you needed.

https://docs.microsoft.com/en-us/power-bi/service-premium-incremental-refresh

Best Regards,
Kelly

Did I answer your question? Mark my post as a solution!