Forum Discussion
incremental refresh and partitionning
how partition will be created. Are they based on the creation_date available in my DWH ?yes
the first time i insert a new client in the dataset, all his data (from all fact table) will have the same creation_date. So all the initial data from one client will be on the same partition ?yes
the max update_date for that partition will change and i will refresh all the data from that client just because of one line change ?Not just from that client. The entire partition will be refreshed
Incremental Refresh expects immutable data. Your data isn't. That means you have additional maintenance effort to make sure there is no data duplication across partitions. That's just how it is.
how partition will be created. Are they based on the creation_date available in my DWH ?yes
the first time i insert a new client in the dataset, all his data (from all fact table) will have the same creation_date. So all the initial data from one client will be on the same partition ?yes
the max update_date for that partition will change and i will refresh all the data from that client just because of one line change ?Not just from that client. The entire partition will be refreshed
Incremental Refresh expects immutable data. Your data isn't. That means you have additional maintenance effort to make sure there is no data duplication across partitions. That's just how it is.
thanks for your answers. It is really weird from a tool like power bi that we can't handle properly incremential update.
You are mixing concepts. What you need is differential refresh. Power BI only offers incremental refresh. (Using the "detect data changes" option makes it worse, by the way. That creates a canary, pretty much doubling the storage cost.)