Power BI report is showing duplicate records using incremental refresh

[prasad08mm]

Hello All,

I am getting duplicate records using incremental refresh. 

Following observations, analysis has been done - 

1. Without incremental refresh i.e full load data is showing perfectly.

2. Incremental policy setting is correctly defined i.e. Range Start and Range End 

3. After the incremental policy is set, the first load is shown correct records and after a few delta schedules, the records show duplicates, what I feel is that the changed records are not updating into their respective partition tables. 

Is there any way to check partition data, as the partition is done by power bi?

Please guide.

[miiihiir]

@prasad08mm is this issue solved? I'm also facing the same issue

[prasad08mm]

Anybody please respond? and guide

[Tutu_in_YYC]

Hey Prasad,
Can you check your RangeStart and RangeEnd filters?

For RangeStart : is after and equal to
For RangeEnd : is before

[prasad08mm]

Hello Tutu_in_YYC

Yes the setting is same you mentioned. BUt still the same issue 

[Tutu_in_YYC]

You can use SSMS to check the partition. Check this tutorial out.

https://youtu.be/QQegWCeRako?t=321

[ibarrau]

Hi. There. Do you have the detect data changes option on? There is a way to implement incremental refresh that will replace the last N period everytime, but there is another way to configure that will only add rows when data is changed. In that last scenario you really need to understand how it works to avoid duplications. You Measures will need to look for the last value of each duplicated row.

Send us the configuration picture or try turning that option off unless you really have a requirement for it and we might help with the understanding.

I hope that helps,

[prasad08mm]

Could you please elaborate more on this " There is a way to implement incremental refresh that will replace the last N period everytime, but there is another way to configure that will only add rows when data is changed"

My current setting of Incremental Poilcy 

[ibarrau]

Be sure to start over once you changed "detect data changes" to OFF. Refresh all over again before starting incremental refreshes.

Regards

[prasad08mm]

Hello ibarrau, I did that many  a times. still same issue (duplicates)

[prasad08mm]

Thank you for your reply.

Early yes,  detect date change was ON but now we have turned off the option. But for both scenarios, we had found the duplicates.