Forum Discussion
Json Incremental Refresh
I understand.
But even if I do a refresh over a year, I could never guarantee that my data is perfect.
I take my previous example. If I sort my data by modification date and remove duplicates (step after the custom filter on RangeStart/RangeEnd in my query). Can this work?
That might work but it will become very expensive very quickly. You would have to sort the entire data source in descending order before applying the "Remove duplicates" transform
It will be easier to run a full refresh every once in a while, even if done by partition.
unfortunately, that doesn't work.
I think only new data is considered in the power Query. I tested by ordering and then removing the duplicates, but the duplicates remain.
Power bi considers that its archive is never modified.