Forum Discussion

BigLo's avatar
BigLo
Icon for Helper III rankHelper III
4 years ago

Json Incremental Refresh

Hello, I've a webservice to import a json file. This json contains a lot of data and I'd like to not have to import the whole json anymore. So I added a parameter to my service to filter the dat...
Need Help
Show and Tell
lbendlin's avatar
lbendlin
Icon for Super User rankSuper User
4 years ago

That might work but it will become very expensive very quickly. You would have to sort the entire data source in descending order before applying the "Remove duplicates" transform 

 

It will be easier to run a full refresh every once in a while, even if done by partition.

BigLo's avatar
BigLo
Icon for Helper III rankHelper III
4 years ago

unfortunately, that doesn't work.
I think only new data is considered in the power Query. I tested by ordering and then removing the duplicates, but the duplicates remain.
Power bi considers that its archive is never modified.