Monitor a dataset daily and generate another dataset with changes only from original dataset

[_Victor_]

Hello,

I'm wondering if anyone has any ideas how to approach the following task.
We have a dataset refreshed daily in workspace, with Read Only access. There are downstream applications relying on it. Each record in a dataset must be properly processed in downstream applications. The dataset is not that big, but the overhead is significant, and daily refresh is not feasible. However, Dataset changes are less than 1%, if we isolate delta changes, it becomes an easy task.
The question is how to build another dataset with original dataset with changes only. Unfortunately, there is no datetime stamp on record modification. If we clone a dataset once a week in our workspace we can compare it with daily snapshot, find differences to be processed by downstream apps. 
How if possible in terms of datasets, dataflow, API, M this can be achieved?

[lbendlin]

I can't believe I am saying that but you can self reference a dataset in Power Query.  Warning! Your head will spin from this.

Self Referencing Tables in Power Query 🔗- Excelerator BI

[_Victor_]

Thanks,lbendlin.
Would it be easier to export data into some other platform (SQL?) and import back results?

[lbendlin]

Where is the original data coming from? Can you do the change tracking there?

[_Victor_]

Unfortunately, we only have read access to dataset, upstream data source cannot changed or accessed. 
My guess is we need to start with making a clone dataset or copying into external storage. Is it REST API only way to go?

[lbendlin · Accepted solution]

The REST API is not a good way of doing that, it is mostly focused on managing the meta data.

I would go with PowerShell scripts running XMLA extracts.

[_Victor_]

Thanks for the advice, I'll look into it.