Forum Discussion

MrBirch's avatar
MrBirch
New Member
4 years ago

Append new query data to static data

Hi Team,

 

Applogies in advance data is not my first language.

 

I've got a problem, I need to manually refresh a modest dataset (10-15M rows) every day and publish to a Workspace for others to consume.

I can only connect to my source using "import". This means I can't use any of the incremental refresh options or schedule updates.

 

The data set that I connect to only changes 10% each day, I can clearly identify which lines need to be updated. to speed things up I've tried to create 2 versions of the query, 1 filters out the static data and the other filters out the dynamic data. I've tried to set the set the dynamic query to Update on Refresh and then append the two queries.  When I've tired this it seems to try and update the static data query as well. so I don't save any time.

 

Any ideas?  

thanks in advance.

 

2 Replies