Forum Discussion
Appending To Large Static Data Set
Hi Kfausch
I think what you have done till step 2 is correct. Then try using Append on table 2 where enable load should be enabled (this refers to your table that would be visible in the model or not). Just come to table 2 with Refresh disabled (static data) and then do append table 1 to this table. I believe it should work.
Also, you can avoid this by using incremental refresh as well, where probably you can store the whole data and then just refresh the latest data (1-10 days to months, quarters or years as per your requirement).
I have been wrestling with old and new (fresh) data for some time now. I refresh the data for the last year every day, and I want the previous ones to be static tables (not to be refreshed daily).
So far, I have a pro license and am not in a Fabbric environment, I have discovered two ways.
#1 One is using DAX UNION with the fact that for previous years I have turned off refresh for those queries (include in report refresh) - it will be automatically refreshed the first time.
#2 The second is using Dataflow (Gen1) which does not refresh except the first time.
It's probably different in Fabric, but fabric is very expensive, but we'll get to that.