Forum Discussion
Appending To Large Static Data Set
Hi Kfausch
I think what you have done till step 2 is correct. Then try using Append on table 2 where enable load should be enabled (this refers to your table that would be visible in the model or not). Just come to table 2 with Refresh disabled (static data) and then do append table 1 to this table. I believe it should work.
Also, you can avoid this by using incremental refresh as well, where probably you can store the whole data and then just refresh the latest data (1-10 days to months, quarters or years as per your requirement).
Hi Kfausch
I think what you have done till step 2 is correct. Then try using Append on table 2 where enable load should be enabled (this refers to your table that would be visible in the model or not). Just come to table 2 with Refresh disabled (static data) and then do append table 1 to this table. I believe it should work.
Also, you can avoid this by using incremental refresh as well, where probably you can store the whole data and then just refresh the latest data (1-10 days to months, quarters or years as per your requirement).
Appending the current data to the history table did reduce the refresh time further. Thanks for your recommendation!
Kevin
Kfausch How did you load data to Table 2 and publish it if its size is 8.7 GB?
I have a similar issue, however my historical data is too big to load and then publish to Power BI Service. I need to use XMLA endpoints to load the data using SQL Serve Management Studios Analysis Services connection.... which requires the table to be loaded before I publish it in order for me to access it via XMLA endpoints ( am I mistaken?).
I do not know how to disable the load of a table using XMLA endpoints or other tools, or if its even possible, so I now have a historical table that's loaded ... but only used for appending with current data which is not ideal.
Please let me know if you have any advice Kfausch , and any input you have AnkitKukreja would be welcome as well.
