Making changes to data model using incremental refresh

[meirlicht]

I am using incremental refresh for one of the tables in my model since that table is large and takes over 5 hours to refresh.

My concern with using incremental refresh is that now I can barely make any changes to the data, such as changes to the other tables in PowerQuery, editing measures, or adding calculated columns. This is important to me since I am continously updating and improving the report, displaying new and different information.

I understand I can store the incrementally refreshed table in one PBIX file and the report itself in another PBIX file so that I can still make changes to the data, but then the incrementally refreshed table needs to use a direct query connection which is too slow for me since there are nearly 100 milion rows in that table.

How can I go about this? Can I make all these changes in Tabular Editor? Thank you so much!

[Anonymous]

Hi @meirlicht,

I think the incremental refresh should not be suitable for this scenario. 

AFAIK, current power bi does not include any historical data feature. If your old records will be updated/replaced when new records added, I'd like to suggest you export these daily changed records to a new table and add a custom field to remark the operation timestamps.(change the edit operation to add new records with new timestamps)

After these steps, you can simply getting data from the new table to analysis the historical data changes.

Regards,

Xiaoxin Sheng