Append Daily from one csv file to master table - Need Power Query

Question

HI Members,I have one excel file in a folder which gets replaced everyday with that's day data .I need to append everyday to the master table i built in power bi.I used code like:=table.combine(#"master table","daily data")&nbsp;but every time i refresh the dataset it replace prior day's combine data with current day data.&nbsp;for ex - when i ran the query yestreday it combined (master data+5/21/2020) data.But, today when i ran it again it provides me combined(master data+5/22/2020) data and deleted 5/21/2020 data.I have seen videos of incremental refresh but cudnt relate it to my issue. KIndly help me on this either with query or way to modify incremental refresh&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;

greg_deckler · Answer

Right, so without incremental refresh, Power BI replaces all of the data in a table with the refreshed data. There are laborious work-a-rounds for this where you create a new query every day and add it to your table.combine and such, turning off data load for the last query. Another (better) work-a-round is to simply rename the file every day so that they build up in the folder and use a Folder query.

anonymous · Answer

So, How do I achieve using incremental refresh? I know the basic steps but not able to relate it with my issue.

v-juanli-msft · Answer

Hi&nbsp;Anonymous&nbsp;
If you have problem, share with us.
https://blog.crossjoin.co.uk/2020/04/13/keep-the-existing-data-in-your-power-bi-dataset-and-add-new-data-to-it-using-incremental-refresh/
https://docs.microsoft.com/en-us/power-bi/admin/service-premium-incremental-refresh
https://docs.microsoft.com/en-us/power-bi/transform-model/service-dataflows-incremental-refresh
&nbsp;
Best Regards
Maggie

anonymous · Answer

My question remains the same .&nbsp; How do i append data&nbsp; to master table from a daily report which keeps replacing its entire data set daily. Everyday it replaces previous day data and paste current day data. I need to add everyday data to master table

v-juanli-msft · Answer

Hi&nbsp;Anonymous&nbsp;
If the "daily data" from your source file changes every day, then it would update to newest data in power bi and remove the yesterday's data.
You can create a folder which you add everyday data into, then import the folder into power bi.
https://insightsoftware.com/blog/power-bi-load-data-from-folder/
After some transformation, you can append the daily data table with the master data table.
&nbsp;
Best Regards
Maggie

Forum Discussion

Append Daily from one csv file to master table - Need Power Query

6 Replies

Related content

Dataflow - Append

Appending Data

Storage increasing daily

Daily Append Datasets Refresh Strategy

Master Data Service in Fabric ?