Append Daily from one csv file to master table - Need Power Query

[Anonymous]

HI Members,

I have one excel file in a folder which gets replaced everyday with that's day data .

I need to append everyday to the master table i built in power bi.

I used code like:

=table.combine(#"master table","daily data") 

but every time i refresh the dataset it replace prior day's combine data with current day data. 

for ex - when i ran the query yestreday it combined (master data+5/21/2020) data.

But, today when i ran it again it provides me combined(master data+5/22/2020) data and deleted 5/21/2020 data.

I have seen videos of incremental refresh but cudnt relate it to my issue. KIndly help me on this either with query or way to modify incremental refresh

[Greg_Deckler]

Right, so without incremental refresh, Power BI replaces all of the data in a table with the refreshed data. There are laborious work-a-rounds for this where you create a new query every day and add it to your table.combine and such, turning off data load for the last query. Another (better) work-a-round is to simply rename the file every day so that they build up in the folder and use a Folder query.

[Anonymous]

So, How do I achieve using incremental refresh? I know the basic steps but not able to relate it with my issue.

[v-juanli-msft]

Hi @Anonymous 

If you have problem, share with us.

https://blog.crossjoin.co.uk/2020/04/13/keep-the-existing-data-in-your-power-bi-dataset-and-add-new-data-to-it-using-incremental-refresh/

https://docs.microsoft.com/en-us/power-bi/admin/service-premium-incremental-refresh

https://docs.microsoft.com/en-us/power-bi/transform-model/service-dataflows-incremental-refresh

Best Regards

Maggie

[Anonymous]

My question remains the same .  How do i append data  to master table from a daily report which keeps replacing its entire data set daily. Everyday it replaces previous day data and paste current day data. I need to add everyday data to master table

[Anonymous]

Did you try using the Append Query in the power query editor?  

[v-juanli-msft]

Hi @Anonymous 

If the "daily data" from your source file changes every day, then it would update to newest data in power bi and remove the yesterday's data.

You can create a folder which you add everyday data into, then import the folder into power bi.

https://insightsoftware.com/blog/power-bi-load-data-from-folder/

After some transformation, you can append the daily data table with the master data table.

Best Regards

Maggie