Daily data refresh from the same file

[mcali]

Hello everyone and thank you in advance for your help!

I need to do a daily data refresh from a file and I don't understand how to do it, here is the situation:

I have a folder named "IN" in which everyday is overwritten the same csv file with yesterday data. Before beeing overwritten the file is copied into a BCK folder with da date append to the file name.

I woud like to create a query that read the data from the csv and append the data to the one read the day before.

So for exampe today I have:

IN --> data.csv (contains data about 14/01/2021)

tomorrow I'll have:

IN --> data.csv (contains data about 15/01/2021)

How can I have a single table in powerbi that contains both data and that can be refreshed everyday appending the new data?

Thank you!

[v-jingzhang · Accepted solution]

Hi @mcali 

First create two separate queries to get today's file and yesterday's file, then use Append Queries feature to append both data into one query.

As yesterday's file name contains date and you hope to refresh the data automatically, you can use parameter to get the date automatically and replace the date part in file name. To realize this goal, please follow below steps.

At last, append two queries as shown in the first image.

Additionally, if you want to combine today's data and all the data in BCK folder, you can use Folder connector to combine all the files in BCK folder, then append today's data to that.

Kindly let me know if this helps.
Community Support Team _ Jing Zhang
If this post helps, please consider Accept it as the solution to help other members find it.

[amitchandak]

@mcali , refer to this DAX append technique

https://blog.crossjoin.co.uk/2020/04/13/keep-the-existing-data-in-your-power-bi-dataset-and-add-new-...
https://www.thebiccountant.com/2017/01/11/incremental-load-in-powerbi-using-dax-union/