Automation through Msft Flow or other extensions

[GabAquino]

Hi all,

I'm new to using powerbi and wanted to know if there were ways of doing the following in an automated way through Msft Flow or other extensions to:

1. Append tables with similar columns
2. Remove similar rows from appended tables
3. Load / combine smaller CSV Files to a larger CSV file in Sharepoint with similar columns (To be done on a daily basis)

Challenges:

• When I choose "Get Data" > Sharepoint Folder, it shows me all the files in the folder, and it is difficult to find the most recent file if done on a daily basis
• I upload files manually, and append it to the larger CSV file through the Append Queries option, and choose "Remove Rows" > Remove Duplicates.
  • Is my data integrity affected when doing this?
  • Is there a faster way to do this?

Thank you all in advance!

[v-lili6-msft]

hi  @GabAquino 

You could sort the Date created or Date modified column to find the most recent file.

https://medium.com/@wkrzywiec/getting-data-from-the-latest-file-in-a-folder-using-power-query-51dfa4...

For "choose "Remove Rows" > Remove Duplicates", it will keep the first row value row for each duplicate value, this won't consider other columns.

Regards,

Lin