Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Reply
debrauxjg
New Member

Import data from excel and append data into another table

Hi,

I'm receiving daily excel files with a table in which a field shows the extraction date of the data.
I want to keep track of all data and store all historical values into a data model to make some basic performance comparisons.

 

Excel file 1 would look like this
Test         Value     ExtractionDate

A313/02/2023
B813/02/2023
C613/02/2023
D913/02/2023

 

 

Excel file 2 would look like this (Values & extraction date will change the following day)
Test        Value      ExtractionDate

A314/02/2023
B914/02/2023
C1014/02/2023
D514/02/2023

 

I want to write a basic script to append Excel file 1 & 2 and 3 and 4 and...... N (each time a new file is posted on the Shared folder into a table so that I can have all data into one sole table and that the import process would be fluid.

 

Is there a simple solution to do this? Did try several methods including Table.InsertRows but I could not figure that out.

 

My script is (didn't work)

let
// Import Excel data from the source table
Source = Excel.Workbook(File.Contents("MyFile.xlsx"), null, true),
Table1 = Source{[Item="Table1",Kind="Table"]}[Data],

// Determine the maximum extraction date in the destination table
MaxExtractionDate = List.Max(Base[ExtractionDate]),

// Filter the source table based on the extraction date
FilteredTable1 = Table.SelectRows(Table1, each [ExtractionDate] > MaxExtractionDate),

// Append the filtered rows to the destination table
Import = Table.InsertRows(Base, 1, FilteredTable1)
in
Import

 

Can you please give me a hand?

3 REPLIES 3
Daryl-Lynch-Bzy
Resident Rockstar
Resident Rockstar

Hi @debrauxjg - the question I have - how is Excel erasing and replacing the data each day?  Ideally, Power BI/Power Query would connect to the original source rather than the Excel.

You could consider changing this daily process to take of copy the file/data before erasing and replacing so you have the daily backup for the history to import into Power Query while maintaining the current file process.  Power Automate can help with this scenario - copy the excel file then start the Power BI refresh.

Note - Power BI incremental load features (in the dataset or dataflow) could also help to read the current file and append to the previous loads.  While this is an option, I would hesitate to use because the Incremental Load will not be able access the history to reload the previous partitions should it break.  I have tried something like this in the past, but it breaks.  A good backup strategy is hard to implement.

Daryl-Lynch-Bzy
Resident Rockstar
Resident Rockstar

Hi @debrauxjg - there are many YouTube videoes with example.  Here is one Combining Multiple Files from a folder using Power Query in Excel or Power BI - YouTube

Many thanks @Daryl-Lynch-Bzy but it is slightly more complex than that.  There is only one excel file that is fed constantly with daily data and then erased the following day and replaced by today's data and so on. If I retrieve data from this file I'd see day 1 data. The following day day 2 data. And so non. But how to consolidate day 1 & 2 in the same table even if the file is changing every day? I was hoping to do that with a PowerQuery script.  

Helpful resources

Announcements
Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors