Import multiple parquet files but with partial updates

[P_Stavis]

Hello!

I extract data from SAP each month, generating multiple .xlsx files (one per month). To optimize the size of the database, I decided to convert each of these monthly files into separate .parquet files.

However, when I load these files into Power BI, every time I hit "refresh," it reloads all the files, which demands a significant amount of processing power. Is there a way to configure some files as static and others as dynamic, so only the dynamic files are refreshed?

My initial approach was to create two separate folders—one for static files and one for dynamic files—allowing me to configure different refresh settings for each. However, I have not been able to make this setup work yet.

[lbendlin · Accepted solution]

The only way to avoid a full load is to use incremental refresh with a foldable source.  Doesn't seem to be an option in your scenario.

Having the data in Parquet format should already give you the highest possible performance. If you put the Parquet files into a data lake you could also consider using the DirectLake connection (where the data is connected live).  Performance will be slightly inferior to import mode.