Skip to main content
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.

Reply
3ug3n
Regular Visitor

Power BI file optimisation based on many csv/xls daily exports

‎08-19-2023 08:09 AM

Hello everyone,

 

I'm contacting the group because after many research I can't find any satisfactory answer - with any certainty.

 

I have a Power BI file based on merging and analysing around 20 local folders containing CSVs and XLS files produced/gathered on a daily basis (~3000).

As can be expected, it takes a very long time to refresh the daily reports.

 

What do you recommend I do to optimise this - knowing that i cannot merge those files upstreams ?

 

Is there a way to create a table (internal or linked somewhere) to store the merged CSVs and then to:
- update it incrementally (even if it eventually bloats)
- do the ~30 daily analysis queries from it without having to recompile/filter/transform everything every time ?

 

Thank you very much in advance for your suggestions and possible links that detail/explain them !

 

Have a nice weekend

 

 

 

 

Labels:
Message 1 of 2
170 Views
0
1 REPLY 1
parry2k
Super User
Super User

‎08-19-2023 11:34 AM

@3ug3n this post has the answer to your question How you can incrementally refresh any Power BI data source (This example is a CSV File) - FourMoo | ...



Subscribe to the @PowerBIHowTo YT channel for an upcoming video on List and Record functions in Power Query!!

Learn Power BI and Fabric - subscribe to our YT channel - Click here: @PowerBIHowTo

If my solution proved useful, I'd be delighted to receive Kudos. When you put effort into asking a question, it's equally thoughtful to acknowledge and give Kudos to the individual who helped you solve the problem. It's a small gesture that shows appreciation and encouragement! ❤


Did I answer your question? Mark my post as a solution. Proud to be a Super User! Appreciate your Kudos 🙂
Feel free to email me with any of your BI needs.

Message 2 of 2
139 Views
0

Helpful resources

Announcements
RTI Forums Carousel3

New forum boards available in Real-Time Intelligence.

Ask questions in Eventhouse and KQL, Eventstream, and Reflex.

LearnSurvey

Fabric certifications survey

Certification feedback opportunity for the community.

View All