Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Be one of the first to start using Fabric Databases. View on-demand sessions with database experts and the Microsoft product team to learn just how easy it is to get started. Watch now

Reply
tex628
Community Champion
Community Champion

Heavy data load

Hello, I'm looking for some advice! 

I'm currently running a report with an increadibly large .txt file as a source. The file is on 2.5GB and therefore the loadtime from the query is really long, making it quite difficult to work. 

 

In the query I'm placing filters on date that should mean that i load less than half of the table but it still insists of loading all of the 2,5GB... 

 

Anyone have any advice on how to lessen the load?

 

Br,

Johannes


Connect on LinkedIn
2 REPLIES 2
Stachu
Community Champion
Community Champion

I'd try disabling background preview download, it may improve the experience

Is the load long when you apply the steps, or when you transform the query?
If it's the transformation then you can create a smaller version of the file, e.g. 100MB to do the initial setup and later just switch connection to the bigger file

 



Did I answer your question? Mark my post as a solution!
Thank you for the kudos 🙂

tex628
Community Champion
Community Champion

Working within the query is no problems at all, but any change in the big file triggers the load and you're stuck with a 15 min query load. I'm gonna see if i can put a filter in the software that produces the sourcefile, might be possible to reduce it there. I'm not confident enough to go into the txt file manually and reduce it!

 

Thanks! 

/ J


Connect on LinkedIn

Helpful resources

Announcements
Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount!

Dec Fabric Community Survey

We want your feedback!

Your insights matter. That’s why we created a quick survey to learn about your experience finding answers to technical questions.

ArunFabCon

Microsoft Fabric Community Conference 2025

Arun Ulag shares exciting details about the Microsoft Fabric Conference 2025, which will be held in Las Vegas, NV.

December 2024

A Year in Review - December 2024

Find out what content was popular in the Fabric community during 2024.