Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
saurabhabh
Frequent Visitor

Importing data from Azure Blob storage is too slow

I was trying to import my data stored as JSON in Azure blob storage. But the connector seems to be too slow and takes around 2 hrs to bring 15 mb of data. Can we improve on this ? Why does it take so long ? 

 

My blob storage configuration is 

 

Performance/Access tier
Standard/Hot
Replication
Read-access geo-redundant storage (RA-GRS)
Account kind
StorageV2 (general purpose v2)
 
Let me know what can I do about it. 
 
 
 
 
1 ACCEPTED SOLUTION
v-juanli-msft
Community Support
Community Support

Hi @saurabhabh 

When importing data from Azure Blob storage, it is slow generally.

 

Some tips you may have a try:

1. Disable "Backgroud data" and "parallel loading of tables" in current files under options. 

2. If you files have many columns of date/time type, you can disabled Data Load -> Auto Date/Time, so behind the scenes it doesn't create a date table automatically, which takes time.

3. Please disable load query.

Disabling Load doesn’t mean the query won’t be refreshed, it only means the query won’t be loaded into the memory. 

4. You can filter your resource table and just retain the neccesary data before you load all the data into model.

Change JSON Blob data during import

 

In addtion, you can follow those two articles to optimize data model to improve the performance:

Data Import Best Practices in Power BI
Power BI Performance Tips and Techniques

 

Best Regards
Maggie

 

Community Support Team _ Maggie Li
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

3 REPLIES 3
brentleslie
New Member

This is classic Microsoft.  Get eveyrone to use their storage medium, then not optimize the storage medium for writes/reads. "We have your data, but if you want to access it, good luck".

user2jenny
Frequent Visitor

It's still very slow

v-juanli-msft
Community Support
Community Support

Hi @saurabhabh 

When importing data from Azure Blob storage, it is slow generally.

 

Some tips you may have a try:

1. Disable "Backgroud data" and "parallel loading of tables" in current files under options. 

2. If you files have many columns of date/time type, you can disabled Data Load -> Auto Date/Time, so behind the scenes it doesn't create a date table automatically, which takes time.

3. Please disable load query.

Disabling Load doesn’t mean the query won’t be refreshed, it only means the query won’t be loaded into the memory. 

4. You can filter your resource table and just retain the neccesary data before you load all the data into model.

Change JSON Blob data during import

 

In addtion, you can follow those two articles to optimize data model to improve the performance:

Data Import Best Practices in Power BI
Power BI Performance Tips and Techniques

 

Best Regards
Maggie

 

Community Support Team _ Maggie Li
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.