Importing data from Azure Blob storage is too slow

[saurabhabh]

I was trying to import my data stored as JSON in Azure blob storage. But the connector seems to be too slow and takes around 2 hrs to bring 15 mb of data. Can we improve on this ? Why does it take so long ? 

My blob storage configuration is 

Performance/Access tier

Standard/Hot

Replication

Read-access geo-redundant storage (RA-GRS)

Account kind

StorageV2 (general purpose v2)

Let me know what can I do about it. 

[brentleslie]

This is classic Microsoft.  Get eveyrone to use their storage medium, then not optimize the storage medium for writes/reads. "We have your data, but if you want to access it, good luck".

[user2jenny]

It's still very slow

[v-juanli-msft · Accepted solution]

Hi @saurabhabh 

When importing data from Azure Blob storage, it is slow generally.

Some tips you may have a try:

1. Disable "Backgroud data" and "parallel loading of tables" in current files under options. 

2. If you files have many columns of date/time type, you can disabled Data Load -> Auto Date/Time, so behind the scenes it doesn't create a date table automatically, which takes time.

3. Please disable load query.

Disabling Load doesn’t mean the query won’t be refreshed, it only means the query won’t be loaded into the memory. 

4. You can filter your resource table and just retain the neccesary data before you load all the data into model.

Change JSON Blob data during import

In addtion, you can follow those two articles to optimize data model to improve the performance:

Data Import Best Practices in Power BI
Power BI Performance Tips and Techniques

Best Regards
Maggie

Community Support Team _ Maggie Li
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.