Data Upload Issue

[JyotiBora]

 Hi Team,

I am trying to fetch data in Power BI desktop using data flows in import mode. It was working fine until we tried adding 1 more column. The dataflows are fine, giving the results but due to that column, data size got increased and now it's giving memory error in Power BI Desktop. I am not sure what should I do? Which memory this refers.

Any help is appreciated.

Thanks

[ryan654321]

hi  @JyotiBora 
I hope the answers provided helped resolve your issue.
If our solution was helpful, could you please mark it as the accepted solution? This will help other community members facing a similar situation find the solution more easily.

[ryan654321]

Hi  @JyotiBora 

Could you please confirm if the issue has been resolved? If not, feel free to reach out if you have any further questions.

Your update would be helpful for other members who may face a similar issue.

[SamInogic]

Hi,

From the screenshots, the issue looks like Power BI Desktop RAM/memory exhaustion during model refresh, not a problem with the dataflow itself. Since the problem started after adding one extra column to a table with ~16M rows, that new column may have significantly increased the model size (especially if it has high cardinality / many unique values).

A few things to check:

1. Check your machine RAM

Can you confirm:

• RAM size (8GB / 16GB / 32GB+)
• Whether you are using 64-bit Power BI Desktop
• Your license type (Pro / PPU / Fabric Capacity)

This helps identify if it is a local Desktop limitation or service capacity issue.

2. Disable parallel loading (important for large models)

Power BI may be trying to load multiple large tables at once, causing memory spikes.

Go to:
File → Options & Settings → Options → Current File → Data Load
Then under Parallel loading of tables, set it to One (disable parallel loading).

This often helps when importing very large tables.

3. Review the new column

Since refresh worked before adding the column:

• Check if the new column has high cardinality (many unique values)
• Ensure the datatype is optimized (Whole Number / Decimal / Date instead of Text where possible)
• Remove unnecessary text columns if not required in reporting
  
  4. Reduce Desktop data during development

For development, try loading only recent data using a parameter/date filter, then load full data in the Service.

For example:

• Load only last 3–6 months in Desktop
• Publish to Service
• Refresh full dataset there
  
  5. Incremental Refresh (recommended long-term)

Since your data is daily from 2019 onward, Incremental Refresh would be the best long-term solution:

• Keep historical partitions static
• Refresh only recent data
• Lower memory consumption
• Faster refresh in both Desktop and Service

Also, if you are on Power BI Pro, very large import models can hit memory/resource limits more quickly.

Could you share your RAM size and whether the newly added column is text with many unique values? That will help narrow down the exact cause.

Hope this helps.

Thanks!

[v-nmadadi-msft]

Hi @JyotiBora 

May I check if this issue has been resolved? If not, Please feel free to contact us if you have any further questions.

Thank you

[v-nmadadi-msft]

Hi @JyotiBora ,

I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions.

Thank you.

[Balqis_Adan]

What is your Power BI License type?
Free
Pro
Premium Per User (PPU)
Microsoft Fabric / Premium Capacity
And what is your Computer RAM?
8GB
16GB
32GB
This will help give you the exact right solution for your specific situation.

please tell us to gain your the solution sin your probleem from RAM

[stoic-harsh]

Hey @JyotiBora,

As per your ss, Nielsen alone has ~16M rows. (Add a new column × 16M rows) × 5 tables loading simultaneously, and your RAM spikes during import. That's the culprit.

As an immediate fix, disable parallel loading: 

• Go to File > Options & Settings > Options > Current File > Data Load > Under "Parallel loading of tables", select One (disable parallel loading).

Consider incremental refresh, a long-term fix (RAM problem solved + Refresh time drops significantly):

• Add RangeStart / RangeEnd parameter on your date column.
• Define the incremental refresh policy in PBI Desktop.
• Publish to Service.

On DirectQuery, it will solve the memory issue too, but kills report performance at runtime. Not a great trade-off for 16M rows, I'd say (until you club incremental refresh and Direct Query ofc).

Hope this helps!

Best,

Harshit

[tharunkumarRTK · Accepted solution]

@JyotiBora 

1. Create a M Query parameter.

2. Reduce the amount of data that you load in to power bi desktop using a conditional logic based on the parameter value.

3. Complete your development activities. 

4. Publish your report to power bi service.

5. Change the parameter value in power bi service. And supply necessary data source connection credentails or connection mapping.

6. Trigger an on demand data refresh.

7. Once the refresh is completed. If you want you can download the report again and perform any other activities in your file. 

Checkout this page for sample steps: Chris Webb's BI Blog: Limit The Amount Of Data You Work With In Power BI Desktop Using Parameters An... 

Connect on LinkedIn

Read my blogs here: TechieTips

Did I answer your question? Mark my post as a solution!
If I helped you, click on the Thumbs Up to give Kudos. Proud to be a Super User!

[krishnakanth240]

Hi @JyotiBora 

Error refering to Power BI Desktop RAM usage during import refresh process and not only dataflow itself. As issue started after adding new column to 16 million row table, column will increase the model size due to cardinality.

Try removing unnecessary columns, reduce data volume before loading, ensure correct datatypes,  and confirm usage of 64-bit Power BI Desktop. If the dataset continues to grow then consider DirectQuery

[danextian]

How cardinal is the new column for it to be causing a memory issue during refresh? When this happens to me, I'd normally just clear the data cache.

[JyotiBora]

I have tried that already. But it's not working