Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Enhance your career with this limited time 50% discount on Fabric and Power BI exams. Ends August 31st. Request your voucher.

Reply
Anonymous
Not applicable

Add additional RAM into my laptop can make my query load faster?

Hi all,

 

Currently my laptop have 16gb of RAM and I working with million of data in Powerbi. It take forever to load 100million of data now. So my action is to add additional RAM into my laptop from 16gb to 64gb.

 

Do you think that the data load of power query will be faster than before after I upgrade my RAM?

 

Regards,

Nuha

1 ACCEPTED SOLUTION
Anonymous
Not applicable

I can offer you some intuitive advice, just from long experience with Power Query.

Thats a lot of filters, before an unpivot and then some more filters, and then removing duplicates.  There's no way query folding is surviving that--I guarantee it's folding no further than after you remove duplicates, 
It looks like these are all tables from the same database--they may even be properly keyed and indexed.  I would do all of the table joins after one or two of the filters that cut down the most data--but as far upstream as possible.  I would left join them as Table.Join, as opposed to Table.NestedJoin, so you may have to adjust some column names for your joins.  Nevertheless, I am often surprised at how easily you can join quite a few tables without issue, because it's all folding to SQL, which can handle it easily.  "Do all the joins while you are still folding, no matter how many rows it returns."
You could probably still fold after applying the current month fillter--which you should calculate as a single date value in another query Named ThisMonthStart, so that you have that single date for your main query to use as a filter, and consequently for Power Query to turn into SQL, instead of however you might be doing it now.
ThisMonthStart = let DayNumber = (Date.Day(DateTime.LocalNow())*-1) in Date.AddDays(Date.From(DateTime.LocalNow()), DayNumber)

 

And then you can make your main query's #"Filter Only Current Month" step can be

Table.SelectRows(PriorStepName, each [DateColumnName] > ThisMonthStart)

 

So apply the date filter before the joins and see if that helps.

 

If you are using some of these join steps merely to select a list of values in your table against another table (as in, you are just getting rid of all the columns in the other table right away), and its not too many values or a lot of strings, consider filtering instead, using a buffered list, like
Table.SelectRows(PriorStepName, each List.Contains(List.Buffer(OtherTableQuery[ValuesToKeepColumn]), [ThisTableColumn]))

 

That gives SQL an IN statement with those values in it.

 

Make sure you are extracting dates using Date.From, not DateTime.Date.  They claim DateTime.Date folds, but it will fail you in places where Date.From succeeds.
The Unpivot steps are not necessarily hit or miss as far as folding, but it's not something I can define here.
The group will likely always be slow, so do it as far down as possible, like you already have it.
OK, good luck!

 

--Nate

View solution in original post

6 REPLIES 6
Anonymous
Not applicable

I can offer you some intuitive advice, just from long experience with Power Query.

Thats a lot of filters, before an unpivot and then some more filters, and then removing duplicates.  There's no way query folding is surviving that--I guarantee it's folding no further than after you remove duplicates, 
It looks like these are all tables from the same database--they may even be properly keyed and indexed.  I would do all of the table joins after one or two of the filters that cut down the most data--but as far upstream as possible.  I would left join them as Table.Join, as opposed to Table.NestedJoin, so you may have to adjust some column names for your joins.  Nevertheless, I am often surprised at how easily you can join quite a few tables without issue, because it's all folding to SQL, which can handle it easily.  "Do all the joins while you are still folding, no matter how many rows it returns."
You could probably still fold after applying the current month fillter--which you should calculate as a single date value in another query Named ThisMonthStart, so that you have that single date for your main query to use as a filter, and consequently for Power Query to turn into SQL, instead of however you might be doing it now.
ThisMonthStart = let DayNumber = (Date.Day(DateTime.LocalNow())*-1) in Date.AddDays(Date.From(DateTime.LocalNow()), DayNumber)

 

And then you can make your main query's #"Filter Only Current Month" step can be

Table.SelectRows(PriorStepName, each [DateColumnName] > ThisMonthStart)

 

So apply the date filter before the joins and see if that helps.

 

If you are using some of these join steps merely to select a list of values in your table against another table (as in, you are just getting rid of all the columns in the other table right away), and its not too many values or a lot of strings, consider filtering instead, using a buffered list, like
Table.SelectRows(PriorStepName, each List.Contains(List.Buffer(OtherTableQuery[ValuesToKeepColumn]), [ThisTableColumn]))

 

That gives SQL an IN statement with those values in it.

 

Make sure you are extracting dates using Date.From, not DateTime.Date.  They claim DateTime.Date folds, but it will fail you in places where Date.From succeeds.
The Unpivot steps are not necessarily hit or miss as far as folding, but it's not something I can define here.
The group will likely always be slow, so do it as far down as possible, like you already have it.
OK, good luck!

 

--Nate

Anonymous
Not applicable

Hi @Anonymous ,

 

Woow what a long statement and thank you for all of your suggestion.

But can you explain more what is "folding to SQL" means? What is "fold" even means?

 

Regards,

Nuha

Anonymous
Not applicable

Food means "to send a query to a data source by transforming the M query into the language of the data source". For example, if you query a SQL Server database, Power Query can fetch and transform the data using SQL, and the work gets done at the data source--which is almost always faster and less work for your computer. If your query is too complex, or uses certain transforms, then Power Query will instead import the data and transform it on your machine, usually in chunks and at the cost of your limited computer resources.

 

I've run thousands of complex queries, against just about every data source, and 16GB of RAM has pretty much always been enough.

 

Try my strategies!

 

--Nate

Anonymous
Not applicable

Hi @Anonymous ,

 

Okay sir. Noted. Thank you for your explanation and advice.

 

Regards,

Nuha

Anonymous
Not applicable

"It depends." Can we have a look at your query?

 

--Nate

Anonymous
Not applicable

Hi @Anonymous ,

 

I'm not sure from which view you want to see my query. I have some screenshot for you below. Currently I'm loading 50million of data (already more than 4 hour).

 

Regards,

Nuha

 

nuhasan_1-1687328749982.png

 

 

 

nuhasan_0-1687328723096.png

 

nuhasan_2-1687328769973.png

 

 

Helpful resources

Announcements
July PBI25 Carousel

Power BI Monthly Update - July 2025

Check out the July 2025 Power BI update to learn about new features.

Join our Fabric User Panel

Join our Fabric User Panel

This is your chance to engage directly with the engineering team behind Fabric and Power BI. Share your experiences and shape the future.

June 2025 community update carousel

Fabric Community Update - June 2025

Find out what's new and trending in the Fabric community.