Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.

Reply
Courtsheagraham
Regular Visitor

Using List.Contains causing long load time

Ok...let me see if I can explain this and get some help.

I've got static data in an external excel sheet.

I've loaded that into power query, then Transformed  a column to Convert to List and named it "InServiceXIDs_List"

I've then gone through multiple tables, and used "... each (List.Contains(InServiceXIDs_List,[unit_id])=true))" to filter "unit_id" according to the list.

 

I've also used two of my tables to create a Merge Queries as New. In the new merged queries I do not filter the data because it's already been filtered by the tables that merge being filtered...if that makes sense.

 

Everything is working, but when I Close & Apply it takes a very long time, 20 minutes or so, to load that data!

Can anyone explain as to why? Is the List.Contains going through each row and does that have to process from the excel sheet? I ask that because when I watch the Refresh window load it says the name of the excel sheet for each of the tables where I've used that List.Contains function.

So, does it go through each row, filtering the unit_id, then process that through and to the merged querie on each row!?! That would be extremely time intensive regarding the amount of data it would be parsing/filtering.

 

Thanks.

2 ACCEPTED SOLUTIONS
ppm1
Solution Sage
Solution Sage

In each table/query where you use the list, add a step up front that buffers your list with

 

BufferedList = List.Buffer(InServiceXIDs)

 

Then use BufferedList in place of InServiceXIDs in your later query step(s).  Also, you don't need to have "=true" after List.Contains.

 

Pat

Microsoft Employee

View solution in original post

watkinnc
Super User
Super User

Try this:

 

(List.Contains(List.Buffer(InServiceXIDs_List),[unit_id])=true))"

 

--Nate


I’m usually answering from my phone, which means the results are visualized only in my mind. You’ll need to use my answer to know that it works—but it will work!!

View solution in original post

5 REPLIES 5
watkinnc
Super User
Super User

Try this:

 

(List.Contains(List.Buffer(InServiceXIDs_List),[unit_id])=true))"

 

--Nate


I’m usually answering from my phone, which means the results are visualized only in my mind. You’ll need to use my answer to know that it works—but it will work!!

Thank you for the information! Seems to be working, about to run it through a similar report and see what the results there are. Thanks again.

ppm1
Solution Sage
Solution Sage

In each table/query where you use the list, add a step up front that buffers your list with

 

BufferedList = List.Buffer(InServiceXIDs)

 

Then use BufferedList in place of InServiceXIDs in your later query step(s).  Also, you don't need to have "=true" after List.Contains.

 

Pat

Microsoft Employee

worked like magic, thanks

Thank you for the info and feedback. I need to start doing exactly what you've done here with using the defined variables. I've not turned that to a habbit yet! And I was wondering about using "true", wasn't sure that I needed it, but the tutorial I referenced had it, so here I am! haa. Thanks again.

Helpful resources

Announcements
RTI Forums Carousel3

New forum boards available in Real-Time Intelligence.

Ask questions in Eventhouse and KQL, Eventstream, and Reflex.

MayPowerBICarousel

Power BI Monthly Update - May 2024

Check out the May 2024 Power BI update to learn about new features.

LearnSurvey

Fabric certifications survey

Certification feedback opportunity for the community.

Top Solution Authors
Top Kudoed Authors