Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Get certified in Microsoft Fabric—for free! For a limited time, get a free DP-600 exam voucher to use by the end of 2024. Register now

Reply
osinquinvdm
Advocate II
Advocate II

why filtering the query does not change the volume of data that get loaded?

In order to make load time faster during the testing phase I'd like to filter the data and work on a subset.

Unfortunately after I applied a filtering criteria that should filter out 80% of the data, I still see that PBI loads the exact same volume of data.

Why is that?

How can I reduce the amount of data I work with during the dev phase (besides exporting to Excel and reimporting a filtered out csv...)

thanks 

1 ACCEPTED SOLUTION

Then the filter will not be able to affect the load-process, as no query-folding will take place on this source.

 

Subsequent calculations should be faster, (but not necessarily, as the commands will be translated to an internal execuation plan that the user cannot influence).

Try using a Table.Buffer for your filter-expression.

Imke Feldmann (The BIccountant)

If you liked my solution, please give it a thumbs up. And if I did answer your question, please mark this post as a solution. Thanks!

How to integrate M-code into your solution -- How to get your questions answered quickly -- How to provide sample data -- Check out more PBI- learning resources here -- Performance Tipps for M-queries

View solution in original post

9 REPLIES 9
Anonymous
Not applicable

Hi osqinquinvdm

If you use a DB, please check in the Query Editor, if Query folding is broken.

 

0.png

ImkeF
Super User
Super User

Are you connecting to a SQL-server or what is your data source?

Imke Feldmann (The BIccountant)

If you liked my solution, please give it a thumbs up. And if I did answer your question, please mark this post as a solution. Thanks!

How to integrate M-code into your solution -- How to get your questions answered quickly -- How to provide sample data -- Check out more PBI- learning resources here -- Performance Tipps for M-queries

The source is a simple csv with next to zero processing

2017-03-03 15_40_17-311-extract - Query Editor.png

Then the filter will not be able to affect the load-process, as no query-folding will take place on this source.

 

Subsequent calculations should be faster, (but not necessarily, as the commands will be translated to an internal execuation plan that the user cannot influence).

Try using a Table.Buffer for your filter-expression.

Imke Feldmann (The BIccountant)

If you liked my solution, please give it a thumbs up. And if I did answer your question, please mark this post as a solution. Thanks!

How to integrate M-code into your solution -- How to get your questions answered quickly -- How to provide sample data -- Check out more PBI- learning resources here -- Performance Tipps for M-queries

Thanks for your comments. Where can I find further details on that "query folding" aspect

Greg_Deckler
Super User
Super User

Where are you basing that notion? On the count of rows in the table, on the size of the PBIX file, ?



Follow on LinkedIn
@ me in replies or I'll lose your thread!!!
Instead of a Kudo, please vote for this idea
Become an expert!: Enterprise DNA
External Tools: MSHGQM
YouTube Channel!: Microsoft Hates Greg
Latest book!:
Power BI Cookbook Third Edition (Color)

DAX is easy, CALCULATE makes DAX hard...

on what the pop-up window says and on the time it takes.

 

thanks for jumping in

Might be related to what type of filter you are using. Is it a date field filter? I have seen some weirdness with that. My first question would be, is it really importing everything. So, if you create a quick COUNT aggregation or look at the bottom of Desktop client in the data area, how many rows do you get with and without the filter in the final table? Does it match what is being reported in the pop-up. If those are the same with and without and match the pop-up, then my guess would be that your filter isn't doing what you think it is doing. If the final row counts in the table are different but the pop-up is the same number, then it might have something to do with the date filter weirdness I mentioned.



Follow on LinkedIn
@ me in replies or I'll lose your thread!!!
Instead of a Kudo, please vote for this idea
Become an expert!: Enterprise DNA
External Tools: MSHGQM
YouTube Channel!: Microsoft Hates Greg
Latest book!:
Power BI Cookbook Third Edition (Color)

DAX is easy, CALCULATE makes DAX hard...

Helpful resources

Announcements
November Carousel

Fabric Community Update - November 2024

Find out what's new and trending in the Fabric Community.

Live Sessions with Fabric DB

Be one of the first to start using Fabric Databases

Starting December 3, join live sessions with database experts and the Fabric product team to learn just how easy it is to get started.

Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount! Early Bird pricing ends December 9th.

Nov PBI Update Carousel

Power BI Monthly Update - November 2024

Check out the November 2024 Power BI update to learn about new features.