Full Data Table Loading instead of just the Query defined in Power Query

[lowdman87]

Due to confidentiality within my company I cannot share my dataset, but I will do my best to share some details.  I have a table that gets approximately 2.5 million rowas added to it each day as an archiving process.  As part of the archiving process, a timestamp is applied to each new row.  The table is currently around 350 million rows.  For a specific project, I only need about 15 million of those rows. 

In Power Query, I am filtering the rows by the days needed, then I am also filtering on a few other conditions to yield the desired 15 million rows.  The refresh in Power Query takes a long time, but eventually it shows the desired result with only the dates intended.  When I close Power Query and Apply Settings, the file starts the load.  After some time, the table is showing 293 million rows has been loaded so far, so it seems that ALL of the data in that table is loaded, then it is filtered.  I have not noticed it with past tables, but it could be because they are smaller.  

1. Is this how Power BI Desktop normally works?  Load the full table then filter to defined requirements?
2. If so, why does the same table need to load completely again when I change my date condition to only pull 10 million of the rows?  It seems if the entire data set is already loaded then this would be a quick operation, but it is not.  

[PabloDeheza]

Hi there!

What you could do to prevent from Power BI to read the whole table is using a native query so it applies the filter before it reaches Power BI. If your connection supports native queries then you should try this.

If you are not sure what these are, here is the Microsoft documentation.

https://docs.microsoft.com/en-us/power-query/native-database-query

If you have any question please let me know!

[amitchandak · Accepted solution]

@lowdman87 , if the query folding is not proper then power query may load full data

https://docs.microsoft.com/en-us/power-query/power-query-folding

so check for that.

Also if possible use a query in advance option of connection or create a view on database side and use