Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.

Reply
Kirt1965
Helper III
Helper III

Why does Power Query repull gigs of data for the smallest change?

Hello... Can someone explain why Power Query forces a refresh for the simplest things?  A case in point: It is forcing the pull of several gigs of data (seen in the image below) when all I did was tell it to sort the 18 result rows on the TcID column.  I could kind of understand if I had inserted that step in the middle of other steps, but that's not what I did.  I added the sort as the last step after the 18 rows were already returned.  Why does it force a complete refresh pulling gigs of data when it already had the 18 rows I wanted it to sort?  It does this kind of thing all the time and, frankly, it seems idiotic.  It should be able to retain what was already there with me having to wait long periods of time for a change like this.

Capture.PNG

1 ACCEPTED SOLUTION
v-stephen-msft
Community Support
Community Support

Hi @Kirt1965 ,

 

When you refresh data in Power BI, it performs a full refresh and not an incremental one. This means that whenever you refresh data in a dataset, Power BI uses information in the dataset to connect to the data sources defined for it, query for updated data, and then loads the updated data into the dataset. Any visualizations in your reports or dashboards based on the data are updated automatically. If your dataset is configured for a live or DirectQuery connection, datasets are refreshed approximately each hour or when interaction with the data occurs. It is by design behavior of Power query to send multiple queries, one to find the schema and the other query to get the actual data.

 

Best Regards,

Stephen Tao

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.           

View solution in original post

2 REPLIES 2
v-stephen-msft
Community Support
Community Support

Hi @Kirt1965 ,

 

When you refresh data in Power BI, it performs a full refresh and not an incremental one. This means that whenever you refresh data in a dataset, Power BI uses information in the dataset to connect to the data sources defined for it, query for updated data, and then loads the updated data into the dataset. Any visualizations in your reports or dashboards based on the data are updated automatically. If your dataset is configured for a live or DirectQuery connection, datasets are refreshed approximately each hour or when interaction with the data occurs. It is by design behavior of Power query to send multiple queries, one to find the schema and the other query to get the actual data.

 

Best Regards,

Stephen Tao

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.           

ppm1
Solution Sage
Solution Sage

Two things:

1. Confirm that the data preview in the background box is turned off. That is a global setting but since your file is already in progress, you may need to turn it off in the Data File section of options too.

ppm1_0-1688166799557.png

 

 

2. Sorting can be an expensive operation as the whole table has to be loaded into memory. #1 should address the issue but if not, consider a different way to get same result.

 

Pat

Microsoft Employee

Helpful resources

Announcements
RTI Forums Carousel3

New forum boards available in Real-Time Intelligence.

Ask questions in Eventhouse and KQL, Eventstream, and Reflex.

MayPowerBICarousel

Power BI Monthly Update - May 2024

Check out the May 2024 Power BI update to learn about new features.

LearnSurvey

Fabric certifications survey

Certification feedback opportunity for the community.

Top Solution Authors
Top Kudoed Authors