Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
Bamak
Helper V
Helper V

Dataset incremental refresh end memory consumption

HI , 

 

I read that the refresh of dataset, loaded twice the dataset in memory
Once for the refresh process & once for a possible report consumption (rendering)

 

I don't know how it was in incremental refresh ? 

Is only the partition to refresh which is loaded in memory + the entire dataset for rendering ?

 

Can we consider please ,  that  DS incremental refresh improve not only volumetry to integrate but also memory consumption ? 

 

Thanks a lot for any help and best regards

 

Christophe 

1 ACCEPTED SOLUTION
v-deddai1-msft
Community Support
Community Support

Hi @Bamak ,

 

>>Is only the partition to refresh which is loaded in memory + the entire dataset for rendering ?

 

Yes, you are right. We need to refresh the whole dataset ,so it loads twice size of dataset in memory for refresh(Your dataSet is refreshing and someone is using the report). For incremental refresh , Only data that has changed needs to be refreshed. so it will not load the whole dataset into memory.  Whenever a user is using the report, the entire dataSet is loaded into memory.

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

 

Best Regards,

Dedmon Dai

View solution in original post

3 REPLIES 3
v-deddai1-msft
Community Support
Community Support

Hi @Bamak ,

 

>>Is only the partition to refresh which is loaded in memory + the entire dataset for rendering ?

 

Yes, you are right. We need to refresh the whole dataset ,so it loads twice size of dataset in memory for refresh(Your dataSet is refreshing and someone is using the report). For incremental refresh , Only data that has changed needs to be refreshed. so it will not load the whole dataset into memory.  Whenever a user is using the report, the entire dataSet is loaded into memory.

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

 

Best Regards,

Dedmon Dai

What a pleasure to see that someone has read my post (very clear) correctly and has therefore answered precisely

Kind regards 

StefanoGrimaldi
Resident Rockstar
Resident Rockstar

not very clear of the question here Im assuming you want to know hoy incremental refresh works? if thats the case what it does it will identify on the sources records data those that had any type of change, and will only refresh those record, get new ones, and deelted ones changes, and them recalculate the model based on those changes, so finally with incremental refresh you use less memory to refresh and render data, and less time consuption for this case. 

 

if not what you where asking about please elaborate a little more as its a little unclear the question. 

 

if this helped please give some kudos if solved your question mark as solution for others to find. 





Did I answer your question? Mark my post as a solution! / Did it help? Give some Kudos!

Proud to be a Super User!




Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors