Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
AmiraBedh
Resident Rockstar
Resident Rockstar

Understand the incremental refresh partitions

I have a table where I configured fact_transactions (the source is a SQL Server table) the incremental refresh like below :

 

AmiraBedh_0-1717437335846.png

 

The fact_transactions contains data of a monthly basis trasactions, so all the records for one month have the same date value (for example the 1st business day of each month)

We don't have the scheduled refresh for Saturday and Sunday.

Now I through SSMS, I managed to connect via the endpoint of the workspace of my dataset and I get the following :

AmiraBedh_1-1717437550370.png

AmiraBedh_2-1717437590825.png

 

I don't understand why there is only one partition having rows knowing that the table has data from 2021 until today

and even if I proceed with a full load on the partition it gets data only for May 2024


Proud to be a Power BI Super User !

Microsoft Community : https://docs.microsoft.com/en-us/users/AmiraBedhiafi
Linkedin : https://www.linkedin.com/in/amira-bedhiafi/
StackOverflow : https://stackoverflow.com/users/9517769/amira-bedhiafi
C-Sharp Corner : https://www.c-sharpcorner.com/members/amira-bedhiafi
Power BI Community :https://community.powerbi.com/t5/user/viewprofilepage/user-id/332696
7 REPLIES 7
lbendlin
Super User
Super User

The # Rows information in SSMS is notoriously unreliable.  Use DAX Studio instead to check how many actual rows are in each partition. 

 

Note: specifying 31 days will result in a maximum of 61 day partitions. You may not want that. Instead, use 1 Month.

Can you please clarify this part ? :

Note: specifying 31 days will result in a maximum of 61 day partitions. You may not want that. Instead, use 1 Month.

 

Proud to be a Power BI Super User !

Microsoft Community : https://docs.microsoft.com/en-us/users/AmiraBedhiafi
Linkedin : https://www.linkedin.com/in/amira-bedhiafi/
StackOverflow : https://stackoverflow.com/users/9517769/amira-bedhiafi
C-Sharp Corner : https://www.c-sharpcorner.com/members/amira-bedhiafi
Power BI Community :https://community.powerbi.com/t5/user/viewprofilepage/user-id/332696

When you say "31" days but you are in the last days of August (say, August 30) then "31 days"  will also include July 31, which will automatically include ALL days in July - hence 61 partitions.  Day partitions are only aggregrated into monthly partitions once they are no longer part of the refresh window.

 

Switching from "31 days"  to "1 month"  will reduce the number of partitions from 61 to 2 (ignoring the cold partitions for now)

 

How many rows are in each of the daily partitions?

I have data coming each 2nd business day for each month with the transaction date having the same value (as the 2nd business day) about 1000000 rows.

So what do you think ?


Proud to be a Power BI Super User !

Microsoft Community : https://docs.microsoft.com/en-us/users/AmiraBedhiafi
Linkedin : https://www.linkedin.com/in/amira-bedhiafi/
StackOverflow : https://stackoverflow.com/users/9517769/amira-bedhiafi
C-Sharp Corner : https://www.c-sharpcorner.com/members/amira-bedhiafi
Power BI Community :https://community.powerbi.com/t5/user/viewprofilepage/user-id/332696

As long as the monthly partitions refresh in well  under 5 hours that is ok.

Can you explain more how we can get 61 partitions? As I can't get anything to help me understand it in the documentation.


Proud to be a Power BI Super User !

Microsoft Community : https://docs.microsoft.com/en-us/users/AmiraBedhiafi
Linkedin : https://www.linkedin.com/in/amira-bedhiafi/
StackOverflow : https://stackoverflow.com/users/9517769/amira-bedhiafi
C-Sharp Corner : https://www.c-sharpcorner.com/members/amira-bedhiafi
Power BI Community :https://community.powerbi.com/t5/user/viewprofilepage/user-id/332696

Helpful resources

Announcements
July 2024 Power BI Update

Power BI Monthly Update - July 2024

Check out the July 2024 Power BI update to learn about new features.

July Newsletter

Fabric Community Update - July 2024

Find out what's new and trending in the Fabric Community.