Forum Discussion
Optimizing running total with a reset measure
Hello
I am currently using a running total that resets every time the measure [Pos to Neg] = 1. The running totals is just counting the dates/days.
The measure works fine, however it is too slow to actually be useful in production. The fact table contains about 1.4 million rows.
I am using most of the pattern from this SQLBI video: Implementing running total from arbitrary dates in DAX - Unplugged #34 - YouTube
RT With Reset =
VAR _RefDate =
MAX ( 'Calendar (Period)'[Period Date] )
VAR _AllNegativeDates =
FILTER ( ALL ( 'Calendar (Period)'[Period Date] ), [No of Days (Internal)] = 1 )
VAR _NegativeDatesBeforeNow =
FILTER ( _AllNegativeDates, 'Calendar (Period)'[Period Date] <= _RefDate )
VAR _LastNegative =
MAXX ( _NegativeDatesBeforeNow, 'Calendar (Period)'[Period Date] )
VAR _AllResetDates =
FILTER ( ALL ( 'Calendar (Period)'[Period Date] ), [Pos to Neg] = 1 )
VAR _ResetDatesBeforeNow =
FILTER ( _AllResetDates, 'Calendar (Period)'[Period Date] <= _RefDate )
VAR _LastReset =
MAXX ( _ResetDatesBeforeNow, 'Calendar (Period)'[Period Date] )
VAR _DatesToUse =
DATESBETWEEN ( 'Calendar (Period)'[Period Date], _LastReset, _LastNegative )
VAR _Result =
COUNTROWS ( _DatesToUse )
VAR _LastDateWithQty =
MAX ( 'Calendar (Period)'[Period Date] )
VAR _FirstVisibleDate =
MIN ( 'Calendar (Period)'[Period Date] )
RETURN
IF (
_FirstVisibleDate <= _LastDateWithQty
&& [No of Days (Internal)] <> BLANK(),
_Result
)
The DAX query takes around 40 secs to run, with a filter on one specific product. If no product is filtered, it will likely go on for a lot of time.
In the final report, all products should be visible, which is why this is an issue.
I believe my issue with the measure is that I have so many steps related to the date, but I am not sure how I could optimize it. The other measures that are referenced in the measure is now slow.
This is how it looks in a matrix table visualization for reference
| Date | RT With Reset | Pos to neg |
| 04-03-2022 | 1 | |
| 05-03-2022 | 1 | |
| 06-03-2022 | 2 | |
| 07-03-2022 | 3 | |
| 08-03-2022 | 1 | |
| 09-03-2022 | 1 |
Hi Anonymous
I happened to come across your post. I have had a play around and have attached a PBIX with some potential improvements.
The main things I looked at:
- Minimise nested measures (some overhead involved):
- Rewrote [Qty on Hand Total] using SUMX.
- Eliminated [Pos to Neg] and [No of Days (Internal)] from the final measure.
- Rather than using ALL ( 'Calendar'[Period Date] ), use a smaller date range covering global min/max Stock[Date].
- Rewrote the logic a bit to detect when the count should start. See AllRequiredDatesFlag variable.
- Rather than counting dates in a table containing a contiguous range of dates as in original _DatesToUse
variable, take the difference between date bounds & add 1.
A matrix with just Period Date & Working (IMPROVED) Measure now takes a bit under 2 seconds for me, or a bit over 3 seconds with Qty Projected as well.
The relevant updated measures are:
Qty On Hand Total = SUMX ( Stock, Stock[Qty On Hand] + Stock[Qty Supply] - Stock[Qty Demand] )Qty Projected = VAR MinStockDate = CALCULATE ( MIN ( Stock[Date] ), REMOVEFILTERS ( ) ) RETURN CALCULATE ( [Qty On Hand Total], DATESBETWEEN ( 'Calendar'[Period Date], MinStockDate, MAX ( 'Calendar'[Period Date] ) ) )Working (IMPROVED) Measure = VAR _RefDate = MAX ( Calendar[Period Date] ) VAR MinStockDate = CALCULATE ( MIN ( Stock[Date] ), REMOVEFILTERS ( ) ) VAR MaxStockDate = CALCULATE ( MAX ( Stock[Date] ), REMOVEFILTERS ( ) ) VAR AllRequiredDates = DATESBETWEEN ( Calendar[Period Date], MinStockDate, MaxStockDate ) VAR AllRequiredDatesFlag = ADDCOLUMNS ( AllRequiredDates, "@Pos", [Qty Projected] >= 0 ) VAR RelevantDatesFlag = FILTER ( AllRequiredDatesFlag, Calendar[Period Date] <= _RefDate ) VAR LastPositive = MAXX ( FILTER ( RelevantDatesFlag, [@Pos] ), Calendar[Period Date] ) VAR LastNegative = MAXX ( FILTER ( RelevantDatesFlag, NOT [@Pos] ), Calendar[Period Date] ) RETURN IF ( LastNegative = _RefDate, INT ( LastNegative - LastPositive ) )There's bound to be room for improvement, but hopefully this is of some use.
Regards,
Owen
- Minimise nested measures (some overhead involved):
19 Replies
- johnt75
Super User
I've no idea if this will be more efficient, but you could try something like
Running Total with reset =
var currentDate = SELECTEDVALUE('Calendar'[Date])
var summaryTable = ADDCOLUMNS(
SUMMARIZE( FILTER('Calendar', 'Calendar'[Date] < currentDate ), 'Calendar'[Date]), "@pos to neg", [Pos to Neg] )
var lastReset = SELECTCOLUMNS( TOPN(1, FILTER( summaryTable, [@pos to neg] = 1 ), [Date], ASC ),
"@last date", [Date] )
return COUNTROWS( DATESBETWEEN( 'Calendar'[Date], lastReset, currentDate ) )- AnonymousNot applicable
Hello and thank you for taking interest in my question.
Sadly this does not work. It calculates a running total, but does not reset. The lastReset variables does not return anything.- johnt75
Super User
could you share a small sample of data, ideally in a pbix file ?
- OwenAuger
Super User
Hi Anonymous
I happened to come across your post. I have had a play around and have attached a PBIX with some potential improvements.
The main things I looked at:
- Minimise nested measures (some overhead involved):
- Rewrote [Qty on Hand Total] using SUMX.
- Eliminated [Pos to Neg] and [No of Days (Internal)] from the final measure.
- Rather than using ALL ( 'Calendar'[Period Date] ), use a smaller date range covering global min/max Stock[Date].
- Rewrote the logic a bit to detect when the count should start. See AllRequiredDatesFlag variable.
- Rather than counting dates in a table containing a contiguous range of dates as in original _DatesToUse
variable, take the difference between date bounds & add 1.
A matrix with just Period Date & Working (IMPROVED) Measure now takes a bit under 2 seconds for me, or a bit over 3 seconds with Qty Projected as well.
The relevant updated measures are:
Qty On Hand Total = SUMX ( Stock, Stock[Qty On Hand] + Stock[Qty Supply] - Stock[Qty Demand] )Qty Projected = VAR MinStockDate = CALCULATE ( MIN ( Stock[Date] ), REMOVEFILTERS ( ) ) RETURN CALCULATE ( [Qty On Hand Total], DATESBETWEEN ( 'Calendar'[Period Date], MinStockDate, MAX ( 'Calendar'[Period Date] ) ) )Working (IMPROVED) Measure = VAR _RefDate = MAX ( Calendar[Period Date] ) VAR MinStockDate = CALCULATE ( MIN ( Stock[Date] ), REMOVEFILTERS ( ) ) VAR MaxStockDate = CALCULATE ( MAX ( Stock[Date] ), REMOVEFILTERS ( ) ) VAR AllRequiredDates = DATESBETWEEN ( Calendar[Period Date], MinStockDate, MaxStockDate ) VAR AllRequiredDatesFlag = ADDCOLUMNS ( AllRequiredDates, "@Pos", [Qty Projected] >= 0 ) VAR RelevantDatesFlag = FILTER ( AllRequiredDatesFlag, Calendar[Period Date] <= _RefDate ) VAR LastPositive = MAXX ( FILTER ( RelevantDatesFlag, [@Pos] ), Calendar[Period Date] ) VAR LastNegative = MAXX ( FILTER ( RelevantDatesFlag, NOT [@Pos] ), Calendar[Period Date] ) RETURN IF ( LastNegative = _RefDate, INT ( LastNegative - LastPositive ) )There's bound to be room for improvement, but hopefully this is of some use.
Regards,
Owen
- AnonymousNot applicable
Hi OwenAuger and thank you for taking the time.
You certainly optimized it a lot, and it works as intended. Whether it's fast enough for production, I will have to analyze further. Before marking this as a solution I am exploring the table mentioned by johnt75. Although I believe this should be marked as the solution, even if the table functions better in production, since a measure was my original question.- Mafesa_SasolFrequent Visitor
Hi Anonymous johnt75 OwenAuger I have been trying to follow the thread and looks like could be of assistance to my challenge. As can be seen from the attached snapshot. I am trying to develop a calculated column that does a running total for every specific employee using the date between standby column. That column is the Consecutive Employee Check. Please ignore the first value (11). However, I need it to reset after each value on the Days Between Standby Column that is greater than 1. For example, I need to reset 46, 22 and 8. However, the running total continues throughout which is not what I want. Your assistance would be highly appreciated. This is the sample for each unique ID and resets after each new ID but running total continues through for each and every unique ID and does not reset after each new value in the Days Between Standby greater than 1.
This is the DAX for this column:Consecutive Employee Check =VAR CurrentStandbyDays = LOOKUPVALUE(Standby[Standby Days], Standby[Index], Standby[Index])VAR ConsecStandbyDays =IF(LOOKUPVALUE(Standby[Personnel Number], Standby[Index], Standby[Index]) = LOOKUPVALUE(Standby[Personnel Number], Standby[Index], Standby[Index]+1)&& LOOKUPVALUE(Standby[Days Between Standby], Standby[Index], Standby[Index]) < 7 ,CALCULATE (SUM ( 'Standby'[Standby Days] ),FILTER ( ALLEXCEPT(Standby,'Standby'[Personnel Number], 'Standby'[Days Between Standby]), 'Standby'[Index] <= EARLIER ('Standby'[Index]))),CurrentStandbyDays)Return ConsecStandbyDays
- Minimise nested measures (some overhead involved):