Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.

Reply
li3112
Helper I
Helper I

Substraction that tallies from Running Sum

Consider the following dataset where placed time goes from 4am to 8pm and fulfillment time goes from 7am to 8pm as well.

li3112_1-1658183941085.png

I calculated the Running Sum, resulting in this matrix

li3112_0-1658183864597.png

 

For every timeslot in the y axis, I can only fulfill 100 orders. I want to calculate a matrix in the same format as above that tells me the remaining orders to fulfill every hour after that 100 runs out. This is the expected result for 4:00 am:

 

li3112_2-1658185629919.png

 Which I calculated using the following formula:

Column = max(Table'[Running Total]-100,0)

 


The issue is that I want every row to pick back up with the first non-zero value from the previous row. In this case, that value would be 40. In other words, since I can only fulfill 100 orders in an hour, at 5 am I am tackling the 40 orders that I couldn't get to at 4 am + the ones that were placed at 5 am. Does anyone have any suggestions for my requierement?

1 ACCEPTED SOLUTION

I had alredy added the comments in the function

    FillShift = (tbl, slot) => let
        // buffer the table
        buf = Table.Buffer(tbl),
        // save away already processed entries
        prc = Table.SelectRows(buf,each [Processed]<> null),
        // items that need processing. consider only placed orders.
        src=Table.SelectRows(buf,each [Processed] = null and [Placed Time] <= #time(slot,0,0)),
        // defer (orders were placed later)
        def = Table.SelectRows(buf,each [Processed] = null and [Placed Time] > #time(slot,0,0)),
        // sort by fulfilled, then placed
        srt = Table.Sort(src,{{"Fulfillment Time", Order.Ascending}, {"Placed Time", Order.Ascending}}),
        // add index
        idx = Table.AddIndexColumn(srt, "Index", 0, 1, Int64.Type),
        // tag lines to process
        rep = Table.ReplaceValue(idx,null,each if [Placed Time]<=#time(slot,0,0) and [Index]<140 then #time(slot,0,0) else null,Replacer.ReplaceValue,{"Processed"}),
        // drop index
        res = Table.RemoveColumns(rep,{"Index"})
        in  
        //concatenate result with saved portions
        prc & res & def

 

The function expects a table with the four required columns Placed Time, Fulfillment Time, Order identifier and Processed.  (The order identifier is generated when exploding the number of received orders into a list)

 

The function also expects a time slot for which the computation needs to happen . That is the hour number.

 

First step is to buffer the entire table to improve query performance

Next step is to save away the portion of the orders (prc) that have already been allocated (that have a Processed value)

Then we identify all orders that can be processed - meaning the Placed Time is before or equal to the time slot

Then we save away (defer) all orders that cannot yet be processed as the Placed Time is after the time slot

Next we sort the eligible orders by fulfillment time and placed time

Then we tag the first 140 orders (or less) with the processed time stamp of the current slot

And finally we combine the three parts back together, ready for the next timeslot.

 

The function is then called for each available timeslot (from 4 to 19) sequentially. 

 

View solution in original post

40 REPLIES 40
lbendlin
Super User
Super User

@li3112 

I think I figured it out (again).

 

For each processing time slot I re-sort the remaining orders by fulfillment time first, then by placed time.  That way changes in priority can be handled.

lbendlin_0-1658448816268.png

let
    //process new timeslot
    FillShift = (tbl, slot) => let
        // save away already processed entries
        prc = Table.SelectRows(Table.Buffer(tbl),each [Processed]<> null),
        // items that need processing
        src=Table.SelectRows(tbl,each [Processed] = null),
        // sort by fulfilled, then placed
        srt = Table.Sort(src,{{"Fulfillment Time", Order.Ascending}, {"Placed Time", Order.Ascending}}),
        // add index
        idx = Table.AddIndexColumn(srt, "Index", 0, 1, Int64.Type),
        // tag lines to process
        rep = Table.ReplaceValue(idx,null,each if [Placed Time]<=#time(slot,0,0) and [Index]<140 then #time(slot,0,0) else null,Replacer.ReplaceValue,{"Processed"}),
        // drop index
        res = Table.RemoveColumns(rep,{"Index"})
        in  
        //concatenate result with saved portion
        prc & res,
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("bZdLkiMhDAXv4nUvLMRPc5WOuf81ZlGIqafIFeEMTKG0oJ5/fz/9z/f7+fmsZ2jx+ftz4T6DvWGcL3zf0L7POGWq2TOOKbQ9Y29C/dAutNO+bDyj67rzPG0Jzcp0Zzv3IvTU5m/YTmlPvUN0CdvAzoLjzYwWTFUmsBF0gp3WHASPpSaQ6jEqKBUJVEUTFE1RZG8WMC8VycRU5AIbQac1O80cBLORBC6auelBVJEqWqBoQRctUeRvpoqWKmpDaLaRznWkx9IUeBtK6LUtNOsQeKrqAuPdiYs8bfC0wdOGVtrqqQs0mtlE3i6altB7Pwm950HobRah4mmrJ90BldXk5w/wFOApwFPQrRTqaQqUIxeqyZfQvEO30NQ0hOZJ0XXzqAhMCQLj/UMHacoaxVPCTTAIqqpLDWl7G7zU3wovzRuq4ME4fS3FeZC64jw2SoP2oWcw61oEN8EgaLhokWYqrVAXlabSutLizNRZxXm3N8X3ClEctI8irZG0RtIaSWsoraG0htKaSis0s5zSm0UU3zeg4lNbL0vfe1OxSmsq7Xh3kuYkzUmaozRHaY7SHKW5Siv0JjDFU126SmsF54WmIsrxdJTWSVonaZ2kdZTWUVpHaR2ldZRWAnyhN/0pxtJun23F970hOJ0dk5TijWK8DXKGQd4GOhvobKAzDPOGad5KnC90qcmS6LvS7LImuLQZpXqbpIxyvU1UNlHZRGWY7W2isonKJior+b5QCWRWEn4XerPGOdsU8m/qIxgEi7GFxhYaW2hsobGFxhYaW2isRP1CJevfW1BPJaX9m1sIBsEiDBP//7//SB1pRzqQTqRYmWFphrXdFjuYcn/CTTAIFmOBxgKNBRoLNBZoLNBYoLFAY4HGyh+Ah+anRXATDIKGi6qwSxtSR9qRDqQTKVZmWJphbS9hf/8B", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Placed Time" = _t, #"Fulfillment Time" = _t, Orders = _t]),
    #"Filtered Rows" = Table.SelectRows(Source, each ([Orders] <> "0")),
    #"Changed Type" = Table.TransformColumnTypes(#"Filtered Rows",{{"Placed Time", type time}, {"Fulfillment Time", type time}, {"Orders", Int64.Type}}),
    #"Added Custom" = Table.AddColumn(#"Changed Type", "Order Number", each {1..[Orders]}),
    #"Expanded Order Number" = Table.ExpandListColumn(#"Added Custom", "Order Number"),
    #"Removed Other Columns" = Table.SelectColumns(#"Expanded Order Number",{"Placed Time", "Fulfillment Time", "Order Number"}),
    #"Added Custom1" = Table.AddColumn(#"Removed Other Columns", "Processed", each null),
    Filled = List.Accumulate({4..19},#"Added Custom1",(state,current)=>FillShift(state,current)),    
    #"Changed Type1" = Table.TransformColumnTypes(#"Filled",{{"Processed", type time}})

in
    #"Changed Type1"

This is almost exactly what I need!!!! Maybe I'm not understanding the full implications of the Table.Sort step but, for example, why does it not bump down some of the 11 am orders that were placed by 4 am to be processed at 5 am with the rest? Ideally I want to use up my capacity early in the day to avoid the late orders in the afternoon. 

 

In other words, how do I make it prioritize being on-time until inevitable?

li3112_0-1658514798647.png

 

That is a problem with your sort logic.  Remember, for each remaining slot the orders are sorted first by fulfillment time and then by placed time.

 

lbendlin_0-1658521797982.png

 

 

This will create gaps in the 140 orders bucket that cannot be compensated unless you change the logic.

 

The proper logic should be:  fill each production slot with orders based on fulfillment time first, placed time second until either:

- the 140 limit is hit

- the placed time limit is hit

- there are no more orders

 

This likely requires a total refactoring of the code, and the use of recursive functions - something you rather want to avoid, as this increases the complexity from "82 orders placed"  to "touch every single order"

You are absolutely correct. You've helped me figure out the logic behind what I was already doing on paper by instinct 🙂

 

I don't mind the query running slower. What modifications would i need to make to the dataset for the recursive function? What would that look like? 

I managed to do it without recursions.  See the comments in the code.

 

let
    //process new timeslot
    FillShift = (tbl, slot) => let
        // buffer the table
        buf = Table.Buffer(tbl),
        // save away already processed entries
        prc = Table.SelectRows(buf,each [Processed]<> null),
        // items that need processing. consider only placed orders.
        src=Table.SelectRows(buf,each [Processed] = null and [Placed Time] <= #time(slot,0,0)),
        // defer (orders were placed later)
        def = Table.SelectRows(buf,each [Processed] = null and [Placed Time] > #time(slot,0,0)),
        // sort by fulfilled, then placed
        srt = Table.Sort(src,{{"Fulfillment Time", Order.Ascending}, {"Placed Time", Order.Ascending}}),
        // add index
        idx = Table.AddIndexColumn(srt, "Index", 0, 1, Int64.Type),
        // tag lines to process
        rep = Table.ReplaceValue(idx,null,each if [Placed Time]<=#time(slot,0,0) and [Index]<140 then #time(slot,0,0) else null,Replacer.ReplaceValue,{"Processed"}),
        // drop index
        res = Table.RemoveColumns(rep,{"Index"})
        in  
        //concatenate result with saved portions
        prc & res & def,
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("bZdLkiMhDAXv4nUvLMRPc5WOuf81ZlGIqafIFeEMTKG0oJ5/fz/9z/f7+fmsZ2jx+ftz4T6DvWGcL3zf0L7POGWq2TOOKbQ9Y29C/dAutNO+bDyj67rzPG0Jzcp0Zzv3IvTU5m/YTmlPvUN0CdvAzoLjzYwWTFUmsBF0gp3WHASPpSaQ6jEqKBUJVEUTFE1RZG8WMC8VycRU5AIbQac1O80cBLORBC6auelBVJEqWqBoQRctUeRvpoqWKmpDaLaRznWkx9IUeBtK6LUtNOsQeKrqAuPdiYs8bfC0wdOGVtrqqQs0mtlE3i6altB7Pwm950HobRah4mmrJ90BldXk5w/wFOApwFPQrRTqaQqUIxeqyZfQvEO30NQ0hOZJ0XXzqAhMCQLj/UMHacoaxVPCTTAIqqpLDWl7G7zU3wovzRuq4ME4fS3FeZC64jw2SoP2oWcw61oEN8EgaLhokWYqrVAXlabSutLizNRZxXm3N8X3ClEctI8irZG0RtIaSWsoraG0htKaSis0s5zSm0UU3zeg4lNbL0vfe1OxSmsq7Xh3kuYkzUmaozRHaY7SHKW5Siv0JjDFU126SmsF54WmIsrxdJTWSVonaZ2kdZTWUVpHaR2ldZRWAnyhN/0pxtJun23F970hOJ0dk5TijWK8DXKGQd4GOhvobKAzDPOGad5KnC90qcmS6LvS7LImuLQZpXqbpIxyvU1UNlHZRGWY7W2isonKJior+b5QCWRWEn4XerPGOdsU8m/qIxgEi7GFxhYaW2hsobGFxhYaW2isRP1CJevfW1BPJaX9m1sIBsEiDBP//7//SB1pRzqQTqRYmWFphrXdFjuYcn/CTTAIFmOBxgKNBRoLNBZoLNBYoLFAY4HGyh+Ah+anRXATDIKGi6qwSxtSR9qRDqQTKVZmWJphbS9hf/8B", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Placed Time" = _t, #"Fulfillment Time" = _t, Orders = _t]),
    #"Filtered Rows" = Table.SelectRows(Source, each ([Orders] <> "0")),
    #"Changed Type" = Table.TransformColumnTypes(#"Filtered Rows",{{"Placed Time", type time}, {"Fulfillment Time", type time}, {"Orders", Int64.Type}}),
    #"Added Custom" = Table.AddColumn(#"Changed Type", "Order Number", each {1..[Orders]}),
    #"Expanded Order Number" = Table.ExpandListColumn(#"Added Custom", "Order Number"),
    #"Removed Other Columns" = Table.SelectColumns(#"Expanded Order Number",{"Placed Time", "Fulfillment Time", "Order Number"}),
    #"Added Custom1" = Table.AddColumn(#"Removed Other Columns", "Processed", each null),
    Filled = List.Accumulate({4..19},#"Added Custom1",(state,current)=>FillShift(state,current)),    
    #"Changed Type1" = Table.TransformColumnTypes(#"Filled",{{"Processed", type time}})

in
    #"Changed Type1"

@lbendlin Spot on what i needed! Thanks so much for the help!

 

Can you walk me thorugh how the FillShift function works? I've been playing around invoking the function with test parameters and I can't seem to get the correct format. I just want to understand how it knows what the slot is at any point 

 

li3112_0-1658765103729.png

 

I had alredy added the comments in the function

    FillShift = (tbl, slot) => let
        // buffer the table
        buf = Table.Buffer(tbl),
        // save away already processed entries
        prc = Table.SelectRows(buf,each [Processed]<> null),
        // items that need processing. consider only placed orders.
        src=Table.SelectRows(buf,each [Processed] = null and [Placed Time] <= #time(slot,0,0)),
        // defer (orders were placed later)
        def = Table.SelectRows(buf,each [Processed] = null and [Placed Time] > #time(slot,0,0)),
        // sort by fulfilled, then placed
        srt = Table.Sort(src,{{"Fulfillment Time", Order.Ascending}, {"Placed Time", Order.Ascending}}),
        // add index
        idx = Table.AddIndexColumn(srt, "Index", 0, 1, Int64.Type),
        // tag lines to process
        rep = Table.ReplaceValue(idx,null,each if [Placed Time]<=#time(slot,0,0) and [Index]<140 then #time(slot,0,0) else null,Replacer.ReplaceValue,{"Processed"}),
        // drop index
        res = Table.RemoveColumns(rep,{"Index"})
        in  
        //concatenate result with saved portions
        prc & res & def

 

The function expects a table with the four required columns Placed Time, Fulfillment Time, Order identifier and Processed.  (The order identifier is generated when exploding the number of received orders into a list)

 

The function also expects a time slot for which the computation needs to happen . That is the hour number.

 

First step is to buffer the entire table to improve query performance

Next step is to save away the portion of the orders (prc) that have already been allocated (that have a Processed value)

Then we identify all orders that can be processed - meaning the Placed Time is before or equal to the time slot

Then we save away (defer) all orders that cannot yet be processed as the Placed Time is after the time slot

Next we sort the eligible orders by fulfillment time and placed time

Then we tag the first 140 orders (or less) with the processed time stamp of the current slot

And finally we combine the three parts back together, ready for the next timeslot.

 

The function is then called for each available timeslot (from 4 to 19) sequentially. 

 

lbendlin
Super User
Super User

That's likely not something you can do in DAX.  It might be possible in Power Query.

Please provide sanitized sample data that fully covers your issue. If you paste the data into a table in your post or use one of the file services it will be easier to assist you. I cannot use screenshots of your source data.

Please show the expected outcome based on the sample data you provided. Screenshots of the expected outcome are ok.

https://community.powerbi.com/t5/Desktop/How-to-Get-Your-Question-Answered-Quickly/m-p/1447523

This is my dataset. I calculated the Running Total wit the following formula:

 

calculate(sum(table[Orders]), all(table),table[Placed Time] <= earlier(table[Placed Time ]), table[Fulfillment Time] <= earlier(table[Fulfillment Time]))

 

 

Placed Time Fulfillment TimeOrders
4:007:0029
4:008:0081
4:009:0070
4:0010:0061
4:0011:0056
4:0012:0042
4:0013:0044
4:0014:0029
4:0015:0036
4:0016:0057
4:0017:0021
4:0018:0012
4:0019:003
4:0020:000
5:007:000
5:008:000
5:009:005
5:0010:000
5:0011:001
5:0012:001
5:0013:001
5:0014:000
5:0015:000
5:0016:002
5:0017:000
5:0018:000
5:0019:000
5:0020:000
6:007:000
6:008:001
6:009:000
6:0010:001
6:0011:003
6:0012:003
6:0013:000
6:0014:003
6:0015:003
6:0016:005
6:0017:003
6:0018:000
6:0019:000
6:0020:000
7:007:000
7:008:000
7:009:003
7:0010:001
7:0011:0025
7:0012:0011
7:0013:0011
7:0014:006
7:0015:0015
7:0016:0013
7:0017:008
7:0018:004
7:0019:002
7:0020:000
8:007:000
8:008:000
8:009:000
8:0010:004
8:0011:000
8:0012:0025
8:0013:0017
8:0014:0014
8:0015:0020
8:0016:0018
8:0017:008
8:0018:005
8:0019:000
8:0020:001
9:007:000
9:008:000
9:009:000
9:0010:000
9:0011:006
9:0012:003
9:0013:0037
9:0014:0028
9:0015:0025
9:0016:0023
9:0017:007
9:0018:002
9:0019:004
9:0020:001
10:007:000
10:008:000
10:009:000
10:0010:000
10:0011:000
10:0012:006
10:0013:003
10:0014:0033
10:0015:0033
10:0016:0027
10:0017:0024
10:0018:008
10:0019:003
10:0020:000
11:007:000
11:008:000
11:009:000
11:0010:000
11:0011:000
11:0012:000
11:0013:006
11:0014:004
11:0015:0033
11:0016:0033
11:0017:0032
11:0018:0010
11:0019:006
11:0020:000
12:007:000
12:008:000
12:009:000
12:0010:000
12:0011:000
12:0012:000
12:0013:000
12:0014:009
12:0015:0011
12:0016:0017
12:0017:0040
12:0018:0013
12:0019:006
12:0020:002
13:007:000
13:008:000
13:009:000
13:0010:000
13:0011:000
13:0012:000
13:0013:000
13:0014:000
13:0015:0014
13:0016:0011
13:0017:0021
13:0018:0022
13:0019:003
13:0020:002
14:007:000
14:008:000
14:009:000
14:0010:000
14:0011:000
14:0012:000
14:0013:000
14:0014:000
14:0015:000
14:0016:0010
14:0017:000
14:0018:0018
14:0019:0017
14:0020:003
15:007:000
15:008:000
15:009:000
15:0010:000
15:0011:000
15:0012:000
15:0013:000
15:0014:000
15:0015:000
15:0016:000
15:0017:0010
15:0018:004
15:0019:0032
15:0020:002
16:007:000
16:008:000
16:009:000
16:0010:000
16:0011:000
16:0012:000
16:0013:000
16:0014:000
16:0015:000
16:0016:000
16:0017:000
16:0018:005
16:0019:004
16:0020:0011
17:007:000
17:008:000
17:009:000
17:0010:000
17:0011:000
17:0012:000
17:0013:000
17:0014:000
17:0015:000
17:0016:000
17:0017:000
17:0018:000
17:0019:002
17:0020:003
18:007:000
18:008:000
18:009:000
18:0010:000
18:0011:000
18:0012:000
18:0013:000
18:0014:000
18:0015:000
18:0016:000
18:0017:000
18:0018:000
18:0019:000
18:0020:0010
19:007:000
19:008:000
19:009:000
19:0010:000
19:0011:000
19:0012:000
19:0013:000
19:0014:000
19:0015:000
19:0016:000
19:0017:000
19:0018:000
19:0019:000
19:0020:000
20:007:000
20:008:000
20:009:000
20:0010:000
20:0011:000
20:0012:000
20:0013:000
20:0014:000
20:0015:000
20:0016:000
20:0017:000
20:0018:000
20:0019:000
20:0020:000

Thank you for providing the sample data.  Please walk me through the logic again for the 4am row based on the sample data.

lbendlin_0-1658253730068.png

Will any order placed before 7am  be put on hold until then? 

lbendlin_1-1658253999861.png

Does 4am actually mean "overnight, after 8pm of the prior day" ?

yes, 4 am placed time just means overnight or any day prior to the fulfillment date. I just group it all under 4 am for simplicity

The logic is the following:

For every 'Order Placed' timeslot, I can only fulfill 140 orders (Sorry, my original message and formulas said 100, but the screenshot calculations are right). From your screenshot, that would mean I can fulfill all the orders due at 7 am (29), all the orders due at 8 am (81), and only 30 out of the 70 that are due at 9 am (and so on so forth till the end of the row, as the screenshot in my original post shows).

 

Once 5 am hits, I would takle first the 40 orders for 9 am that i didnt get to in the previous timeslot plus the other 5 orders underneath. and so on so forth till the end of the row.

 

li3112_0-1658254740548.png

In summary, for every "placed time" (matrix rows), I can only fulfill 140 orders, but since I have more orders than I have capacity, I need to make up for it on the next timeslot together with the orders at that timeslot. My goal with this matrix is to show where the whole system breaks and at how many orders. Hope the context helps! Let me know if you have other questions! Thanks for the help!

What is the priority?  The orders placed at the 5am slot or the leftover orders from the 4am slot?

It doesn't really matter as long as we add the leftover orders to the next time slot.

The priority works vertically in the order orders are due. So at 4 am we were only able to fully the 7 and 8 am timeslots. Up next I care the most about the orders due at 9.

by the way here is my version of the running total for the Placed date:

RT = 
var f = max('Table'[Fulfillment Time]) return CALCULATE(sum('Table'[Orders]),'Table'[Fulfillment Time]<=f)

 

lbendlin_3-1658255074007.png

 

Thank you, by the way, this is what I expect the math at 5 am to be like:

40 +5+101=146.     146-140=6.     

 

Then, I would have 6 orders leftover (due at 10 am) that get carried out to the next time slot (6 am).

Yeah, this is back to my initial comment. Not something you can do in DAX as it has no concept of global variables. 

 

Would it be ok to implement in Power Query?  That would mean the result cannot be modified by filtering.

Power Query is not ideal but it should get the job done!

At the end of the day (pardon the pun)  does it really matter when the order was placed?  Ca we assume that your production starts at 4am and can handle up to 140 orders per hour until close at 8pm ?

Yes that's a fair point! It doesn't matter as far as priorities go.

Helpful resources

Announcements
LearnSurvey

Fabric certifications survey

Certification feedback opportunity for the community.

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.