Forum Discussion
Running count starting over when the counted value is not present previous date
- 5 years ago
Hi Anonymous ,
you have to modify the code like so:
let fnIdx = (myTable as table) => let partition = Table.Buffer(myTable), dates = AllDatesWthIndex, #"Merged Queries" = Table.NestedJoin(partition, {"Date"}, dates, {"Date"}, "Changed Type", JoinKind.LeftOuter), #"Expanded Changed Type" = Table.ExpandTableColumn(#"Merged Queries", "Changed Type", {"Index"}, {"Index"}), Custom1 = Table.ToColumns(#"Expanded Changed Type"), IndexedTable = Table.Buffer( Table.Sort(Table.FromColumns(Custom1 & { {null} & List.RemoveLastN(List.Last(Custom1),1) }, {"ID", "Date", "Warehouse", "Index", "PrevIndex"}), "Index") ), Initial = IndexedTable{0} & [Counter = 0, Idx = 1], Custom3 = List.Generate( ()=> Initial, each [Counter] < Table.RowCount(IndexedTable), each [ Counter = [Counter] + 1, Idx = if IndexedTable[Index]{Counter} - IndexedTable[PrevIndex]{Counter} = 1 then [Idx] + 1 else 1 ]), #"Converted to Table" = Table.FromList(Custom3, Splitter.SplitByNothing(), null, null, ExtraValues.Error), #"Expanded Column1" = Table.ExpandRecordColumn(#"Converted to Table", "Column1", {"Idx"}, {"Idx"})[Idx], Custom2 = Table.FromColumns( Table.ToColumns( IndexedTable ) & { #"Expanded Column1" }) in Custom2, Origine = dataSample, Selection = Table.FirstN(Origine,CountRecordsImke), AllDates=Table.Buffer( Table.Distinct(Table.SelectColumns(Selection, {"Date"})) ), AllDatesWthIndex = Table.AddIndexColumn(AllDates, "Index", 0, 1, Int64.Type), AllProducts = Origine, #"Grouped Rows" = Table.Group(Selection, {"ProductID"}, {{"ProductPartition", each _}}), #"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each fnIdx([ProductPartition])), #"Expanded Custom" = Table.ExpandTableColumn(#"Added Custom", "Custom", {"Column2", "Column3", "Column6"}, {"Date", "Warehouse", "Index"}), #"Removed Columns" = Table.RemoveColumns(#"Expanded Custom",{"ProductPartition"}), #"Changed Type" = Table.TransformColumnTypes(#"Removed Columns",{{"Date", type date}, {"Index", Int64.Type}, {"ProductID", type text}}) in #"Changed Type"Please also see the file attached.
- Anonymous5 years ago
Hi Anonymous .
First of all, I congratulate you for being an excellent tester. But then I tell you that I do not agree on the expected result that you indicate on the demo_real file. I am attaching a screen with the result that, from your previous descriptions, I would expect. I have corrected the script and am always curious to know how long it takes to work your complete file.
The script assumes that the data are sorted by date, otherwise you have to add a statement to that effect at the beginning.
Anonymous
exaggerated. Try this one should be even faster than my last one.
I don't know if faster than ImkeF 's, but this doesn't use List.generate, and you caùn execute all the steps (except the one who does the grouping) with the UI
You have also the extra column [Warehouse]
let
Origine = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUTLTM9QzMjAyUIrViVZyQhdwRhcAabFE12KJrgVFwAWbFkNDdFMNTTDUmKCbiyoC1mWKocYU3XIkkVgA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [prid = _t, date = _t]),
#"Modificato tipo" = Table.TransformColumnTypes(Origine,{{"date", type date}}),
#"Raggruppate righe" = Table.Group(#"Modificato tipo", {"date"}, {{"all", each _, type table [prid=nullable text, date=nullable text]}}),
#"Aggiunta colonna indice" = Table.AddIndexColumn(#"Raggruppate righe", "Indice", 1, 1, Int64.Type),
#"Tabella all espansa" = Table.ExpandTableColumn(#"Aggiunta colonna indice", "all", {"prid"}, {"prid"}),
#"Ordinate righe" = Table.Sort(#"Tabella all espansa",{{"prid", Order.Ascending}, {"Indice", Order.Ascending}}),
#"Aggiunta colonna indice1" = Table.AddIndexColumn(#"Ordinate righe", "dist", 0, 1, Int64.Type),
#"Raggruppate righe1" = Table.Group(#"Aggiunta colonna indice1", "prid", {"ccc",each Table.Group(_, {"Indice","dist"}, {{"all", (c)=> Table.AddIndexColumn(c,"counter",1,1)}},GroupKind.Local,(x,y)=>Number.From(y[Indice]-x[Indice]>y[dist]))}),
#"Tabella ccc espansa1" = Table.ExpandTableColumn(#"Raggruppate righe1", "ccc", {"Indice", "dist", "all"}, {"Indice", "dist", "all"}),
#"Tabella all espansa1" = Table.ExpandTableColumn(#"Tabella ccc espansa1", "all", {"date", "counter"}, {"date", "counter"}),
#"Ordinate righe1" = Table.Sort(#"Tabella all espansa1",{{"date", Order.Ascending}}),
#"Rimosse colonne" = Table.RemoveColumns(#"Ordinate righe1",{"Indice", "dist"}),
#"Riordinate colonne" = Table.ReorderColumns(#"Rimosse colonne",{"date", "prid", "counter"})
in
#"Riordinate colonne"
Hi Anonymous,
Thank you for that. I tested it on my own but for some reason the last script (meaning the one in "ABCricomincia3b.pbix" file and teble including Warehouse column). I randomly picked ProductID and usualy the result was this:
I might have done something wrong but the snapshot was taken from the file unchanged (mentioned above). Idk what exactly caused that but for thos moment I'll use the solution by ImkeF. Thank you for all the help you provided, I appreciate it a lot.
Honzik
- Anonymous5 years agoNot applicable
The problem seems to be due to this expression, which should be equivalent to the second, but for some strange reason it doesn't work. Try now. I'm curious to know if you can measure the execution speed.
= Table.Sort(#"Tabella all espansa",{{"ProductID", Order.Ascending}, {"Indice", Order.Ascending}})
= Table.Sort(#"Tabella all espansa",{{"ProductID", Order.Ascending}, {"Date", Order.Ascending}})
- Anonymous5 years agoNot applicable
Hi Anonymous,
thank you for the update. The previous issue was solved and it runs very fast. I ran it through small sample of data to test it if it starts counting over when ProductID value is missing the previous present Date and unfortunately it did not work out as expected (see below).
This is the sample a ran it through, just 17 rows (https://drive.google.com/file/d/1ctZSZHt8R8M37I5aaQmAii5leVQrMOW0/view?usp=sharing)
Thanks
Honzik
- Anonymous5 years agoNot applicable
Hi Anonymous .
First of all, I congratulate you for being an excellent tester. But then I tell you that I do not agree on the expected result that you indicate on the demo_real file. I am attaching a screen with the result that, from your previous descriptions, I would expect. I have corrected the script and am always curious to know how long it takes to work your complete file.
The script assumes that the data are sorted by date, otherwise you have to add a statement to that effect at the beginning.
- Anonymous5 years agoNot applicable
Thank you so much for the info, I didn't see that was the issue. Now everything seems to work perfectly and smoothly. It took just 1 or 2 sec tu run through all the data which means the speed is way more than good enough. It is also not complicated so that's awesome for someone who deesn't have much experience with M-code. Once again, thank you fo all your help!
Honzik
- Anonymous5 years agoNot applicable
Hi Anonymous ,
change the statement making the groupings is like the following:
= Table.Group(#"Ordinate righe", "ProductID", {"ccc",each Table.Group(Table.AddIndexColumn(_,"dist",0,1), {"Indice","dist"}, {{"all", (c)=> Table.AddIndexColumn(c,"counter",1,1)}},GroupKind.Local,(x,y)=>Number.From(y[Indice]-x[Indice]>y[dist]-x[dist]))})I am glad that you find it easy to follow myr solution. I assure you that from the conceptual point of view (not the strictly implementation one) it is not one of the simplest.
When I have time I try to illustrate in detail the idea behind it.
In this way you will be able to submit it to a more stringent and meaningful check.😁