Forum Discussion
Running count starting over when the counted value is not present previous date
- 5 years ago
Hi Anonymous ,
you have to modify the code like so:
let fnIdx = (myTable as table) => let partition = Table.Buffer(myTable), dates = AllDatesWthIndex, #"Merged Queries" = Table.NestedJoin(partition, {"Date"}, dates, {"Date"}, "Changed Type", JoinKind.LeftOuter), #"Expanded Changed Type" = Table.ExpandTableColumn(#"Merged Queries", "Changed Type", {"Index"}, {"Index"}), Custom1 = Table.ToColumns(#"Expanded Changed Type"), IndexedTable = Table.Buffer( Table.Sort(Table.FromColumns(Custom1 & { {null} & List.RemoveLastN(List.Last(Custom1),1) }, {"ID", "Date", "Warehouse", "Index", "PrevIndex"}), "Index") ), Initial = IndexedTable{0} & [Counter = 0, Idx = 1], Custom3 = List.Generate( ()=> Initial, each [Counter] < Table.RowCount(IndexedTable), each [ Counter = [Counter] + 1, Idx = if IndexedTable[Index]{Counter} - IndexedTable[PrevIndex]{Counter} = 1 then [Idx] + 1 else 1 ]), #"Converted to Table" = Table.FromList(Custom3, Splitter.SplitByNothing(), null, null, ExtraValues.Error), #"Expanded Column1" = Table.ExpandRecordColumn(#"Converted to Table", "Column1", {"Idx"}, {"Idx"})[Idx], Custom2 = Table.FromColumns( Table.ToColumns( IndexedTable ) & { #"Expanded Column1" }) in Custom2, Origine = dataSample, Selection = Table.FirstN(Origine,CountRecordsImke), AllDates=Table.Buffer( Table.Distinct(Table.SelectColumns(Selection, {"Date"})) ), AllDatesWthIndex = Table.AddIndexColumn(AllDates, "Index", 0, 1, Int64.Type), AllProducts = Origine, #"Grouped Rows" = Table.Group(Selection, {"ProductID"}, {{"ProductPartition", each _}}), #"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each fnIdx([ProductPartition])), #"Expanded Custom" = Table.ExpandTableColumn(#"Added Custom", "Custom", {"Column2", "Column3", "Column6"}, {"Date", "Warehouse", "Index"}), #"Removed Columns" = Table.RemoveColumns(#"Expanded Custom",{"ProductPartition"}), #"Changed Type" = Table.TransformColumnTypes(#"Removed Columns",{{"Date", type date}, {"Index", Int64.Type}, {"ProductID", type text}}) in #"Changed Type"Please also see the file attached.
- Anonymous5 years ago
Hi Anonymous .
First of all, I congratulate you for being an excellent tester. But then I tell you that I do not agree on the expected result that you indicate on the demo_real file. I am attaching a screen with the result that, from your previous descriptions, I would expect. I have corrected the script and am always curious to know how long it takes to work your complete file.
The script assumes that the data are sorted by date, otherwise you have to add a statement to that effect at the beginning.
I ran the last script on the first 25k records of your file. in my laptop it took about 6 '.
You're genius!
And hopefully the last thing - how do I add back the warehouse column please? The way I meant it was that it has to remain in the final table. I suppose there's no need for any big change in your script but I'm unable to figure out how to run it with original 3-column table (ProductID, Date, Warehouse). The ProductID-Date-Warehouse combination is supposed to be the same as in the raw table. Is there any easy way how to modify it so it remains there? Sorry that I did't explain in clearly in the last reply. I already feel bad for not being able to make that change on my own and bothering you with all that but my guess is it should not be anything too hardt though my skills are currently insufficient.
Thank you!
Honzik
- Anonymous5 years agoNot applicable
Anonymous
exaggerated. Try this one should be even faster than my last one.
I don't know if faster than ImkeF 's, but this doesn't use List.generate, and you caùn execute all the steps (except the one who does the grouping) with the UI
You have also the extra column [Warehouse]
let Origine = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUTLTM9QzMjAyUIrViVZyQhdwRhcAabFE12KJrgVFwAWbFkNDdFMNTTDUmKCbiyoC1mWKocYU3XIkkVgA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [prid = _t, date = _t]), #"Modificato tipo" = Table.TransformColumnTypes(Origine,{{"date", type date}}), #"Raggruppate righe" = Table.Group(#"Modificato tipo", {"date"}, {{"all", each _, type table [prid=nullable text, date=nullable text]}}), #"Aggiunta colonna indice" = Table.AddIndexColumn(#"Raggruppate righe", "Indice", 1, 1, Int64.Type), #"Tabella all espansa" = Table.ExpandTableColumn(#"Aggiunta colonna indice", "all", {"prid"}, {"prid"}), #"Ordinate righe" = Table.Sort(#"Tabella all espansa",{{"prid", Order.Ascending}, {"Indice", Order.Ascending}}), #"Aggiunta colonna indice1" = Table.AddIndexColumn(#"Ordinate righe", "dist", 0, 1, Int64.Type), #"Raggruppate righe1" = Table.Group(#"Aggiunta colonna indice1", "prid", {"ccc",each Table.Group(_, {"Indice","dist"}, {{"all", (c)=> Table.AddIndexColumn(c,"counter",1,1)}},GroupKind.Local,(x,y)=>Number.From(y[Indice]-x[Indice]>y[dist]))}), #"Tabella ccc espansa1" = Table.ExpandTableColumn(#"Raggruppate righe1", "ccc", {"Indice", "dist", "all"}, {"Indice", "dist", "all"}), #"Tabella all espansa1" = Table.ExpandTableColumn(#"Tabella ccc espansa1", "all", {"date", "counter"}, {"date", "counter"}), #"Ordinate righe1" = Table.Sort(#"Tabella all espansa1",{{"date", Order.Ascending}}), #"Rimosse colonne" = Table.RemoveColumns(#"Ordinate righe1",{"Indice", "dist"}), #"Riordinate colonne" = Table.ReorderColumns(#"Rimosse colonne",{"date", "prid", "counter"}) in #"Riordinate colonne"- Anonymous5 years agoNot applicable
Hi Anonymous,
Thank you for that. I tested it on my own but for some reason the last script (meaning the one in "ABCricomincia3b.pbix" file and teble including Warehouse column). I randomly picked ProductID and usualy the result was this:
I might have done something wrong but the snapshot was taken from the file unchanged (mentioned above). Idk what exactly caused that but for thos moment I'll use the solution by ImkeF. Thank you for all the help you provided, I appreciate it a lot.
Honzik
- Anonymous5 years agoNot applicable
The problem seems to be due to this expression, which should be equivalent to the second, but for some strange reason it doesn't work. Try now. I'm curious to know if you can measure the execution speed.
= Table.Sort(#"Tabella all espansa",{{"ProductID", Order.Ascending}, {"Indice", Order.Ascending}})
= Table.Sort(#"Tabella all espansa",{{"ProductID", Order.Ascending}, {"Date", Order.Ascending}})