Forum Discussion
Helper IGrouping consecutive dates into a single row
I’m working on an object hire report that needs to group objects with consecutive borrowed on dates into a single row in a table visual. The available columns are;
- Object No
Borrowed Date
Returned Date
Using SWITCH to calculate the Start Date and End Date dynamically based on a date slicer. What I need to do now is combine those two into a single line when the borrowed date(s) are consecutive days.
In this screen shot, the last two lines need to be combined into a single row with the start date being 18/01/2021 and the end date being 19/01/2021. The dates in the slicer are 01/01/2021 to 30/01/2021 which is why the first lines start and end dates are blank.
The table returned looks like this
Object No Borrowed Date Returned Date Start Date End Date
| 2011-Z267 | 18/12/2020 | 18/12/2020 | 18/12/2020 | 18/12/2020 |
| 2011-Z267 | 8/01/2021 | 16/01/2021 | 8/01/2021 | 16/01/2021 |
| 2011-Z267 | 18/01/2021 | 18/01/2021 | 18/01/2021 | 18/01/2021 |
| 2011-Z267 | 19/01/2021 | 19/01/2021 | 19/01/2021 | 19/01/2021 |
The result I am looking for is this where the last two rows above are combined into a single row as show below. They only need to be combined if the Borrowed Date(s) are consecutive days. If not they remain in a single row for each entry.
Object No Borrowed Date Returned Date Start Date End Date
| 2011-Z267 | 18/12/2020 | 18/12/2020 | 18/12/2020 | 18/12/2020 |
| 2011-Z267 | 8/01/2021 | 16/01/2021 | 8/01/2021 | 16/01/2021 |
| 2011-Z267 | 18/01/2021 | 19/01/2021 |
I will upload a redacted version of the model. I've tried various techniques to get this working correctly but so far have not been successful.
See it all at work in the attached file.
I'm getting a bit lost. The example that you show (2504-Z004) has only one row in the data you've shared, with dates 18/01/2021 - 18/01/2021 , so the result you show is the correct one
Most of the Object No in your data have only one row. I had a look at the one with the larger number or rows (you can see them in the query Object Billing Export (TESTS) that I created to help in the debugging and they seem to work fine.
I uncommented the S2_ and S3_ ( to use the latest versions I'd suggested, you had the previous version active) and included sorting by Returned Date on this step, to get the dates completely sorted out before the crucial step:
Sorted by Object No = Table.Sort(#"Reordered Columns",{{"Object No", Order.Ascending}, {"Borrowed Date", Order.Ascending}, {"Returned Date", Order.Ascending}})plus it looks like the error you got (Error - Unable to convert type null to type logical) was caused by cases like
3433-Z062 that had a null not as the very latest date in Borrowed Date. So I temporarily replaced those nulls with a date far in the future (31/12/9999) so that the row gets placed at the end when sorting ascending (the null woud be placed at the beginning since it is treated as as 0). Then at the end convert that 31/12/9999 back to null
Please accept the solution when done and consider giving a thumbs up if posts are helpful.
Contact me privately for support with any larger-scale BI needs, tutoring, etc.
Cheers
12 Replies
- AlB
Community Champion
Hi matthewkaess
Something very similar was soved on this thread. Check it out:
Please mark the question solved when done and consider giving a thumbs up if posts are helpful.
Contact me privately for support with any larger-scale BI needs, tutoring, etc.
Cheers
matthewkaessThat link is very close to what I am trying to achieve. If the data wasn't inconsistent, this would work. Problem is when the borrowed date and returned date overlap but are still 'continuous'. The first image is the raw data. The second is the result of the query. Notice rows 5-7 below. The problem is row 6 where. Just not sure yet how to change the query to fix this.
- AlB
See it all at work in the attached file.
I'm getting a bit lost. The example that you show (2504-Z004) has only one row in the data you've shared, with dates 18/01/2021 - 18/01/2021 , so the result you show is the correct one
Most of the Object No in your data have only one row. I had a look at the one with the larger number or rows (you can see them in the query Object Billing Export (TESTS) that I created to help in the debugging and they seem to work fine.
I uncommented the S2_ and S3_ ( to use the latest versions I'd suggested, you had the previous version active) and included sorting by Returned Date on this step, to get the dates completely sorted out before the crucial step:
Sorted by Object No = Table.Sort(#"Reordered Columns",{{"Object No", Order.Ascending}, {"Borrowed Date", Order.Ascending}, {"Returned Date", Order.Ascending}})plus it looks like the error you got (Error - Unable to convert type null to type logical) was caused by cases like
3433-Z062 that had a null not as the very latest date in Borrowed Date. So I temporarily replaced those nulls with a date far in the future (31/12/9999) so that the row gets placed at the end when sorting ascending (the null woud be placed at the beginning since it is treated as as 0). Then at the end convert that 31/12/9999 back to null
Please accept the solution when done and consider giving a thumbs up if posts are helpful.
Contact me privately for support with any larger-scale BI needs, tutoring, etc.
Cheers
- matthewkaess
It may have been something I did when redacting the data into an excel table. The actual datasource is PostgreSQL table(s).
As you can see in the screen shot from my live copy with your changes, it looks correct. Very much appreciated! Now onto the DAX problem with those dates and a date slicer. 😮 I may need to hit you up for some help with that too.
Cheers!
- AlB
Ok, we'll have to tweak it a bit, yes.
There's something i don't understand, though. Why are you not merging the two last rows in the expected result you show above? the last two rows should be one rowwith dates:
2021-01-16 2021-01-19
shouldn't it? Otherwise it's inconsistent with what you are doing with the other rows
Please mark the question solved when done and consider giving a thumbs up if posts are helpful.
Contact me privately for support with any larger-scale BI needs, tutoring, etc.
Cheers
- matthewkaess
That is exactly the result I'm looking for but something in the M code logic isn't quite right. The logic section in the query below. I'm just not sure how to change it to cover this scenario.
#"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each let inputT_ = [Grouped],
colStart_ = inputT_[Borrowed Date],
colEnd_ = inputT_[Returned Date],
S1_ = Table.AddIndexColumn(inputT_,"Index",0),
S2_ = Table.AddColumn(S1_, "Date1", each if try colEnd_{[Index]-1}=Date.AddDays([Borrowed Date],-1) otherwise false then null else [Borrowed Date], type date),
S3_ = Table.AddColumn(S2_, "Date2", each if try colStart_{[Index]+1}=Date.AddDays([Returned Date], 1) otherwise false then null else [Returned Date], type date),
#"Filled Up" = Table.FillUp(S3_,{"Date2"}),
#"Filtered Rows" = Table.SelectRows(#"Filled Up", each [Date1] <> null)
- AlB
Try this variation of the solution posted in the other thread S2_ and S3_ are the only steps that have been updated
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("lZPPDoMgDIdfxXg2gf5gKs9iPGyJWXbQLTttbz9g1D9BDXBpjX5f2xS7riQoVVblx0YbyB4bpudUvK7fx3QvxmG8DW/3SpCApMalECo89JVX6EhxgLdrvGX8ktOBmVNIFtQZAkvFgmZXMJMQmMnQ/cK2icODvCmkATZJnYf6tK7PCi1zhsdaAVbQX+HWCPhvdhXyZIkakeNsgYwbxlVaC8sKsV2h1ruC4xUae5gNF9AFpfwPkXN/dR3hm9KnczeJpaPO+x8=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"customer key" = _t, #"Cust source key" = _t, #"member number" = _t, Status = _t, #"Start Date" = _t, #"End Date" = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"customer key", Int64.Type}, {"Cust source key", type text}, {"member number", Int64.Type}, {"Status", type text}, {"Start Date", type date}, {"End Date", type date}}), #"Sorted Rows" = Table.Sort(#"Changed Type",{{"customer key", Order.Ascending}}), #"Grouped Rows" = Table.Group(#"Sorted Rows", {"Cust source key", "member number", "Status"}, {{"Count", each _, type table [customer key=nullable number, Cust source key=nullable text, member number=nullable number, Status=nullable text, Start Date=nullable date, End Date=nullable text]}}), inputT_ = #"Grouped Rows"{[#"Cust source key"="x123",#"member number"=1111,Status="non paying member"]}[Count], colStart_ = inputT_[Start Date], colEnd_ = inputT_[End Date], S1_ = Table.AddIndexColumn(inputT_,"Index",0), S2_ = Table.AddColumn(S1_, "Date1", each if try Number.From([Start Date]-Date.From(colEnd_{[Index]-1}))<=1 otherwise false then null else [Start Date], type date), S3_ = Table.AddColumn(S2_, "Date2", each if try Number.From(Date.From(colStart_{[Index]+1})-[End Date])<=1 otherwise false then null else [End Date], type date), #"Filled Up" = Table.FillUp(S3_,{"Date2"}), #"Filtered Rows" = Table.SelectRows(#"Filled Up", each [Date1] <> null) in #"Filtered Rows"Please mark the question solved when done and consider giving a thumbs up if posts are helpful.
Contact me privately for support with any larger-scale BI needs, tutoring, etc.
Cheers
- matthewkaess
Error - Unable to convert type null to type logical 😞
- AlB
It's working on the data I posted earlier. Share the data you are running it on on an excel file or pbix
Please mark the question solved when done and consider giving a thumbs up if posts are helpful.
Contact me privately for support with any larger-scale BI needs, tutoring, etc.
Cheers
- matthewkaess
Try this link and let me know how you go. Thanks for all your help! Much appreciated.
- AlB
Where is the query? Where is the error?
Please mark the question solved when done and consider giving a thumbs up if posts are helpful.
Contact me privately for support with any larger-scale BI needs, tutoring, etc.
Cheers