Check your eligibility for this 50% exam voucher offer and join us for free live learning sessions to get prepared for Exam DP-700.
Get StartedDon't miss out! 2025 Microsoft Fabric Community Conference, March 31 - April 2, Las Vegas, Nevada. Use code MSCUST for a $150 discount. Prices go up February 11th. Register now.
Hi there,
I have a dataset of patients discharged from hospital. This is used to generate a list of patient experience surveys to be sent out. HOWEVER, we only want to send out a survey for the same patient if it's been more than 6 months since they were last surveyed. I have been trying for days to figure out how to achieve this in power query M (since I'm constructing this in a dataflow), but I'm all out of ideas. I can generate a 'ReSurveyDate' column easily in Excel, just can't work out how to get the equivalent in Power Query M. Sample data would be something like:
ID | RecordDate | ReSurveyDate |
1 | 01/01/2024 | 01/07/2024 |
1 | 03/02/2024 | 01/07/2024 |
1 | 04/05/2024 | 01/07/2024 |
1 | 07/09/2024 | 07/03/2025 |
1 | 10/10/2024 | 07/03/2025 |
1 | 21/04/2025 | 21/10/2025 |
1 | 18/06/2025 | 21/10/2025 |
1 | 20/07/2025 | 21/10/2025 |
1 | 30/10/2026 | 30/04/2027 |
2 | 07/02/2024 | 07/08/2024 |
2 | 12/04/2024 | 07/08/2024 |
2 | 16/09/2024 | 16/03/2025 |
3 | 05/01/2024 | 05/07/2024 |
3 | 18/04/2024 | 05/07/2024 |
3 | 01/05/2024 | 05/07/2024 |
4 | 01/03/2024 | 01/09/2024 |
4 | 18/05/2024 | 01/09/2024 |
5 | 11/05/2024 | 11/11/2024 |
In Excel, I can simply set the ReSurveyDate for the first record for each ID group as the RecordDate+6 months, then each subsequent one uses the formula =IF(RecordDate<Previous ReSurveyDate, ReSurveyDate,RecordDate+6 months)
Once I have the ReSurveyDate generated I can then easily just select the first record within each group of ID+ReSurveyDate.
Any help on how to generate the ReSurveyDate as above would be greatly appreciated!!!
EDIT: Sorry, I should have mentioned that the data is currently sitting in several .csv files (approx 3 months data in each file) that get imported and combined in PowerBI into a single file. So there's no 'ReSurveyDate' field in the .csv data, it needs to be generated within PowerBI. I've just included that as an example of the expected output for that field 🙂
Solved! Go to Solution.
Try this in Power Query
let
// Load your data source
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
// Convert RecordDate and ReSurveyDate columns to Date type
ChangedType = Table.TransformColumnTypes(Source,{{"RecordDate", type date}, {"ReSurveyDate", type date}}),
// Sort the data by ID and RecordDate
SortedRows = Table.Sort(ChangedType,{{"ID", Order.Ascending}, {"RecordDate", Order.Ascending}}),
// Add a new column for ReSurveyDate calculation
AddReSurveyDate = Table.AddColumn(SortedRows, "ReSurveyDate", each
let
currentId = [ID],
currentRecordDate = [RecordDate],
previousRows = Table.SelectRows(SortedRows, each [ID] = currentId and [RecordDate] < currentRecordDate),
maxPreviousReSurveyDate = if Table.IsEmpty(previousRows) then null else List.Max(previousRows[ReSurveyDate])
in
if maxPreviousReSurveyDate = null or currentRecordDate > maxPreviousReSurveyDate then
Date.AddMonths(currentRecordDate, 6)
else
maxPreviousReSurveyDate
),
// Remove the old ReSurveyDate column if you want to replace it with the new one
RemoveOldColumn = Table.RemoveColumns(AddReSurveyDate,{"ReSurveyDate"}),
// Rename the new ReSurveyDate column
RenameColumn = Table.RenameColumns(RemoveOldColumn,{{"ReSurveyDate", "ReSurveyDate"}}),
// Group by ID and ReSurveyDate and keep the first row in each group
GroupedRows = Table.Group(RenameColumn, {"ID", "ReSurveyDate"}, {{"FirstRecord", each Table.FirstN(_, 1), type table [ID=Int64.Type, RecordDate=Date.Type, ReSurveyDate=Date.Type]}}),
// Expand the grouped table to get the final result
ExpandedRows = Table.ExpandTableColumn(GroupedRows, "FirstRecord", {"ID", "RecordDate", "ReSurveyDate"})
in
ExpandedRows
Try this in Power Query
let
// Load your data source
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
// Convert RecordDate and ReSurveyDate columns to Date type
ChangedType = Table.TransformColumnTypes(Source,{{"RecordDate", type date}, {"ReSurveyDate", type date}}),
// Sort the data by ID and RecordDate
SortedRows = Table.Sort(ChangedType,{{"ID", Order.Ascending}, {"RecordDate", Order.Ascending}}),
// Add a new column for ReSurveyDate calculation
AddReSurveyDate = Table.AddColumn(SortedRows, "ReSurveyDate", each
let
currentId = [ID],
currentRecordDate = [RecordDate],
previousRows = Table.SelectRows(SortedRows, each [ID] = currentId and [RecordDate] < currentRecordDate),
maxPreviousReSurveyDate = if Table.IsEmpty(previousRows) then null else List.Max(previousRows[ReSurveyDate])
in
if maxPreviousReSurveyDate = null or currentRecordDate > maxPreviousReSurveyDate then
Date.AddMonths(currentRecordDate, 6)
else
maxPreviousReSurveyDate
),
// Remove the old ReSurveyDate column if you want to replace it with the new one
RemoveOldColumn = Table.RemoveColumns(AddReSurveyDate,{"ReSurveyDate"}),
// Rename the new ReSurveyDate column
RenameColumn = Table.RenameColumns(RemoveOldColumn,{{"ReSurveyDate", "ReSurveyDate"}}),
// Group by ID and ReSurveyDate and keep the first row in each group
GroupedRows = Table.Group(RenameColumn, {"ID", "ReSurveyDate"}, {{"FirstRecord", each Table.FirstN(_, 1), type table [ID=Int64.Type, RecordDate=Date.Type, ReSurveyDate=Date.Type]}}),
// Expand the grouped table to get the final result
ExpandedRows = Table.ExpandTableColumn(GroupedRows, "FirstRecord", {"ID", "RecordDate", "ReSurveyDate"})
in
ExpandedRows
Argh, I just realised this is not working because I think it was assumed I already had a 'ReSurveyDate' column? As an example, here's one set of records, where you can see row index 7 is incorrect. The ReSurveyDate should only be updated if it's been 6 months+ since the previous ReSurveyDate. But since the code is used to generate the ReSurveyDate in the first place, I don't know how I would get it to refer to the previous result (rather than the 'first resurvey' field, which just applies to all of the patient's records).
MRN | Disch Date | First resurvey | Index | What I should get | What I do get |
1 | 10/01/2024 | 10/07/2024 | 1 | 10/07/2024 | 10/07/2024 |
1 | 24/02/2024 | 10/07/2024 | 2 | 10/07/2024 | 10/07/2024 |
1 | 21/03/2024 | 10/07/2024 | 3 | 10/07/2024 | 10/07/2024 |
1 | 28/03/2024 | 10/07/2024 | 4 | 10/07/2024 | 10/07/202 |
1 | 09/06/2024 | 10/07/2024 | 5 | 10/07/2024 | 10/07/2024 |
1 | 18/07/2024 | 10/07/2024 | 6 | 18/01/2025 | 18/01/2025 |
1 | 26/08/2024 | 10/07/2024 | 7 | 18/01/2025 | 26/02/2025 |
The code I put in was:
I cannot thankyou enough for this! You have absolutely saved my sanity!
User | Count |
---|---|
117 | |
75 | |
61 | |
50 | |
44 |
User | Count |
---|---|
175 | |
125 | |
60 | |
60 | |
58 |