Forum Discussion
Transforming Exchange server data with recurring appointments in Power BI/ Power Query
- 9 years ago
When connecting Exchange data, it only has one record for an appointment, no matter it's Single or RecurringMaster. You can expand the attributes columns into the MeetingRequest or Calendar table.
let Source = Exchange.Contents("[email protected]"), Calendar1 = Source{[Name="Calendar"]}[Data], #"Expanded Attributes" = Table.ExpandRecordColumn(Calendar1, "Attributes", {"AppointmentType", "Recurrence"}, {"Attributes.AppointmentType", "Attributes.Recurrence"}), #"Filtered Rows" = Table.SelectRows(#"Expanded Attributes", each ([Attributes.AppointmentType] = "RecurringMaster")), #"Expanded Attributes.Recurrence" = Table.ExpandRecordColumn(#"Filtered Rows", "Attributes.Recurrence", {"StartDate", "EndDate", "Pattern", "Interval"}, {"Attributes.Recurrence.StartDate", "Attributes.Recurrence.EndDate", "Attributes.Recurrence.Pattern", "Attributes.Recurrence.Interval"}) in #"Expanded Attributes.Recurrence"It's not a good practice to have those recurring appointments appeared in the calendar table, because you need to genereate a list of dates between start date and end date for each recurring appointment. It will be a huge table. Then calculate the datediff and mod interval to get those recurring days for each appointment. And it can't work for appointments with NO END DATE. Please see my sample below, I started with above filtered Recurring Appointments records table:
let Source = Exchange.Contents("[email protected]"), Calendar1 = Source{[Name="Calendar"]}[Data], #"Expanded Attributes" = Table.ExpandRecordColumn(Calendar1, "Attributes", {"AppointmentType", "Recurrence"}, {"Attributes.AppointmentType", "Attributes.Recurrence"}), #"Filtered Rows" = Table.SelectRows(#"Expanded Attributes", each ([Attributes.AppointmentType] = "RecurringMaster")), #"Expanded Attributes.Recurrence" = Table.ExpandRecordColumn(#"Filtered Rows", "Attributes.Recurrence", {"StartDate", "EndDate", "Pattern", "Interval"}, {"Attributes.Recurrence.StartDate", "Attributes.Recurrence.EndDate", "Attributes.Recurrence.Pattern", "Attributes.Recurrence.Interval"}), #"Added Custom" = Table.AddColumn(#"Expanded Attributes.Recurrence", "RecurrenceStartDate", each [Attributes.Recurrence.StartDate]), #"Added Custom1" = Table.AddColumn(#"Added Custom", "RecurrenceEndDate", each [Attributes.Recurrence.EndDate]), #"Filtered Rows1" = Table.SelectRows(#"Added Custom1", each ([RecurrenceEndDate] <> null)), #"Changed Type" = Table.TransformColumnTypes(#"Filtered Rows1",{{"RecurrenceStartDate", type date}, {"RecurrenceEndDate", type date}}), #"Added Custom2" = Table.AddColumn(#"Changed Type", "DatesInPeriod", each List.Dates([RecurrenceStartDate],Duration.Days(Duration.From([RecurrenceEndDate]-[RecurrenceStartDate])),#duration(1,0,0,0))), #"Expanded DatesInPeriod" = Table.ExpandListColumn(#"Added Custom2", "DatesInPeriod"), #"Added Custom3" = Table.AddColumn(#"Expanded DatesInPeriod", "DateDiffModInterval", each Number.Mod(Duration.Days(Duration.From([DatesInPeriod]-[RecurrenceStartDate])),7*[Attributes.Recurrence.Interval])), #"Filtered Rows2" = Table.SelectRows(#"Added Custom3", each ([DateDiffModInterval] = 0)) in #"Filtered Rows2"Regards,
Thank you.
Your code helped me a lot. I also found out the correct way to import the Recurrence Calendar into PBI. I just did some options: every month, other month, and every 3 months.
Just leave here in here if anyone needs to take a look.
let
Source = Exchange.Contents("Shared Canlendar"),
Calendar1 = Source{[Name="Calendar"]}[Data],
#"Removed Other Columns" = Table.SelectColumns(Calendar1,{"Subject", "Location", "Start", "End", "Attributes", "Body", "Id"}),
#"Expanded Attributes" = Table.ExpandRecordColumn(#"Removed Other Columns", "Attributes", {"AppointmentType", "Duration", "ExtendedProperties", "LastOccurrence", "Organizer", "Recurrence"}, {"AppointmentType", "Duration", "ExtendedProperties", "LastOccurrence", "Organizer", "Recurrence"}),
#"Filtered Rows" = Table.SelectRows(#"Expanded Attributes", each ([AppointmentType] = "RecurringMaster")),
#"Replaced Value" = Table.ReplaceValue(#"Filtered Rows",null,"meeting",Replacer.ReplaceValue,{"Location"}),
#"Expanded Recurrence" = Table.ExpandRecordColumn(#"Replaced Value", "Recurrence", {"StartDate", "EndDate", "HasEnd", "NumberOfOccurrences", "Pattern", "Interval", "DayOfMonth", "DayOfTheWeek", "DayOfTheWeekIndex", "DaysOfTheWeek", "FirstDayOfWeek", "Month"}, {"Recurrence.StartDate", "Recurrence.EndDate", "Recurrence.HasEnd", "Recurrence.NumberOfOccurrences", "Recurrence.Pattern", "Recurrence.Interval", "Recurrence.DayOfMonth", "Recurrence.DayOfTheWeek", "Recurrence.DayOfTheWeekIndex", "Recurrence.DaysOfTheWeek", "Recurrence.FirstDayOfWeek", "Recurrence.Month"}),
#"Expanded ExtendedProperties" = Table.ExpandRecordColumn(#"Expanded Recurrence", "ExtendedProperties", {"RecurrencePattern"}, {"ExtendedProperties.RecurrencePattern"}),
#"Expanded LastOccurrence" = Table.ExpandRecordColumn(#"Expanded ExtendedProperties", "LastOccurrence", {"End"}, {"LastOccurrence.End"}),
#"Filtered Rows1 - filter Ruccurence schedule" = Table.SelectRows(#"Expanded LastOccurrence", each ([Recurrence.Pattern] = "RelativeMonthlyPattern")),
#"Changed Type" = Table.TransformColumnTypes(#"Filtered Rows1 - filter Ruccurence schedule",{{"LastOccurrence.End", type date}, {"Recurrence.StartDate", type date}, {"Recurrence.EndDate", type date}}),
#"Added Custom - *" = Table.AddColumn(#"Changed Type", "DatesInPeriod", each List.Dates([Recurrence.StartDate],Duration.Days(Duration.From([LastOccurrence.End]-[Recurrence.StartDate])),#duration(1,0,0,0))),
#"Expanded DatesInPeriod" = Table.ExpandListColumn(#"Added Custom - *", "DatesInPeriod"),
#"Removed Other Columns2" = Table.SelectColumns(#"Expanded DatesInPeriod",{"Subject", "Recurrence.Interval", "Recurrence.DayOfTheWeek", "Recurrence.DayOfTheWeekIndex", "DatesInPeriod"}),
#"Filtered Rows3" = Table.SelectRows(#"Removed Other Columns2", each true),
#"Changed Type1" = Table.TransformColumnTypes(#"Filtered Rows3",{{"DatesInPeriod", type date}}),
#"Filtered Rows2" = Table.SelectRows(#"Changed Type1", each true),
#"Removed Other Columns1" = Table.SelectColumns(#"Filtered Rows2",{"Subject", "Recurrence.Interval", "Recurrence.DayOfTheWeek", "Recurrence.DayOfTheWeekIndex", "DatesInPeriod"}),
#"Renamed Columns1" = Table.RenameColumns(#"Removed Other Columns1",{{"DatesInPeriod", "Start"}, {"Recurrence.DayOfTheWeekIndex", "DayOfTheWeekIndex"}, {"Recurrence.DayOfTheWeek", "DayOfTheWeek"}, {"Recurrence.Interval", "Interval"}}),
#"Added Custom7- **" = Table.AddColumn(#"Renamed Columns1", "Custom", each let // set variable for is Order day of week in a month based on the column START
// Monday
FirstMon = Date.StartOfWeek(Date.AddDays(Date.StartOfMonth([Start]),6),1),
SecondMon = Date.StartOfWeek(#date(Date.Year([Start]),Date.Month([Start]),14),Day.Monday),
ThirdMon = Date.StartOfWeek(#date(Date.Year([Start]),Date.Month([Start]),21),Day.Monday),
FourthMon = Date.StartOfWeek(#date(Date.Year([Start]),Date.Month([Start]),28),Day.Monday),
LastMon = Date.StartOfWeek(Date.EndOfMonth([Start]),1),
// Tuesday
FirstTue = Date.StartOfWeek(Date.AddDays(Date.StartOfMonth([Start]),6),2),
SecondTue = Date.StartOfWeek(#date(Date.Year([Start]),Date.Month([Start]),14),Day.Tuesday),
ThirdTue = Date.StartOfWeek(#date(Date.Year([Start]),Date.Month([Start]),21),Day.Tuesday),
FourthTue = Date.StartOfWeek(#date(Date.Year([Start]),Date.Month([Start]),28),Day.Tuesday),
LastTue = Date.StartOfWeek(Date.EndOfMonth([Start]),2),
//Wednesday
FirstWed = Date.StartOfWeek(Date.AddDays(Date.StartOfMonth([Start]),6),3),
SecondWed = Date.StartOfWeek(#date(Date.Year([Start]),Date.Month([Start]),14),Day.Wednesday),
ThirdWed = Date.StartOfWeek(#date(Date.Year([Start]),Date.Month([Start]),21),Day.Wednesday),
FourthWed = Date.StartOfWeek(#date(Date.Year([Start]),Date.Month([Start]),28),Day.Wednesday),
LastWed = Date.StartOfWeek(Date.EndOfMonth([Start]),3),
//Thurday
FirstThu = Date.StartOfWeek(Date.AddDays(Date.StartOfMonth([Start]),6),4),
SecondThu = Date.StartOfWeek(#date(Date.Year([Start]),Date.Month([Start]),14),Day.Thursday),
ThirdThu = Date.StartOfWeek(#date(Date.Year([Start]),Date.Month([Start]),21),Day.Thursday),
FourthThu = Date.StartOfWeek(#date(Date.Year([Start]),Date.Month([Start]),28),Day.Thursday),
LastThu = Date.StartOfWeek(Date.EndOfMonth([Start]),4),
//Friday
FirstFri = Date.StartOfWeek(Date.AddDays(Date.StartOfMonth([Start]),6),5),
SecondFri = Date.StartOfWeek(#date(Date.Year([Start]),Date.Month([Start]),14),Day.Friday),
ThirdFri = Date.StartOfWeek(#date(Date.Year([Start]),Date.Month([Start]),21),Day.Friday),
FourthFri = Date.StartOfWeek(#date(Date.Year([Start]),Date.Month([Start]),28),Day.Friday),
LastFri = Date.StartOfWeek(Date.EndOfMonth([Start]),5)
in // find all the correct recurring date
// Monday
if [DayOfTheWeek] ="Monday" and [DayOfTheWeekIndex] = "First" and [Start] = FirstMon then "Yes" else
if [DayOfTheWeek] ="Monday" and [DayOfTheWeekIndex] = "Second" and [Start] = SecondMon then "Yes" else
if [DayOfTheWeek] ="Monday" and [DayOfTheWeekIndex] = "Third" and [Start] = ThirdMon then "Yes" else
if [DayOfTheWeek] ="Monday" and [DayOfTheWeekIndex] = "Fourth" and [Start] = FourthMon then "Yes" else
if [DayOfTheWeek] ="Monday" and [DayOfTheWeekIndex] = "Last" and [Start] = LastMon then "Yes" else
// Tuesday
if [DayOfTheWeek] ="Tuesday" and [DayOfTheWeekIndex] = "First" and [Start] = FirstTue then "Yes" else
if [DayOfTheWeek] ="Tuesday" and [DayOfTheWeekIndex] = "Second" and [Start] = SecondTue then "Yes" else
if [DayOfTheWeek] ="Tuesday" and [DayOfTheWeekIndex] = "Third" and [Start] = ThirdTue then "Yes" else
if [DayOfTheWeek] ="Tuesday" and [DayOfTheWeekIndex] = "Fourth" and [Start] = FourthTue then "Yes" else
if [DayOfTheWeek] ="Tuesday" and [DayOfTheWeekIndex] = "Last" and [Start] = LastTue then "Yes" else
//Wednesday
if [DayOfTheWeek] ="Wednesday" and [DayOfTheWeekIndex] = "First" and [Start] = FirstWed then "Yes" else
if [DayOfTheWeek] ="Wednesday" and [DayOfTheWeekIndex] = "Second" and [Start] = SecondWed then "Yes" else
if [DayOfTheWeek] ="Wednesday" and [DayOfTheWeekIndex] = "Third" and [Start] = ThirdWed then "Yes" else
if [DayOfTheWeek] ="Wednesday" and [DayOfTheWeekIndex] = "Fourth" and [Start] = FourthWed then "Yes" else
if [DayOfTheWeek] ="Wednesday" and [DayOfTheWeekIndex] = "Last" and [Start] = LastWed then "Yes" else
//Thurday
if [DayOfTheWeek] ="Thursday" and [DayOfTheWeekIndex] = "First" and [Start] = FirstThu then "Yes" else
if [DayOfTheWeek] ="Thursday" and [DayOfTheWeekIndex] = "Second" and [Start] = SecondThu then "Yes" else
if [DayOfTheWeek] ="Thursday" and [DayOfTheWeekIndex] = "Third" and [Start] = ThirdThu then "Yes" else
if [DayOfTheWeek] ="Thursday" and [DayOfTheWeekIndex] = "Fourth" and [Start] = FourthThu then "Yes" else
if [DayOfTheWeek] ="Thursday" and [DayOfTheWeekIndex] = "Last" and [Start] = LastThu then "Yes" else
//Friday
if [DayOfTheWeek] ="Friday" and [DayOfTheWeekIndex] = "First" and [Start] = FirstFri then "Yes" else
if [DayOfTheWeek] ="Friday" and [DayOfTheWeekIndex] = "Second" and [Start] = SecondFri then "Yes" else
if [DayOfTheWeek] ="Friday" and [DayOfTheWeekIndex] = "Third" and [Start] = ThirdFri then "Yes" else
if [DayOfTheWeek] ="Friday" and [DayOfTheWeekIndex] = "Fourth" and [Start] = FourthFri then "Yes" else
if [DayOfTheWeek] ="Friday" and [DayOfTheWeekIndex] = "Last" and [Start] = LastFri then "Yes" else ""),
#"Filtered Rows5" = Table.SelectRows(#"Added Custom7- **", each ([Custom] = "Yes")),
#"Grouped Rows" = Table.Group(#"Filtered Rows5", {"Subject"}, {{"Count", each _, type table [Subject=nullable text, Interval=number, DayOfTheWeek=nullable text, DayOfTheWeekIndex=nullable text, Start=nullable date, 2nd Tuesday=date, 1st Wed=date, 2nd Wed=date, Last Wed=date, Custom=text]}}),
#"Added Custom9" = Table.AddColumn(#"Grouped Rows", "Custom", each Table.AddIndexColumn([Count],"Index",1)),
#"Expanded Custom" = Table.ExpandTableColumn(#"Added Custom9", "Custom", {"Interval", "Start", "Index"}, {"Custom.Interval", "Custom.Start", "Custom.Index"}),
#"Added Custom10 - *" = Table.AddColumn(#"Expanded Custom", "Custom", each if[Custom.Interval] = 4 and (List.Contains({1,5,9},[Custom.Index])) then "Yes" else
if[Custom.Interval] =3 and (List.Contains({1,4,7,10},[Custom.Index])) then "Yes" else
if[Custom.Interval] =2 and (List.Contains({1,3,5,7,9,11},[Custom.Index])) then "Yes" else
if[Custom.Interval] =1 then "Yes" else "No"),
#"Filtered Rows4" = Table.SelectRows(#"Added Custom10 - *", each ([Custom] = "Yes")),
#"Renamed Columns3" = Table.RenameColumns(#"Filtered Rows4",{{"Custom.Start", "Start"}}),
#"Removed Other Columns3" = Table.SelectColumns(#"Renamed Columns3",{"Subject", "Start"}),
#"Added Custom6" = Table.AddColumn(#"Removed Other Columns3", "End", each [Start]),
#"Changed Type2" = Table.TransformColumnTypes(#"Added Custom6",{{"Start", type date}, {"End", type date}})
in
#"Changed Type2"
Hi LongNguyen123! Any chance you could explain how you introduce your variables? I would like to try your solution but honestly don't know where to begin.