Forum Discussion

quincy_p's avatar
quincy_p
Advocate I
5 months ago
Solved

Building Deadlines based off Start, End Dates and Frequency

Hi there - I am trying to build a deadline table for products and when they need to be serviced. 

Every product has a start date, end date and a service frequency so if its a 5 year contract, it will likely have 5 services (10 in other cases). 

 

For example:

Start: 01/01/2023

End: 31/12/2026

Frequency: 12
Deadline 1: 31/12/2023
Deadline 2: 31/12/2024
Deadline 3: 31/12/2025

Deadline 4: 31/12/2026

 

I have the below query but it is not working properly for the PM_Number in some cases and returns as high PM_NUmber = 90 when it should cap at 5. 

What is wrong with this query or how could I improve it? 

 

let
    // 1️⃣ Source
    Source = #"Covered Product ALL",

    // 2️⃣ Ensure required data types
    FixTypes =
        Table.TransformColumnTypes(
            Source,
            {
                {"SVMXC__Start_Date__c", type date},
                {"SVMXC__End_Date__c", type date},
                {"Product.PM_WO_Frequency__c", type any},
                {"EU_SLA_Terms__c", type text},
                {"Serial_Number__c", type text},
                {"SVMXC__Installed_Product__c", type text},
                {"Name", type text}
            }
        ),

    // 3️⃣ Normalize frequency to number
    NormalizeFrequency =
        Table.TransformColumns(
            FixTypes,
            {
                {
                    "Product.PM_WO_Frequency__c",
                    each try Number.From(_) otherwise null,
                    type number
                }
            }
        ),

    // 4️⃣ Keep only valid rows
    CleanSource =
        Table.SelectRows(
            NormalizeFrequency,
            each
                [SVMXC__Start_Date__c] <> null and
                [SVMXC__End_Date__c] <> null and
                [Product.PM_WO_Frequency__c] <> null and
                [Product.PM_WO_Frequency__c] > 0
        ),

    // 5️⃣ Business rule: only contracts where at least one PM can occur
    FilterValidContracts =
        Table.SelectRows(
            CleanSource,
            each
                Date.AddDays(
                    Date.AddMonths(
                        [SVMXC__Start_Date__c],
                        [Product.PM_WO_Frequency__c]
                    ),
                    -1
                ) <= [SVMXC__End_Date__c]
        ),

    // 6️⃣ Create a strong contract key to keep PM numbering isolated per contract
    AddContractKey =
        Table.AddColumn(
            FilterValidContracts,
            "Contract Key",
            each
                Text.From([Serial_Number__c]) & "|" &
                Text.From([SVMXC__Installed_Product__c]) & "|" &
                Date.ToText([SVMXC__Start_Date__c], "yyyyMMdd") & "|" &
                Date.ToText([SVMXC__End_Date__c], "yyyyMMdd") & "|" &
                Text.From([EU_SLA_Terms__c]),
            type text
        ),

    // 7️⃣ Generate full PM schedule (anchored to original start date, no drift)
    AddServiceDates =
        Table.AddColumn(
            AddContractKey,
            "Service_Dates",
            each
                let
                    StartDate = [SVMXC__Start_Date__c],
                    EndDate = [SVMXC__End_Date__c],
                    Frequency = [Product.PM_WO_Frequency__c]
                in
                    List.Transform(
                        List.Generate(
                            () => 1,
                            (n) =>
                                Date.AddDays(
                                    Date.AddMonths(StartDate, n * Frequency),
                                    -1
                                ) <= EndDate,
                            (n) => n + 1
                        ),
                        (n) =>
                            Date.AddDays(
                                Date.AddMonths(StartDate, n * Frequency),
                                -1
                            )
                    ),
            type list
        ),

    // 8️⃣ Expand generated service dates
    ExpandDates =
        Table.ExpandListColumn(AddServiceDates, "Service_Dates"),

    RenameDeadline =
        Table.RenameColumns(
            ExpandDates,
            {{"Service_Dates", "Service_Deadline"}}
        ),

    // 9️⃣ Group by Contract Key and assign PM_Number / PM_Label
    GroupedForPM =
        Table.Group(
            RenameDeadline,
            {"Contract Key"},
            {
                {
                    "Data",
                    (t as table) =>
                        let
                            Sorted =
                                Table.Sort(
                                    t,
                                    {{"Service_Deadline", Order.Ascending}}
                                ),
                            Indexed =
                                Table.AddIndexColumn(
                                    Sorted,
                                    "PM_Number",
                                    1,
                                    1,
                                    Int64.Type
                                ),
                            AddPMLabel =
                                Table.AddColumn(
                                    Indexed,
                                    "PM_Label",
                                    each "PM" & Text.From([PM_Number]),
                                    type text
                                )
                        in
                            AddPMLabel,
                    type table
                }
            }
        ),

    // 🔟 Expand back out (do NOT expand Contract Key because it's already the group key)
    ExpandPM =
        Table.ExpandTableColumn(
            GroupedForPM,
            "Data",
            {
                "Serial_Number__c",
                "SVMXC__Installed_Product__c",
                "Name",
                "SVMXC__Start_Date__c",
                "SVMXC__End_Date__c",
                "EU_SLA_Terms__c",
                "Product.PM_WO_Frequency__c",
                "Service_Deadline",
                "PM_Number",
                "PM_Label"
            }
        ),

    // 1️⃣1️⃣ Filter to only show deadlines from 2024 onwards
    FilterFrom2024 =
        Table.SelectRows(
            ExpandPM,
            each [Service_Deadline] >= #date(2024, 1, 1)
        ),

    // 1️⃣2️⃣ Add Product Connection
    AddProductConnection =
        Table.AddColumn(
            FilterFrom2024,
            "Product Connection",
            each
                Text.From([SVMXC__Installed_Product__c]) & "_" &
                Text.From([EU_SLA_Terms__c]) & "_" &
                Date.ToText([SVMXC__Start_Date__c], "yyyyMMdd") & "_" &
                Date.ToText([SVMXC__End_Date__c], "yyyyMMdd"),
            type text
        ),

    // 1️⃣3️⃣ Add Buffer Months
    AddBufferMonths =
        Table.AddColumn(
            AddProductConnection,
            "Buffer Months",
            each
                if [Product.PM_WO_Frequency__c] = 6 then 1
                else if [Product.PM_WO_Frequency__c] = 12 then 2
                else 0,
            Int64.Type
        ),

    // 1️⃣4️⃣ Group by Product Connection to calculate Previous Deadline
    GroupedForPreviousDeadline =
        Table.Group(
            AddBufferMonths,
            {"Product Connection"},
            {
                {
                    "Data",
                    (t as table) =>
                        let
                            Sorted =
                                Table.Sort(
                                    t,
                                    {{"Service_Deadline", Order.Ascending}}
                                ),
                            Indexed =
                                Table.AddIndexColumn(
                                    Sorted,
                                    "PrevIndex",
                                    0,
                                    1,
                                    Int64.Type
                                ),
                            PrevDates =
                                {null} & List.RemoveLastN(Indexed[Service_Deadline], 1),
                            AddPreviousDeadline =
                                Table.AddColumn(
                                    Indexed,
                                    "Previous Deadline",
                                    each PrevDates{[PrevIndex]},
                                    type nullable date
                                ),
                            RemoveHelper =
                                Table.RemoveColumns(AddPreviousDeadline, {"PrevIndex"})
                        in
                            RemoveHelper,
                    type table
                }
            }
        ),

    // 1️⃣5️⃣ Expand back out (do NOT expand Product Connection because it's already the group key)
    ExpandPreviousDeadline =
        Table.ExpandTableColumn(
            GroupedForPreviousDeadline,
            "Data",
            {
                "Contract Key",
                "Serial_Number__c",
                "SVMXC__Installed_Product__c",
                "Name",
                "SVMXC__Start_Date__c",
                "SVMXC__End_Date__c",
                "EU_SLA_Terms__c",
                "Product.PM_WO_Frequency__c",
                "Service_Deadline",
                "PM_Number",
                "PM_Label",
                "Buffer Months",
                "Previous Deadline"
            }
        ),

    // 1️⃣6️⃣ Final column order
    ReorderedColumns =
        Table.ReorderColumns(
            ExpandPreviousDeadline,
            {
                "Contract Key",
                "Product Connection",
                "Serial_Number__c",
                "SVMXC__Installed_Product__c",
                "Name",
                "EU_SLA_Terms__c",
                "SVMXC__Start_Date__c",
                "SVMXC__End_Date__c",
                "Product.PM_WO_Frequency__c",
                "Service_Deadline",
                "Previous Deadline",
                "Buffer Months",
                "PM_Number",
                "PM_Label"
            }
        )

in
    ReorderedColumns

 

and here is an example sample of data:

 

NameSerialStart DateEnd DatePrevious DeadlinePM_NumberService_Deadline
SCPN-121245501509701/04/202131/03/202631/03/20264231/03/2026
SCPN-121249242026901/04/202131/03/202631/03/20254231/03/2025
SCPN-124017392108316/04/202115/04/202615/04/20244215/04/2024
SCPN-15400011220903203320/06/202219/06/202819/12/20274219/12/2027
SCPN-15400012220910041320/06/202219/06/202819/12/20274219/12/2027
SCPN-22771193221360443331/10/202430/10/203130/10/20314230/10/2031
SCPN-15400013221541760320/06/202219/06/202819/12/20274219/12/2027
SCPN-22771209230260634331/10/202430/10/203130/10/20314230/10/2031
SCPN-22771199230930780331/10/202430/10/203130/10/20314230/10/2031
       
SCPN-22771203230930781331/10/202430/10/203130/10/20314230/10/2031

 

  • Hello,

    I’m not completely sure, but this usually happens when your grouping key isn’t really unique, so PMs from different contracts get mixed together and the index just keeps growing, that’s why you see values like 42 or 90 your Contract Key looks solid at first glance, but I’d double check for hidden nulls, text vs number mismatches, or small differences like spaces or formats, even one mismatch can merge contracts silently another thing I’d try is to sanity check the Service_Dates list itself, maybe for some rows the List.Generate condition never stops correctly, so you’re generating way more dates than expected, you could temporarily add a List.Count column to confirm if I had to simplify, I’d compute the expected number of PMs first using something like duration / frequency and cap the list explicitly, instead of relying only on the <= EndDate condition, that’s usually more robust

    Best regards,
    Daniele

2 Replies

  • Hello,

    I’m not completely sure, but this usually happens when your grouping key isn’t really unique, so PMs from different contracts get mixed together and the index just keeps growing, that’s why you see values like 42 or 90 your Contract Key looks solid at first glance, but I’d double check for hidden nulls, text vs number mismatches, or small differences like spaces or formats, even one mismatch can merge contracts silently another thing I’d try is to sanity check the Service_Dates list itself, maybe for some rows the List.Generate condition never stops correctly, so you’re generating way more dates than expected, you could temporarily add a List.Count column to confirm if I had to simplify, I’d compute the expected number of PMs first using something like duration / frequency and cap the list explicitly, instead of relying only on the <= EndDate condition, that’s usually more robust

    Best regards,
    Daniele

    • v-menakakota's avatar
      v-menakakota
      Community Support

      Hi quincy_p ,

      Thanks for reaching out to the Microsoft fabric community forum. 

       

      I would also take a moment to thank DanieleUgoCopp   , for actively participating in the community forum and for the solutions you’ve been sharing in the community forum. Your contributions make a real difference.

      I hope the above details help you fix the issue. If you still have any questions or need more help, feel free to reach out. We’re always here to support you .

       

       

      Best Regards, 
      Community Support Team