Forum Discussion

Kateryna_dev's avatar
Kateryna_dev
Frequent Visitor
6 months ago
Solved

Power Query fill in template based on different tabs

I've had a business case related to fill in Excel templates from Share Point file with a lot of tabs and data.

Instead of copy / paste each line and switching between 2 tabs I linked templates and SharePoint into Power Query excel file to generate the template based on if I needed it from pages1 or pages2. It covered 80% jobs of manual interaction and only 20% left to check the accuracy and manually perform.

 

I created a folder with the following structure:

Main Folder:

- PowQ;

- template_pages1 (fill in data from Share Point from the tab pages1);

- template_pages2 (and fill in from the tab pages2);

 

To begin with I added 3 parameters to provide the following information: SharePoint path, Folder path (I think this can be also optimised if saved locally) and Drop down to choose if pages1 or pages2 data tab for preparing template and it linked to below function to pass data from parameters further into queries.

(filePathOrUrl as text, optional sheet as nullable text) as table =>

let
    FileBinary =
        if Text.StartsWith(filePathOrUrl, "http") then
            SharePoint.Files(filePathOrUrl, [ApiVersion=15]{0}[Content])
        else
            File.Contents(filePathOrUrl),


    WorkbookTables = Excel.Workbook(FileBinary, true),
    SheetTable = WorkbookTables{[Item=sheet, Kind="Sheet"]}[Data],
    PromotedHeaders = Table.PromoteHeaders(SheetTable, [PromoteAllScalars=true])


in
    PromotedHeaders

 

Download data for pages1 or pages2 depending on what values is selected in parameter

 Mode = parameterTab,

    TemplateFile = 
        if Mode = "pages1" then "template_pages1.xlsx"
        else if Mode = "pages2" then "template_pages2.xlsx"
        else error "No Tab Selected",

 

And use it whenever I need to

Source = PagesTemplates,
    Sheets = Table.SelectRows(
        Source,
        each[Kind] = "Sheet" 
            and (
                Text.StartsWith(Text.From([Item]), "pages1")
                or Text.StartsWith(Text.From([Item]), "pages2")
            )
    ),

    WithSheetName = Table.AddColumn(
        Sheets,
        "DataWithSheet",
        each
            let
                t = Record.Field(_, "Data"),
                name = Text.From(Record.Field(_, "Item"))
            in
                Table.AddColumn(
                    t,
                    "tab_name",
                    each name,
                    type text
                )
    ),

    Combined = Table.Combine(WithSheetName[DataWithSheet]),
    type_text = Table.TransformColumnTypes(Combined,{{"Column1", type text}, {"Column2", type text}}),

 

 

Also download SharePoint table once and reuse it in different places to obtain data only I need to work with

DataColumn = Table.SelectColumns(
        SharePoint,
            {"Column2"}
    )

 

I don't have regex method in Power Query, so I've used function to remove some lists from the text. Markers are added here manually, so I am not sure if it can be optimised for all list numeric cases and not to remove numbers somewhere in the text.

    t = if text = null then null else Text.TrimStart(text),

    markers = {
        "1. ", "2. ", "3. ", "4. ", "5. ", "6. ", "7. ", 
        "1)", "2)", "3)", "4)", "5)", "6)", "7)", 
        "i)", "ii)", "iii)",
        "i. ", "ii. ", "iii. ", 
        "I)", "II)", 
        "I. ", "II. ", 
        "•",
        "-"     
    },

    cleaned =
        List.Accumulate(
            markers,
            t,
            (state, m) =>
                if Text.StartsWith(state, m)
                then Text.Range(state, Text.Length(m))
                else state
        )

 

Also I don't find the method which delimits the text by lines for more than 1 column and as a result I have different row shifts due to additional line breaks in Share Point file.

 

  • Hi   ,This is a fantastic showcase of how to dynamically switch between templates using Parameters! Using List.Accumulate to clean up bullets is also a very creative "native M" approach.

    I noticed you mentioned two specific pain points: simulating Regex and handling row shifts when splitting multiple columns by new lines.

    Here are two "Super User" patterns that might solve those last 20% of manual work for you.

    1. Solving the "Row Shift" Issue (Multi-Column Split)

    You mentioned that splitting text by lines for more than 1 column causes shifts. This happens because standard splitting creates a Cartesian product (multiplication of rows).

    To keep lines aligned across multiple columns (e.g., Description and Comment both have 3 lines), you need to Zip them together using Table.FromColumns before expanding.

    The Pattern:

     
    let
        // Assume Source table has "Description" and "Notes" columns with multi-line text
        Source = ..., 
        
        // 1. Create a Custom Column that bundles the split lists into a nested table
        AddZippedTable = Table.AddColumn(Source, "SplitData", each 
            Table.FromColumns(
                {
                    Text.Split([Description], "#(lf)"), 
                    Text.Split([Notes], "#(lf)")
                },
                {"Description_Split", "Notes_Split"} // New Column Names
            )
        ),
    
        // 2. Remove original columns and Expand the new nested table
        RemovedOriginals = Table.RemoveColumns(AddZippedTable, {"Description", "Notes"}),
        Expanded = Table.ExpandTableColumn(RemovedOriginals, "SplitData", {"Description_Split", "Notes_Split"})
    in
        Expanded

    Why this works: It treats the split lists as columns of a mini-table for each row, ensuring line 1 of Description stays with line 1 of Notes.

    2. The "Regex" Alternative

    Since Web.Page (which allows JavaScript Regex) often fails in the Power BI Service due to security refresh policies, your List.Accumulate approach is actually the safest native method!

    However, if you want to make it more dynamic (e.g., remove any leading number sequence like "1.", "10.", "1.2.") without listing them all, you can use Text.PositionOfAny to find where the real text starts.

    Dynamic Trimmer Function:

    Kod snippet'i
     
    (text as text) as text =>
    let
        // Define what constitutes "real text" (e.g., A-Z)
        RealTextMarkers = {"a".."z", "A".."Z", "("},
        
        // Find the position of the first real character
        FirstCharPosition = Text.PositionOfAny(text, RealTextMarkers),
        
        // Slice the text from that position
        Result = if FirstCharPosition > 0 then Text.Range(text, FirstCharPosition) else text
    in
        Result

    Great work on the parameterized folder structure, that is a solid architecture for scalability!


    If this helps optimize your template workflow, I'd appreciate a Kudos!
    This response was assisted by AI for translation and formatting purposes.

    Kateryna_dev

2 Replies

  • Hi   ,This is a fantastic showcase of how to dynamically switch between templates using Parameters! Using List.Accumulate to clean up bullets is also a very creative "native M" approach.

    I noticed you mentioned two specific pain points: simulating Regex and handling row shifts when splitting multiple columns by new lines.

    Here are two "Super User" patterns that might solve those last 20% of manual work for you.

    1. Solving the "Row Shift" Issue (Multi-Column Split)

    You mentioned that splitting text by lines for more than 1 column causes shifts. This happens because standard splitting creates a Cartesian product (multiplication of rows).

    To keep lines aligned across multiple columns (e.g., Description and Comment both have 3 lines), you need to Zip them together using Table.FromColumns before expanding.

    The Pattern:

     
    let
        // Assume Source table has "Description" and "Notes" columns with multi-line text
        Source = ..., 
        
        // 1. Create a Custom Column that bundles the split lists into a nested table
        AddZippedTable = Table.AddColumn(Source, "SplitData", each 
            Table.FromColumns(
                {
                    Text.Split([Description], "#(lf)"), 
                    Text.Split([Notes], "#(lf)")
                },
                {"Description_Split", "Notes_Split"} // New Column Names
            )
        ),
    
        // 2. Remove original columns and Expand the new nested table
        RemovedOriginals = Table.RemoveColumns(AddZippedTable, {"Description", "Notes"}),
        Expanded = Table.ExpandTableColumn(RemovedOriginals, "SplitData", {"Description_Split", "Notes_Split"})
    in
        Expanded

    Why this works: It treats the split lists as columns of a mini-table for each row, ensuring line 1 of Description stays with line 1 of Notes.

    2. The "Regex" Alternative

    Since Web.Page (which allows JavaScript Regex) often fails in the Power BI Service due to security refresh policies, your List.Accumulate approach is actually the safest native method!

    However, if you want to make it more dynamic (e.g., remove any leading number sequence like "1.", "10.", "1.2.") without listing them all, you can use Text.PositionOfAny to find where the real text starts.

    Dynamic Trimmer Function:

    Kod snippet'i
     
    (text as text) as text =>
    let
        // Define what constitutes "real text" (e.g., A-Z)
        RealTextMarkers = {"a".."z", "A".."Z", "("},
        
        // Find the position of the first real character
        FirstCharPosition = Text.PositionOfAny(text, RealTextMarkers),
        
        // Slice the text from that position
        Result = if FirstCharPosition > 0 then Text.Range(text, FirstCharPosition) else text
    in
        Result

    Great work on the parameterized folder structure, that is a solid architecture for scalability!


    If this helps optimize your template workflow, I'd appreciate a Kudos!
    This response was assisted by AI for translation and formatting purposes.

    Kateryna_dev

  • Kateryna_dev's avatar
    Kateryna_dev
    Frequent Visitor

    Both solutions work perfectly. The solution to use Table.FromColumns is far more accurate way and I've got better results between lines matches. Thank you!