Forum Discussion
New Columns with Specific Row Values from other Columns in Multiple Excel Files
Hello, I’m loading multiple Excel files from a folder in a PBI query. I want to fill a new column with the value 'Apple' (Column3-row1) and another new column with the value '2025' (Column3-row3). All of this needs to be done in the 'Transform Sample File' query so that it will work in all the Excel files added in the VVO2 query. Could anyone help me with this? Thank you
Charcho, check this
Output (first few columns)
Change folder addres in Source step to folder with your excel files
let Source = Folder.Files("c:\Users\abc\Downloads\PowerQueryForum\Charcho\"), FilteredExtension = Table.SelectRows(Source, each ([Extension] = ".xlsx")), TransformedContent = Table.TransformColumns(FilteredExtension, {{"Content", each [ a = Excel.Workbook(_){0}[Data], // T = BinaryToTable{0}[Content], T = a, R = [ Fruit = T{0}[Column1], Year = T{2}[Column1] ], RemovedTopRows = Table.PromoteHeaders(Table.Skip(T, each not List.Contains(Record.ToList(_), "jan", Comparer.OrdinalIgnoreCase))), FilteredActualYear = Table.FirstN(RemovedTopRows, each not List.Contains(Record.ToList(_), "jan", Comparer.OrdinalIgnoreCase)), FilteredOutDiffAndTotal = Table.SelectRows(FilteredActualYear, each not ([Column1] = null) and not (List.Contains(Record.ToList(_), {"total", "diff"}, (x,y)=> List.AnyTrue(List.Transform(y, (w)=> (Text.Contains(Text.From(x), w, Comparer.OrdinalIgnoreCase))))))), RemovedBlankColumns = [ blankCols = Table.SelectRows(Table.TransformColumnTypes(Table.Profile(FilteredOutDiffAndTotal),{{"NullCount", Int64.Type}, {"Count", Int64.Type}}), each [Count] = [NullCount])[Column], removed = Table.RemoveColumns(FilteredOutDiffAndTotal, blankCols) ][removed], RenamedColumns = Table.RenameColumns(RemovedBlankColumns,{{"Column1", "Numbers"}}), MergedColumns = Table.CombineColumns(RenamedColumns, List.FirstN(List.Skip(Table.ColumnNames(RenamedColumns)), each Text.StartsWith(_, "Column")) ,Combiner.CombineTextByDelimiter("", QuoteStyle.None),"Titles"), RemovedSquareTitles = Table.TransformColumns(MergedColumns,{{"Titles", each Text.Trim(_, {"▪"}), type text}}), RemovedColumns = Table.SelectColumns(RemovedSquareTitles, List.Select(Table.ColumnNames(RemovedSquareTitles), each not Text.StartsWith(_, "Column"))), RenamedColumns2 = Table.RenameColumns(RemovedColumns,{{Table.ColumnNames(RemovedColumns){2}, "DecPrevious"}}), RenamedColumns3 = Table.TransformColumnNames(RenamedColumns2, each Text.BeforeDelimiter(_, "_")), Ad_FuitAndYear = Table.FillDown(Table.FromColumns({{R[Fruit]}} & {{R[Year]}} & Table.ToColumns(RenamedColumns3), {"Fruit", "Year"} & Table.ColumnNames(RenamedColumns3)), {"Fruit", "Year"}) ][Ad_FuitAndYear], type table}}), Combined = Table.Combine(TransformedContent[Content]) in Combined
11 Replies
- AnonymousNot applicable
- dufoq3Community Champion
Hi Charcho, check this:
Before
After
let Source = Table.FromRows({{null, null, "apple"}, {null, null, "red"}, {null, null, 2025}}), Ad_AppleCol = Table.AddColumn(Source, "AppleColumn", each Source{0}[Column3], type text), Ad_YearCol = Table.AddColumn(Ad_AppleCol, "YearColumn", each Source{2}[Column3], Int64.Type) in Ad_YearCol - CharchoHelper I
Thanks dufoq3 , however I would like a generic solution to apply the steps to all Excel files in the folder, with different fruits and years. The following picture is from another Excel file, where I would like 'Melon' and '2025' to be added in the same two new columns (with 'apple', ...)
I have tried writing in the blank query ('Query1') the following, but I'm not sure how to proceed and integrate the solution into the final table 'VVO2' to apply the changes to all Excel files. is it ok? What should be the next step?
- MarkLafSuper User
Changes you make to the 'Transform Sample File' query automatically get duplicated in the 'Transform File' function, which is what gets used in the main combination query ('VVO2' in your case). That said, is there a reason not to add the transformations to 'VVO2' after the binaries are combined?
Regardless of where you add the transformations, it's a little difficult to achieve your desired results because we need additional information to determine which text counts as valid fruit or which numbers count as valid years to put into their own respective columns. Is there info in another column that you can check to determine whether a given row in Column3 is of an applicable category?
If not, you may have to just hard-code in what values should be applicable. Something like the below could work:
let Source = Excel.Workbook(Parameter1, null, true), Sheet = Source{[Name="Sheet1"]}[Data], ValidFruit = {"Apple", "Melon", "Strawberry"}, AddFruit = Table.AddColumn( Sheet, "Fruit", each if List.Contains( ValidFruit, [Column3] ) then [Column3] else null, type text ), ValidYears = {2023, 2024, 2025}, AddYear = Table.AddColumn( AddFruit, "Year", each if List.Contains( ValidYears, [Column3] ) then [Column3] else null, Int64.Type ) in AddYearOutput:
As an aside, in your latest snip, it looks like "Melon" is in your header, so you may want to consider an additional step somewhere that demotes headers (function: Table.DemoteHeaders / button: 'Use Headers as First Row')
- dufoq3Community Champion
Hi, you do not need Transform File in general, but it is easier to manage just transform file for beginners. If you could upload at least 2 files (delete sensitive data in advance), we can try to help you.
- dufoq3Community Champion
Charcho, try to replace whole your Transform Sample File code with this one:
let Source = Excel.Workbook(Parameter1, null, true), Sheet = Source{[Name="Sheet1"]}[Data], F = (tbl as table)=> let col3 = Table.ColumnNames(tbl){2}, Ad_FruitCol = Table.AddColumn(tbl, "FruitColumn", each if col3 = "Column3" then tbl{0}[Column3] else col3, type text), Ad_YearColumn = Table.AddColumn(Ad_FruitCol, "YearColumn", each Int64.From(if col3 = "Column3" then tbl{2}[Column3] else Record.Field(tbl{1}, col3)), Int64.Type) in Ad_YearColumn, Ad_FruitAndYearCols = F(Sheet) in Ad_FruitAndYearCols
- AnonymousNot applicable
Hi Charcho
Thank you for reaching out microsoft fabric community forum.
May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.
Thank you.
- CharchoHelper I
Hello,
Thank you for the answers dufoq3 MarkLaf . The thing is, I have a lot of Excel sheets with specific data in the titles in half of them, and I need to combine them. Creating those two columns with the fruit and the year would be the first step. This should be done before merging the files, right?
I've attached two example Excel files for reference and expected solution. https://www.dropbox.com/scl/fo/9qzy7t1nti2hgt0b2bsnr/AOXdgYX-n0wpPpMvDubjmVk?rlkey=z9g1t839bvk569oyxi550pqdr&st=yt3w5zll&dl=0
- dufoq3Community Champion
Charcho, check this
Output (first few columns)
Change folder addres in Source step to folder with your excel files
let Source = Folder.Files("c:\Users\abc\Downloads\PowerQueryForum\Charcho\"), FilteredExtension = Table.SelectRows(Source, each ([Extension] = ".xlsx")), TransformedContent = Table.TransformColumns(FilteredExtension, {{"Content", each [ a = Excel.Workbook(_){0}[Data], // T = BinaryToTable{0}[Content], T = a, R = [ Fruit = T{0}[Column1], Year = T{2}[Column1] ], RemovedTopRows = Table.PromoteHeaders(Table.Skip(T, each not List.Contains(Record.ToList(_), "jan", Comparer.OrdinalIgnoreCase))), FilteredActualYear = Table.FirstN(RemovedTopRows, each not List.Contains(Record.ToList(_), "jan", Comparer.OrdinalIgnoreCase)), FilteredOutDiffAndTotal = Table.SelectRows(FilteredActualYear, each not ([Column1] = null) and not (List.Contains(Record.ToList(_), {"total", "diff"}, (x,y)=> List.AnyTrue(List.Transform(y, (w)=> (Text.Contains(Text.From(x), w, Comparer.OrdinalIgnoreCase))))))), RemovedBlankColumns = [ blankCols = Table.SelectRows(Table.TransformColumnTypes(Table.Profile(FilteredOutDiffAndTotal),{{"NullCount", Int64.Type}, {"Count", Int64.Type}}), each [Count] = [NullCount])[Column], removed = Table.RemoveColumns(FilteredOutDiffAndTotal, blankCols) ][removed], RenamedColumns = Table.RenameColumns(RemovedBlankColumns,{{"Column1", "Numbers"}}), MergedColumns = Table.CombineColumns(RenamedColumns, List.FirstN(List.Skip(Table.ColumnNames(RenamedColumns)), each Text.StartsWith(_, "Column")) ,Combiner.CombineTextByDelimiter("", QuoteStyle.None),"Titles"), RemovedSquareTitles = Table.TransformColumns(MergedColumns,{{"Titles", each Text.Trim(_, {"▪"}), type text}}), RemovedColumns = Table.SelectColumns(RemovedSquareTitles, List.Select(Table.ColumnNames(RemovedSquareTitles), each not Text.StartsWith(_, "Column"))), RenamedColumns2 = Table.RenameColumns(RemovedColumns,{{Table.ColumnNames(RemovedColumns){2}, "DecPrevious"}}), RenamedColumns3 = Table.TransformColumnNames(RenamedColumns2, each Text.BeforeDelimiter(_, "_")), Ad_FuitAndYear = Table.FillDown(Table.FromColumns({{R[Fruit]}} & {{R[Year]}} & Table.ToColumns(RenamedColumns3), {"Fruit", "Year"} & Table.ColumnNames(RenamedColumns3)), {"Fruit", "Year"}) ][Ad_FuitAndYear], type table}}), Combined = Table.Combine(TransformedContent[Content]) in Combined - dufoq3Community Champion
Sorry, but you should think of all possible scenarios in advance and provide comprehensive sample data with expected result. I have provided 3 different codes, as per your requirements, which you have changed over time