Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

The Power BI Data Visualization World Championships is back! Get ahead of the game and start preparing now! Learn more

Reply
omrangassan
New Member

combine csv files with inconsistent columns and missing columns

hi

 

i have attached many csv files that I want to combine together and get columns matched under same header.

i can combine them in the cmd csv but columns are not matched and inconsistent.

I know that some files don't have data for all these columns, but all csv files have same headers name or partial headers.

for those columns in some csv files that missing columns values : give them null value or keep empty.


https://drive.google.com/drive/folders/1Cqj2iAKf-9iOQ0bPg6TDkKzhXTP_IwlZ 

 

how can i combine all these csv files and rearrange columns to get matched under same header so columns from all csv files will get matched under same headers ? 

 

*** but these are all the headers of the 52 columns in all csv files (some csv files have partial headers, most have them all)

a263ff10 href,d6d4b13f,f397a0c8,b343055e href,b343055e,b343055e href (2),b343055e (2),b343055e href (3),b343055e (3),b343055e href (4),b343055e (4),b343055e href (5),b343055e (5),b343055e href (6),b343055e (6),b343055e href (7),b343055e (7),b343055e href (8),b343055e (8),b343055e href (9),b343055e (9),b343055e href (10),b343055e (10),b343055e href (11),b343055e (11),b343055e href (12),b343055e (12),b343055e href (13),b343055e (13),b343055e href (14),b343055e (14),b343055e href (15),b343055e (15),b343055e href (16),b343055e (16),b343055e href (17),b343055e (17),b343055e href (18),b343055e (18),b343055e href (19),b343055e (19),b343055e href (20),b343055e (20),b343055e href (21),b343055e (21),b343055e href (22),b343055e (22),b343055e href (23),b343055e (23),b343055e href (24),b343055e (24),b343055e href (25)

 

 

please help me to solve this issue and teach me how to do it by myself.

1 ACCEPTED SOLUTION
dufoq3
Super User
Super User

Hi @omrangassan, this will do the job, just change folder address in Source step.

 

Without CSV FileNames:

let
    Source = Folder.Files("c:\Users\YourUser\finbox\"),
    ColNames = List.Buffer(Text.Split("a263ff10 href,d6d4b13f,f397a0c8,b343055e href,b343055e,b343055e href (2),b343055e (2),b343055e href (3),b343055e (3),b343055e href (4),b343055e (4),b343055e href (5),b343055e (5),b343055e href (6),b343055e (6),b343055e href (7),b343055e (7),b343055e href (8),b343055e (8),b343055e href (9),b343055e (9),b343055e href (10),b343055e (10),b343055e href (11),b343055e (11),b343055e href (12),b343055e (12),b343055e href (13),b343055e (13),b343055e href (14),b343055e (14),b343055e href (15),b343055e (15),b343055e href (16),b343055e (16),b343055e href (17),b343055e (17),b343055e href (18),b343055e (18),b343055e href (19),b343055e (19),b343055e href (20),b343055e (20),b343055e href (21),b343055e (21),b343055e href (22),b343055e (22),b343055e href (23),b343055e (23),b343055e href (24),b343055e (24),b343055e href (25)", ",")),
    Combined = Table.Combine(Table.AddColumn(Source, "T", each Table.SelectColumns(Table.PromoteHeaders(Csv.Document([Content],[Delimiter=",", Columns=52, Encoding=1250, QuoteStyle=QuoteStyle.None])), ColNames, MissingField.UseNull))[T])
in
    Combined

 

If you want to preserve CSV FileNames, use this query:

let
    Source = Folder.Files("c:\Users\YourUser\finbox\"),
    ColNames = List.Buffer(Text.Split("a263ff10 href,d6d4b13f,f397a0c8,b343055e href,b343055e,b343055e href (2),b343055e (2),b343055e href (3),b343055e (3),b343055e href (4),b343055e (4),b343055e href (5),b343055e (5),b343055e href (6),b343055e (6),b343055e href (7),b343055e (7),b343055e href (8),b343055e (8),b343055e href (9),b343055e (9),b343055e href (10),b343055e (10),b343055e href (11),b343055e (11),b343055e href (12),b343055e (12),b343055e href (13),b343055e (13),b343055e href (14),b343055e (14),b343055e href (15),b343055e (15),b343055e href (16),b343055e (16),b343055e href (17),b343055e (17),b343055e href (18),b343055e (18),b343055e href (19),b343055e (19),b343055e href (20),b343055e (20),b343055e href (21),b343055e (21),b343055e href (22),b343055e (22),b343055e href (23),b343055e (23),b343055e href (24),b343055e (24),b343055e href (25)", ",")),
    Combined = Table.Combine(Table.AddColumn(Source, "T", each Table.SelectColumns(Table.AddColumn(Table.PromoteHeaders(Csv.Document([Content],[Delimiter=",", Columns=52, Encoding=1250, QuoteStyle=QuoteStyle.None])), "Name", (x)=> [Name]), {"Name"} & ColNames, MissingField.UseNull))[T])
in
    Combined

 

CSV files and their missing columns:

finbox-2025-08-14 (43).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (44).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (45).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (46).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (47).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (48).csv b343055e (4), b343055e (16), b343055e (17)
finbox-2025-08-14 (49).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (7), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (50).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (51).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (52).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (53).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (7), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (54).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (55).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (56).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (57).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (58).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (7), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (59).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (7), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (60).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (6), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (61).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (7), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (62).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (63).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (64).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (65).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (66).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (84).csv b343055e (16)

Note: Check this link to learn how to use my query.
Check this link if you don't know how to provide sample data.

View solution in original post

13 REPLIES 13
v-karpurapud
Community Support
Community Support

Hi @omrangassan 

We would like to confirm whether the issue has been resolved. If it is still outstanding, please share any additional information so we can assist you further.

Thank you.

 

v-karpurapud
Community Support
Community Support

Hi @omrangassan 

We wanted to check if you’ve had a chance to review the information provided. If you have any further questions, please let us know. Has your issue been resolved? If not, please share more details so we can assist you further.

Thank You.

dufoq3
Super User
Super User

Hi @omrangassan, this will do the job, just change folder address in Source step.

 

Without CSV FileNames:

let
    Source = Folder.Files("c:\Users\YourUser\finbox\"),
    ColNames = List.Buffer(Text.Split("a263ff10 href,d6d4b13f,f397a0c8,b343055e href,b343055e,b343055e href (2),b343055e (2),b343055e href (3),b343055e (3),b343055e href (4),b343055e (4),b343055e href (5),b343055e (5),b343055e href (6),b343055e (6),b343055e href (7),b343055e (7),b343055e href (8),b343055e (8),b343055e href (9),b343055e (9),b343055e href (10),b343055e (10),b343055e href (11),b343055e (11),b343055e href (12),b343055e (12),b343055e href (13),b343055e (13),b343055e href (14),b343055e (14),b343055e href (15),b343055e (15),b343055e href (16),b343055e (16),b343055e href (17),b343055e (17),b343055e href (18),b343055e (18),b343055e href (19),b343055e (19),b343055e href (20),b343055e (20),b343055e href (21),b343055e (21),b343055e href (22),b343055e (22),b343055e href (23),b343055e (23),b343055e href (24),b343055e (24),b343055e href (25)", ",")),
    Combined = Table.Combine(Table.AddColumn(Source, "T", each Table.SelectColumns(Table.PromoteHeaders(Csv.Document([Content],[Delimiter=",", Columns=52, Encoding=1250, QuoteStyle=QuoteStyle.None])), ColNames, MissingField.UseNull))[T])
in
    Combined

 

If you want to preserve CSV FileNames, use this query:

let
    Source = Folder.Files("c:\Users\YourUser\finbox\"),
    ColNames = List.Buffer(Text.Split("a263ff10 href,d6d4b13f,f397a0c8,b343055e href,b343055e,b343055e href (2),b343055e (2),b343055e href (3),b343055e (3),b343055e href (4),b343055e (4),b343055e href (5),b343055e (5),b343055e href (6),b343055e (6),b343055e href (7),b343055e (7),b343055e href (8),b343055e (8),b343055e href (9),b343055e (9),b343055e href (10),b343055e (10),b343055e href (11),b343055e (11),b343055e href (12),b343055e (12),b343055e href (13),b343055e (13),b343055e href (14),b343055e (14),b343055e href (15),b343055e (15),b343055e href (16),b343055e (16),b343055e href (17),b343055e (17),b343055e href (18),b343055e (18),b343055e href (19),b343055e (19),b343055e href (20),b343055e (20),b343055e href (21),b343055e (21),b343055e href (22),b343055e (22),b343055e href (23),b343055e (23),b343055e href (24),b343055e (24),b343055e href (25)", ",")),
    Combined = Table.Combine(Table.AddColumn(Source, "T", each Table.SelectColumns(Table.AddColumn(Table.PromoteHeaders(Csv.Document([Content],[Delimiter=",", Columns=52, Encoding=1250, QuoteStyle=QuoteStyle.None])), "Name", (x)=> [Name]), {"Name"} & ColNames, MissingField.UseNull))[T])
in
    Combined

 

CSV files and their missing columns:

finbox-2025-08-14 (43).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (44).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (45).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (46).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (47).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (48).csv b343055e (4), b343055e (16), b343055e (17)
finbox-2025-08-14 (49).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (7), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (50).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (51).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (52).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (53).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (7), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (54).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (55).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (56).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (57).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (58).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (7), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (59).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (7), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (60).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (6), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (61).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (7), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (62).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (5), b343055e (6), b343055e (7), b343055e (8), b343055e (9), b343055e (10), b343055e (12), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (20), b343055e (21), b343055e (24)
finbox-2025-08-14 (63).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (64).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (65).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (66).csv b343055e, b343055e (2), b343055e (3), b343055e (4), b343055e (8), b343055e (9), b343055e (13), b343055e (14), b343055e (15), b343055e (16), b343055e (17), b343055e (18), b343055e (19), b343055e (21), b343055e (24)
finbox-2025-08-14 (84).csv b343055e (16)

Note: Check this link to learn how to use my query.
Check this link if you don't know how to provide sample data.

v-karpurapud
Community Support
Community Support

Hi @omrangassan 

Thank you for submitting your question to the Microsoft Fabric Community Forum, and thanks to @AndreasKiller and @AndreasKiller for offering helpful suggestions.

 

Could you let us know if the suggested solution resolved your issue? This information can assist other community members facing similar challenges.

Thank you.

 

MarkLaf
Super User
Super User

Here is some M to parse the csvs, promote headers, then combine them all. This relies on somehow connecting to your source directory of csvs.

 

E.g., I downloaded your csvs to a local folder and connected via Folder.Files("<folder path>"). The actual folder/directory connection you use will depend on where your csv are in real life. E.g. if in SharePoint you could use SharePoint.Files, SharePoint.Content, or the SharePoint api. 

 

let
    Source = FolderConnection,
    ParseCsvBinary = Table.TransformRows(Source, each Table.PromoteHeaders(Csv.Document([Content]))),
    CombineCsvTables = Table.Combine(ParseCsvBinary)
in
    CombineCsvTables

 

So, transformation outputs look like:

 

Source

MarkLaf_0-1755363740274.png

 

 

ParseCsvBinary

MarkLaf_1-1755363784794.png

 

 

CombineCsvTables

MarkLaf_2-1755363826358.png

 

After looking a little more at the output of the simple CSV combination above, I think I better understand your original request.

 

think what you are aiming for is to pivot all your field/value column pairs, where the field is in the href column and value is in the corresponding non-href column. E.g. [b343055e href] is the "total_rev_trend_score" field (removing extra finbox path text) and [b343055e] is a percentage value like "85.7%"

 

A normal pivot won't work since you have many field/value columns stacked horizontally.

 

Here is M code building off above that does this full transformation.

 

let
    Source = FolderConnection,

    // Combine all csvs as is
    ParseCsvBinary = Table.TransformRows(Source, each Table.PromoteHeaders(Csv.Document([Content]))),
    CombineCsvTables = Table.Buffer( Table.Combine(ParseCsvBinary) ),

    // Perform pivot on every field,value column pair
    FixRows = Table.TransformRows( 
        CombineCsvTables, 
        each [
            all = _, 
            first3rec = Record.SelectFields( all, List.FirstN( Record.FieldNames(all), 3 ) ), 
            fieldvals = Record.ToList( 
                Record.SelectFields( all, List.RemoveFirstN( Record.FieldNames(all), 3 ) ) 
            ), 
            fields = List.Transform( 
                List.Alternate( fieldvals, 1, 1, 1 ), 
                each Text.AfterDelimiter(_,"/",{0,Occurrence.Last}) 
            ), 
            vals = List.ReplaceMatchingItems( 
                List.Alternate( fieldvals, 1, 1 ), 
                {{"-",null},{"",null}}
            ) , 
            fixedfieldvalrec = Record.FromTable( 
                Table.FromColumns( {fields,vals}, type table [Name=text,Value=text] ) 
            ), 
            finalrow = first3rec & fixedfieldvalrec 
        ][finalrow] ),
    DynRowType = [
        firstrec = List.First( FixRows ), 
        fields = Record.FieldNames( firstrec ), 
        fieldcount = Record.FieldCount(firstrec), 
        typerecs = List.Repeat( { [Type=type nullable text,Optional=false] }, fieldcount ), 
        rectypeval = Record.FromList(typerecs,fields), 
        rectype = Type.ForRecord(rectypeval,false) 
    ][rectype],
    RowsToTable = Table.FromRecords( FixRows, type table DynRowType ),
    
    // Transform all columns to proper type
    SimpleTypeConvert = Table.TransformColumnTypes(
        RowsToTable, {
            {"total_rev_trend_score", Percentage.Type}, {"eps_trend_score", Percentage.Type}, 
            {"fcf_levered_trend_score", Percentage.Type}, {"gp_trend_score", Percentage.Type}, 
            {"fin_health_growth_score", type number}, {"fin_health_profit_score", type number}, 
            {"fin_health_cash_flow_score", type number}, {"total_rev_cagr_3y", Percentage.Type}, 
            {"total_rev_cagr_5y", Percentage.Type}, {"total_rev_growth", Percentage.Type}, 
            {"fcf_to_ni", Percentage.Type}, {"fcf_levered_share", Currency.Type}, 
            {"fcf_levered_growth", Percentage.Type}, {"fcf_levered_cagr_3y", Percentage.Type}, 
            {"fcf_levered_cagr_5y", Percentage.Type}, {"fcf_yield_ltm", Percentage.Type}, 
            {"fcf_yield_avg_5y", Percentage.Type}, {"asset_price_return_1y", Percentage.Type}, 
            {"first_trade_date", type date}
        }
    ),
    funcNumberTBMK = Value.ReplaceType( 
        (x as nullable text) as nullable number => 
        if x = null then null 
        else [ 
            parts = Text.Split( x, " " ), 
            numpart = Number.From(parts{0}), 
            tailpart = List.Skip( parts ), 
            tailpartfix = List.First( 
                List.ReplaceMatchingItems( tailpart, {{"T",1e12},{"B",1e9},{"M",1e6},{"K",1e3}} ) 
            ), 
            final = numpart * (tailpartfix ?? 1 )
        ][final], 
        type function (x as text) as nullable Currency.Type 
    ),
    funcNumberX = 
        (x as nullable text) as nullable number => 
        if x = null then null else Number.FromText( Text.Remove(x, "x") ),
    funcOutputType = (x as function) as type => Type.FunctionReturn( Value.Type(x) ),
    ComplexTypeConvert = Table.TransformColumns(
        SimpleTypeConvert, 
        List.Transform(
            {"marketcap","total_rev","fcf_levered","ni_company"}, 
            each {_,funcNumberTBMK, funcOutputType(funcNumberTBMK) }
        ) & List.Transform(
            {"price_to_book"}, 
            each {_,funcNumberX, funcOutputType(funcNumberX) }
        ) 
    )

in
    ComplexTypeConvert

 

Output:

 

MarkLaf_0-1755382289637.png

 

I tried to do this but it gave me this error :
Expression.Error: The import FolderConnection matches no exports. Did you miss a module reference?

 

I have no experience in coding, i just copy/paste to advanced editor.

That is referring to the 

 Source = FolderConnection

line at the top. The text FolderConnection should be replaced with whatever query you already have set up that connects to your folder of csvs. As long as that query has your binaries in a column called Content, the code I shared should work.

 

If you share more about where your csvs are sitting (SharePoint? local folder? etc), then I could provide some more guidance.

thanks bro, but how you did step 3 in details to match all columns from all cvs files together ? 

When you combine tables (same as Append action through UI), PQ will automatically match the column names. We promoted first row as headers in previous step, so everything is matching by the first row of data in the csvs.

AndreasKiller
Frequent Visitor

Start as usual to import all files, e.g. from a folder, click Combine and combine and transform the data. This way PQ creates the necessary functions and queries for you.

 

There is one query to transform the example file, in there we have to perform some steps. Select the first 3 columns, right-click and unpivot other columns, you'll get an Attribute and Value column. In the Attribute column you'll see all the different headers with the index numbers in brackets, e.g. "b343055e href (2)" so the first step is to remove these.

 

In the Transform tab choose Extract, Text before Delimiter and use the "(" as delimter. We know there's a blank after the content so Trim the column also.

 

Basically the next step would be to pivot the Attribute column, but that doesn't work directly, if you try you'll get an error "Expression.Error: There were too many elements in the enumeration to complete the operation."

 

The solution is to group and add a local index.

 

Select the first 4 columns, group and use "All Rows" and column and choose "All Rows" as aggregation and you get this code:

= Table.Group(#"Trimmed Text", {"a263ff10 href", "d6d4b13f", "f397a0c8", "Attribute"}, {{"All Rows", each _, type table [a263ff10 href=nullable text, d6d4b13f=nullable text, f397a0c8=nullable text, Attribute=text, Value=text]}})

 

Modify the code to this code:
= Table.Group(#"Trimmed Text", {"a263ff10 href", "d6d4b13f", "f397a0c8", "Attribute"}, {{"All Rows", each Table.AddIndexColumn(_, "Index")}})

 

Remove all columns, except the "All Rows" column, expand the column, the data is the same as before, just an Index column is added ith an index number of each group.

 

Now you can Pivot the Attribute column and use the Value column as value and choose "Dont' aggregate" as function.

 

The final step is to remove the Index column, done. Below is the code how to transform the example file.

 

Andreas.

 

let
    Source = Csv.Document(Parameter1,[Delimiter=",", Columns=51, Encoding=1252, QuoteStyle=QuoteStyle.None]),
    #"Promoted Headers" = Table.PromoteHeaders(Source, [PromoteAllScalars=true]),
    #"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Promoted Headers", {"a263ff10 href", "d6d4b13f", "f397a0c8"}, "Attribute", "Value"),
    #"Extracted Text Before Delimiter" = Table.TransformColumns(#"Unpivoted Other Columns", {{"Attribute", each Text.BeforeDelimiter(_, "("), type text}}),
    #"Trimmed Text" = Table.TransformColumns(#"Extracted Text Before Delimiter",{{"Attribute", Text.Trim, type text}}),
    #"Grouped Rows" = Table.Group(#"Trimmed Text", {"a263ff10 href", "d6d4b13f", "f397a0c8", "Attribute"}, {{"All Rows", each Table.AddIndexColumn(_, "Index")}}),
    #"Removed Other Columns" = Table.SelectColumns(#"Grouped Rows",{"All Rows"}),
    #"Expanded {0}" = Table.ExpandTableColumn(#"Removed Other Columns", "All Rows", {"a263ff10 href", "d6d4b13f", "f397a0c8", "Attribute", "Value", "Index"}, {"a263ff10 href", "d6d4b13f", "f397a0c8", "Attribute", "Value", "Index"}),
    #"Pivoted Column" = Table.Pivot(#"Expanded {0}", List.Distinct(#"Expanded {0}"[Attribute]), "Attribute", "Value"),
    #"Removed Columns" = Table.RemoveColumns(#"Pivoted Column",{"Index"})
in
    #"Removed Columns"

 

thanks, let me try and see !

Maybe it is a good idea to show a working example, also for all fellow readers, here is an Excel file:

https://www.dropbox.com/scl/fi/n82z453frii7an89jrmfz/FC2025-08-17.xlsx?rlkey=biuy2mpnbjy5fierc3rh1bc... 

 

In that file there is just one row with data in the table, I deleted all other rows to keep the file size small.
I downloaded all the CSV files from your link into Z:\finbox and used that folder as input folder.

 

On your machine you propably have to change the source folder: Data\Get Data\Data Source Settings...\ and choose your local folder. Or edit the Source step of the "finbox" query.

 

The query that transforms each CSV file is named "Transform Sample File".

 

If you refresh the finbox query you'll end up with a bit more then 240.000 rows.

 

Andreas.

Helpful resources

Announcements
Power BI DataViz World Championships

Power BI Dataviz World Championships

The Power BI Data Visualization World Championships is back! Get ahead of the game and start preparing now!

November Power BI Update Carousel

Power BI Monthly Update - November 2025

Check out the November 2025 Power BI update to learn about new features.

FabCon Atlanta 2026 carousel

FabCon Atlanta 2026

Join us at FabCon Atlanta, March 16-20, for the ultimate Fabric, Power BI, AI and SQL community-led event. Save $200 with code FABCOMM.