Forum Discussion
Extracting PDF tables and appending when some column names are not the same
Hello,
I am trying to expand multiple binaries (PDF tables) in multiple nested folders so the table content from these PDF files are all usable as data points. I've put in a custom function with a sample file but the problem is that not all of these PDF's have the same format although majority of them do contain the same column names.
Here are the challenges:
- Col names in each pdf file may be different (majority are the same)
- Some col names are repeated (since in the original pdf file, a column may be stored in two cells stacked on top of each other)
- The number of columns are different for each pdf file. Some files contain
I want to put in an M query that has a logic as such:
* If column X exists in the table I read from the PDF file, then rename as XX else pass
In Python this would look something like:
for table in tables:
for i in table.columns:
if i in c.keys():
mylist.append(c[i])
else:
mylist.append(i)
# VARS defined:
table = pd.DataFrame({"Customer Name": [1,2,3,4], "Content": ["s","a","3","4"], "Not exist": ["s","a","3","4"]})
a = ['Content',
'Filename',
'Extension',
'Date accessed',
'Date modified',
'Date created',
'Folder Path',
'Customer Name',
'Assumed Eff. Date']
b = ['Content Temp',
'Filename Temp',
'Extension Temp',
'Date accessed Temp',
'Date modified Temp',
'Date created Temp',
'Folder Path Temp',
'Customer Name Temp',
'Assumed Eff. Date Temp']
c = dict(zip(a,b))
# CODE
for i in df.columns:
if i in c.keys():
mylist.append(c[i])
else:
mylist.append(i)
Is this possible?
Thank you!!
Hi nimblecat
Here is my solution for transforming column names. You can download the pbix at bottom to see details.
TempNameTable
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45Wcs7PK0nNK1GK1YlWcsvMSc1LzE0Fc1wrgOLFmfl5YJ5zaXFJfm5qkYIfWD4WAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Name = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"Name", type text}}), #"Added Custom" = Table.AddColumn(#"Changed Type", "TempName", each [Name] & " Temp") in #"Added Custom"Table
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUSoG4jSl2FgA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Customer Name" = _t, Content = _t, #"Not exist" = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"Customer Name", Int64.Type}, {"Content", type text}, {"Not exist", type text}}), Name1 = TempNameTable[Name], Name2 = TempNameTable[TempName], ChangeColumnName = Table.TransformColumnNames(#"Changed Type", each if List.Contains(Name1, _) then let _index = List.PositionOf(Name1, _) in Name2{_index} else _) in ChangeColumnNameBest Regards,
Community Support Team _ Jing
If this post helps, please Accept it as Solution to help other members find it.
2 Replies
- v-jingzhang
Community Support
Hi nimblecat
Here is my solution for transforming column names. You can download the pbix at bottom to see details.
TempNameTable
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45Wcs7PK0nNK1GK1YlWcsvMSc1LzE0Fc1wrgOLFmfl5YJ5zaXFJfm5qkYIfWD4WAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Name = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"Name", type text}}), #"Added Custom" = Table.AddColumn(#"Changed Type", "TempName", each [Name] & " Temp") in #"Added Custom"Table
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUSoG4jSl2FgA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Customer Name" = _t, Content = _t, #"Not exist" = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"Customer Name", Int64.Type}, {"Content", type text}, {"Not exist", type text}}), Name1 = TempNameTable[Name], Name2 = TempNameTable[TempName], ChangeColumnName = Table.TransformColumnNames(#"Changed Type", each if List.Contains(Name1, _) then let _index = List.PositionOf(Name1, _) in Name2{_index} else _) in ChangeColumnNameBest Regards,
Community Support Team _ Jing
If this post helps, please Accept it as Solution to help other members find it.- nimblecatFrequent Visitor
Thank you! Very helpful!