Forum Discussion
Power BI picking latest file issue
Hi,
My original Excel data contain blanks (first column and row). I am removed both of empty columns during the query editor but I am receiving the following error message when I am trying to refresh the Power BI desktop in order to get the latest file from the folder.
https://www.dropbox.com/s/h4me5fnnlaqy7bx/PBILF-2.png?dl=0
https://www.dropbox.com/s/gzyyh9buxzy68dt/PBILF-1.png?dl=0
https://www.dropbox.com/s/gzyyh9buxzy68dt/PBILF-1.png?dl=0
https://www.dropbox.com/s/gzyyh9buxzy68dt/PBILF-1.png?dl=0
15 Replies
- amitchandak
Super User
Saxon10 , What is the code you are using to get the latest
also check -https://medium.com/@wkrzywiec/getting-data-from-the-latest-file-in-a-folder-using-power-query-51dfa4bff711
- Saxon10
Post Prodigy
Exactly I am following the same page in order to get the latest file from the folder.
- vivran22
Community Champion
Hey Saxon10
The error is not clear from the image along with the steps. I have tried replicating it, and it works for me:
let Source = Folder.Files("Enter the Path"), #"Filtered Hidden Files1" = Table.SelectRows(Source, each [Attributes]?[Hidden]? <> true), #"Invoke Custom Function1" = Table.AddColumn(#"Filtered Hidden Files1", "Transform File", each #"Transform File"([Content])), #"Renamed Columns1" = Table.RenameColumns(#"Invoke Custom Function1", {"Name", "Source.Name"}), #"Removed Other Columns1" = Table.SelectColumns(#"Renamed Columns1", {"Source.Name", "Transform File"}), #"Expanded Table Column1" = Table.ExpandTableColumn(#"Removed Other Columns1", "Transform File", Table.ColumnNames(#"Transform File"(#"Sample File"))), #"Filtered Rows" = Table.SelectRows(#"Expanded Table Column1", each [Column1] <> null), #"Promoted Headers" = Table.PromoteHeaders(#"Filtered Rows", [PromoteAllScalars=true]), #"Filtered Rows1" = Table.SelectRows(#"Promoted Headers", each [AB] <> "AB"), #"Filtered Rows2" = Table.SelectRows(#"Filtered Rows1", each [AB] <> 133) in #"Filtered Rows2"Cheers!
Vivek
If it helps, please mark it as a solution. Kudos would be a cherry on the top 🙂
If it doesn't, then please share a sample data along with the expected results (preferably an excel file and not an image)
Blog: vivran.in/my-blog
Connect on LinkedIn
Follow on Twitter- Saxon10
Post Prodigy
thanks for your quick reply. I am new to Power BI. Could you please attached file that's would be easy to understand the power BI. Also could you please explain where I need to make changes.
here is the actual error message https://www.dropbox.com/s/owaemrmoyq2w48x/refresh%20error.PNG?dl=0.
Could you please let me know if you need a more details?
- vivran22
Community Champion
@Saxon10 ,
You need to copy the following code in the Advanced Editor of your Power Query:
let Source = Folder.Files("Enter the Path"), #"Filtered Hidden Files1" = Table.SelectRows(Source, each [Attributes]?[ Hidden]? <> true), #"Invoke Custom Function1" = Table.AddColumn(#"Filtered Hidden Files1", "Transform File", each #"Transform File"([Content])), #"Renamed Columns1" = Table.RenameColumns(#"Invoke Custom Function1", {"Name", "Source.Name"}), #"Removed Other Columns1" = Table.SelectColumns(#"Renamed Columns1", {"Source.Name", "Transform File"}), #"Expanded Table Column1" = Table.ExpandTableColumn(#"Removed Other Columns1", "Transform File", Table.ColumnNames(#"Transform File"(#"Sample File"))), #"Filtered Rows" = Table.SelectRows(#"Expanded Table Column1", each [Column1] <> null), #"Promoted Headers" = Table.PromoteHeaders(#"Filtered Rows", [PromoteAllScalars=true]), #"Filtered Rows1" = Table.SelectRows(#"Promoted Headers", each [AB] <> "AB"), #"Filtered Rows2" = Table.SelectRows(#"Filtered Rows1", each [AB] <> 133) in #"Filtered Rows2"Just update the folder location (with the current location of your data folder) where I have mentioned "enter the path" in the first line.
From you error, it seems that it is looking for a column which is not available in the new files.
Cheers!
Vivek
If it helps, please mark it as a solution. Kudos would be a cherry on the top 🙂
If it doesn't, then please share a sample data along with the expected results (preferably an excel file and not an image)
Blog: vivran.in/my-blog
Connect on LinkedIn
Follow on Twitter
- vivran22
Community Champion
Hey Saxon10 ,
The code does the following:
- Go to the folder
- Read all the files
- Filters out the latest file based on the date mentioned in the file name
- Gives the output in the desired format
let Source = Folder.Files( "ENTER FOLDER LOCATION" ), #"Filtered Hidden Files1" = Table.SelectRows(Source, each [Attributes]?[Hidden]? <> true), #"Renamed Columns1" = Table.RenameColumns(#"Filtered Hidden Files1", {"Name", "Source.Name"}), #"Removed Other Columns" = Table.SelectColumns(#"Renamed Columns1", {"Source.Name", "Content"}), #"Extracted Date from SourceName" = Table.AddColumn( #"Removed Other Columns", "Date", each let splitSourceName = Splitter.SplitTextByDelimiter("_EXTRACT_", QuoteStyle.None)( [Source.Name] ), splitsplitSourceName1 = Splitter.SplitTextByDelimiter("_", QuoteStyle.None)( splitSourceName{1}? ) in Text.Combine( { Text.Middle([Source.Name], 13, 2), Text.Combine(List.Transform(splitsplitSourceName1, each Text.Start(_, 2)), "-") } ), type date ), #"Filtered Rows" = Table.SelectRows( #"Extracted Date from SourceName", let latest = List.Max(#"Extracted Date from SourceName"[Date]) in each [Date] = latest ), Content = #"Filtered Rows"{0}[Content], #"Imported Excel" = Excel.Workbook(Content), DATA_Sheet = #"Imported Excel"{[Item = "DATA", Kind = "Sheet"]}[Data], #"Removed Top Rows" = Table.Skip(DATA_Sheet, 1), #"Promoted Headers" = Table.PromoteHeaders(#"Removed Top Rows", [PromoteAllScalars = true]) in #"Promoted Headers"Hope this helps.
Cheers!
Vivek
If it helps, please mark it as a solution. Kudos would be a cherry on the top 🙂
If it doesn't, then please share a sample data along with the expected results (preferably an excel file and not an image)
Blog: vivran.in/my-blog
Connect on LinkedIn
Follow on Twitter- Saxon10
Post Prodigy
Hi. Thanks for your respones again.
Could you please attache your final out file that would be great because it's very complex for Power BI beginners.
I am receving some error could you please advise. please refer the snapshot.
https://www.dropbox.com/s/excy9o5qez6rcf9/Capture111.PNG?dl=0
https://www.dropbox.com/s/cy4heak2trzhm5g/Capture1.PNG?dl=0
let
Source = Folder.Files(
"C:\Users\silam\Desktop\DATA EXPORT"
),
#"Filtered Hidden Files1" = Table.SelectRows(Source, each [Attributes]?[Hidden]? <> true),
#"Invoke Custom Function1" = Table.AddColumn(
#"Filtered Hidden Files1",
"Transform File (3)",
each #"Transform File (3)"([Content])
),
#"Renamed Columns1" = Table.RenameColumns(#"Invoke Custom Function1", {"Name", "Source.Name"}),
#"Removed Other Columns1" = Table.SelectColumns(
#"Renamed Columns1",
{"Source.Name", "Transform File (3)"}
),
#"Extracted Date from SourceName" = Table.AddColumn(
#"Removed Other Columns1",
"Date",
each
let
splitSourceName = Splitter.SplitTextByDelimiter("_EXTRACT_", QuoteStyle.None)(
[Source.Name]
),
splitsplitSourceName1 = Splitter.SplitTextByDelimiter("_", QuoteStyle.None)(
splitSourceName{1}?
)
in
Text.Combine(
{
Text.Middle([Source.Name], 13, 2),
Text.Combine(List.Transform(splitsplitSourceName1, each Text.Start(_, 2)), "-")
}
),
type date
),
#"Filtered Rows" = Table.SelectRows(
#"Extracted Date from SourceName",
let
latest = List.Max(#"Extracted Date from SourceName"[Date])
in
each [Date] = latest
),
#"Transform File (3)1" = #"Filtered Rows"{0}[#"Transform File (3)"],
#"Removed Top Rows" = Table.Skip(#"Transform File (3)1", 1),
#"Promoted Headers" = Table.PromoteHeaders(#"Removed Top Rows", [PromoteAllScalars = true])
in
#"Promoted Headers"- Saxon10
Post Prodigy
Hi,
I am receving two errors when I try to apply the code.
1.Expression.Error: The name 'Transform File (3)' wasn't recognized. Make sure it's spelled correctly.2.
Expression.Error: The import Transform File (3) matches no exports. Did you miss a module reference?
Can you please advise.
- Saxon10
Post Prodigy
Hi. Thanks for your time and your reply.
I copy above mentioned code and I replace the file path but I got different report according to the final output.
1.The code pulling some blanks data according to the headers here is the links https://www.dropbox.com/s/h1r9mcv8jtskvs3/PBI-ERROR.PNG?dl=0 but actually there is some data my original excel file here is the link for the file and snapshot https://www.dropbox.com/s/mq00izdj3pea1jb/EXCEL-DATA.PNG?dl=0 and https://www.dropbox.com/s/22b7bqsup9rhjfb/DATA_EXTRACT-2020-11-16-08-00-09.xlsx?dl=0
2. I can't see the source name in my Power BI here is the link for file https://www.dropbox.com/s/e1mxfbz869s9363/VIVKE.pbix?dl=0
5. Can you please advise what are changes need to be incase file or data name is changed?
6. Can you please provide step by step snap and your final working file it's will help for every one and save your time as well.- Saxon10
Post Prodigy
3. incase if I changed the data sheet name how and where can I edit the code? Example-My actual data sheet name is "DATA EXTRACT" can you please advise.
DATA_Sheet = #"Imported Excel"{[Item = "DATA", Kind = "Sheet"]}[Data], #"Removed Top Rows" = Table.Skip(DATA_Sheet, 1),
4. Where is came from 13,2? I would like to edit the code incase if any changes happen in my original Excel data. Text.Middle([Source.Name], 13, 2), Text.Combine(List.Transform(splitsplitSourceName1, each Text.Start(_, 2)), "-")