Forum Discussion
Jeffbruin
3 years agoNew Member
Handling Power query refresh errors resulting from source file refresh
Good Day, I am using power query to automatically pull data from 3 source files every 5 minutes. These source files automatically update at different frequencies from our ERP system . When the p...
Jeffbruin
3 years agoNew Member
Thanks for the reply. Believe through Phython it pulls the data to a local network folder from the ERP and saves over the previous file. Was hoping an error handling VBA would be able to acknowdlege and subsequently ignore the errors until the next power query refresh iteration.
jbwtp
Memorable Member
3 years agoHi Jeffbruin,
I believe that the problem that you have is because the PQ tries to fetch data while the file is being re-written. Do you mind trying to apply to your code the following pattern:
let
fetch_data = (MaxAttempts, DelayBetweenAttempts) =>
let
Numbers = List.Numbers(1, MaxAttempts),
FileSystemCalls = List.Transform(Numbers, each try Function.InvokeAfter(()=>internal_fetch_data(), if _ > 1 then DelayBetweenAttempts else #duration(0,0,0,0)) otherwise null),
OnlySuccessful = List.Select(FileSystemCalls, each _ <> null),
Result = List.First(OnlySuccessful, null)
in
Result,
internal_fetch_data = () =>
let
// this is your origianl query from the first line to the step that throughs out the error (inclusive on both sides)
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45W8k0tSk9NUYqNBQA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Merged = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Merged", type text}})
in #"Changed Type",
#"Uppercased Text" = Table.TransformColumns(fetch_data(10, #duration(0,0,0,10) /*10 sec delay*/),{{"Merged", Text.Upper, type text}})
in
#"Uppercased Text"
This tries to get the data from the file several times each time with some delay. Maybe this will be sufficient in your case and resolve the problem? Obvoiusly, you will nbeed to adjust it to your case by altering the content of the internal_fetch_data function.
Thanks,
John