New connection Creation

[M0hamedFazil]

Hi all

I'm retrieving and API that give me an export ID and I want to check that whether the API is done processing or not and once processing is done I'll store the file to my warehouse using dataflow gen2. I have written an recursive function for that. For each new export ID it'll be included to the API url. Now my problem for each new export ID it creates new connection which hindrances my flow. Can I avoid saving the connection?

This is the error that pops up each time a new ID is created. I want to automate this using pipeline. If this pops up for each run I can't do that. I'm including the function where this error pops up

export_id = GetExportid[id],

    geturl = (group_id, report_id, access_token, export_id) =>
    let
        check = CheckStatus(group_id, report_id, access_token, export_id),
        status = check[status],

        file_url = if status = "Succeeded" then
            check[resourceLocation]
        else
            Function.InvokeAfter(() => @geturl(group_id, report_id, access_token, export_id), #duration(0, 0, 0, 30))
    in
        file_url,
    
    file_loc = geturl(group_id, report_id, access_token, export_id)

Can anyone help me on this, Thanks in advance

regards
Fazil M

[Anonymous]

Hi @M0hamedFazil 

Whether you can add a retry mechanism

export_id = GetExportid[id],

geturl = (group_id, report_id, access_token, export_id, connection) =>
let
    check = CheckStatus(group_id, report_id, access_token, export_id, connection),
    status = check[status],

    file_url = if status = "Succeeded" then
        check[resourceLocation]
    else
        Function.InvokeAfter(() => @geturl(group_id, report_id, access_token, export_id, connection), #duration(0, 0, 0, 30))
in
    file_url,

// Initialize connection once and pass it to the function
connection = InitializeConnection(),
file_loc = geturl(group_id, report_id, access_token, export_id, connection)

Regards,

Nono Chen

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[M0hamedFazil]

hi @Anonymous, Thanks for the reply

I have solved this and I was facing a problem where the flow started running and I think it sturk at some point. Can you please tell how to initiate the connection, How the function looks like?. I'll try initiate the connection and update with that here. Thanks 

Regards
Fazil M 

[frithjof_v · Accepted solution]

Can you use RelativePath?

https://blog.crossjoin.co.uk/2016/08/16/using-the-relativepath-and-query-options-with-web-contents-i...

[M0hamedFazil]

Hi @frithjof_v, Thanks for the reply.

I read the blog and used the relative path and yes it worked. Thanks

Regards
Fazil M