dynamic web api source schedule refresh on powerbi service

[Anonymous]

Hello, I am having trouble in scheduling refresh to the web api source which is dynamically generated in powerquery with auth0 authentication. I tried few of the online solution recomended by MS blogs and Chris Webb blog to applying "RelativePath" but unable to solve it. Could I get some help on this please. Below is the query which works fine when refrehed from desktop but not in powerbi service.

let
url = "https://websitename.auth0.com/oauth/token",

body = "{ ""client_id"": ""client_id_masked_xxxxx"",
""username"": """ & Username_Parameter & """,
""password"": """ & Password_Parameter & """,
""grant_type"": ""http://auth0.com/oauth/grant-type/password-realm"",
""audience"":""https://websitename.auth0.com/api/v2/"",
""realm"":""Username-Password-Authentication""}",

tokenResponse = Json.Document(Web.Contents(url,[Headers = [#"Content-Type"="application/json"], Content = Text.ToBinary(body) ] )),
AccessToken = tokenResponse[access_token],
AccessTokenHeader = "Bearer " & AccessToken,

data_url = "https://api.websitename.com/im1/run/details?start_date=" & Start_Date_Parameter & "T12:00:20Z&end_date=" & End_Date_Parameter & "T12:00:20Z",
data_body = "{
""authorization"": """& AccessTokenHeader & """,
""content-type"": ""application/json"",
""CATEGORY-ID"":""" & Category_Parameter & """
}",

GetGroups = Json.Document(
Web.Contents(
data_url,
[
Headers = Json.Document(data_body)
]
)
),
#"Converted to Table" = Record.ToTable(GetGroups),
#"Filtered Rows" = Table.SelectRows(#"Converted to Table", each ([Name] = "run_info")),
#"Expanded Value" = Table.ExpandListColumn(#"Filtered Rows", "Value"),
#"Expanded Value1" = Table.ExpandRecordColumn(#"Expanded Value", "Value", {"run_id", "run_start_date_est", "run_start_time_est", "run_end_date_est", "run_end_time_est", "line_id", "line_name", "product_sku", "output", "finished_goods", "rework", "headcount", "potential_seconds_using_output_remove_rework_waste", "potential_seconds_using_finished_goods", "actual_seconds", "total_downtime_seconds", "average_speed_using_output_remove_rework_waste", "average_speed_using_finished_goods", "target_speed", "sets_per_stamp", "target_weight", "lost_time_list"}, {"run_id", "run_start_date_est", "run_start_time_est", "run_end_date_est", "run_end_time_est", "line_id", "line_name", "product_sku", "output", "finished_goods", "rework", "headcount", "potential_seconds_using_output_remove_rework_waste", "potential_seconds_using_finished_goods", "actual_seconds", "total_downtime_seconds", "average_speed_using_output_remove_rework_waste", "average_speed_using_finished_goods", "target_speed", "sets_per_stamp", "target_weight", "lost_time_list"}),
#"Removed Other Columns" = Table.SelectColumns(#"Expanded Value1",{"run_id", "run_start_date_est", "run_start_time_est", "run_end_date_est", "run_end_time_est", "line_id", "line_name", "product_sku", "output", "finished_goods", "rework", "headcount", "potential_seconds_using_output_remove_rework_waste", "potential_seconds_using_finished_goods", "actual_seconds", "total_downtime_seconds", "average_speed_using_output_remove_rework_waste", "average_speed_using_finished_goods", "target_speed", "sets_per_stamp", "target_weight"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Removed Other Columns",{{"run_start_date_est", type date}, {"run_start_time_est", type time}, {"run_end_date_est", type date}, {"run_end_time_est", type time}, {"line_id", Int64.Type}, {"line_name", type text}, {"product_sku", type text}, {"output", Int64.Type}, {"finished_goods", Int64.Type}, {"rework", Int64.Type}, {"potential_seconds_using_output_remove_rework_waste", Int64.Type}, {"potential_seconds_using_finished_goods", Int64.Type}, {"actual_seconds", Int64.Type}, {"total_downtime_seconds", Int64.Type}, {"average_speed_using_output_remove_rework_waste", type number}, {"average_speed_using_finished_goods", type number}, {"target_speed", Int64.Type}, {"sets_per_stamp", Int64.Type}, {"target_weight", Int64.Type}, {"run_id", type text}}),
#"Renamed Columns" = Table.RenameColumns(#"Changed Type1",{{"run_start_date_est", "Start Date"}, {"run_start_time_est", "Start Time"}, {"product_sku", "SKU"}, {"line_name", "Line"}, {"average_speed_using_output_remove_rework_waste", "Avg. Speed"}, {"target_speed", "Target Speed"}}),
#"Added Conditional Column" = Table.AddColumn(#"Renamed Columns", "Potential Time (Clean)", each if [potential_seconds_using_finished_goods] <> 0 then [potential_seconds_using_finished_goods] else [potential_seconds_using_output_remove_rework_waste]),
#"Changed Type2" = Table.TransformColumnTypes(#"Added Conditional Column",{{"Potential Time (Clean)", Int64.Type}}),
#"Added Custom" = Table.AddColumn(#"Changed Type2", "Start Time (Clean)", each [Start Time] - #duration(0,3,0,0)),
#"Changed Type" = Table.TransformColumnTypes(#"Added Custom",{{"Start Time (Clean)", type time}})
in
#"Changed Type"

[Daryl-Lynch-Bzy · Accepted solution]

Hi @Anonymous - it difficult to validate the following but I think it relates to the following line in the above:

data_url = "https://api.websitename.com/im1/run/details?start_date=" & Start_Date_Parameter & "T12:00:20Z&end_date=" & End_Date_Parameter & "T12:00:20Z",

The concatenation of the URL string including the "?start_date" may not work as Chris describes in his blog.  This component needs to be moved in the "Relative Path".  i.e. not included in the main URL.  This will allow Power BI to check the credentials based on the main url instead of the dynamic url that includes dates.

[Daryl-Lynch-Bzy · Accepted solution]

Hi @Anonymous - it difficult to validate the following but I think it relates to the following line in the above:

data_url = "https://api.websitename.com/im1/run/details?start_date=" & Start_Date_Parameter & "T12:00:20Z&end_date=" & End_Date_Parameter & "T12:00:20Z",

The concatenation of the URL string including the "?start_date" may not work as Chris describes in his blog.  This component needs to be moved in the "Relative Path".  i.e. not included in the main URL.  This will allow Power BI to check the credentials based on the main url instead of the dynamic url that includes dates.

[Anonymous · Accepted solution]

Hi @Daryl-Lynch-Bzy 

Thanks! finally got it working after continuous try

data_url = "https://api.websitename.io/im1/run/details",
data_body = "{
""authorization"": """& AccessTokenHeader & """,
""content-type"": ""application/json"",
""CATEGORY-ID"":""" & Category_Parameter & """
}",

GetGroups = Json.Document(
Web.Contents(
data_url, [Query=[start_date=Start_Date_Parameter & "T12:00:20Z", end_date=End_Date_Parameter & "T12:00:20Z"],

Headers = Json.Document(data_body)
]
)
)

[Daryl-Lynch-Bzy]

Well done @Anonymous - it does make sense that the "im1/run/details" is still included in the main URL.  Only the parts after the ?.

[Anonymous]

Hi @Daryl-Lynch-Bzy , thank you for the quick response.

I tried the following and few other "RelativePath" wraping on "Web.Contents(" and no luck, (Expression.Error: 3 arguments were passed to a function which expects between 1 and 2. Details: Pattern= Arguments=[List])

data_url = "https://api.websitename.io",
data_body = "{
""authorization"": """& AccessTokenHeader & """,
""content-type"": ""application/json"",
""CATEGORY-ID"":""" & Category_Parameter & """
}",

GetGroups = Json.Document(
Web.Contents(
data_url, [
RelativePath="im1/run/details",
Query=[start_date = "Start_Date_Parameter" & "T12:00:20Z"
,end_date = "End_Date_Parameter" & "T12:00:20Z"]
],
[
Headers = Json.Document(data_body)
]
)
),