Forum Discussion
Get data from web error: A web API key can only be specified when a web API key name is provided
Ok, since you're going down the rabbit hole :) Here's some M code you should be able to adapt to do what you need. It's taken from a different project I worked on some time ago so it will show you the way, but you'll need to craft your solution. It relies on using M queries as functions.
TPalmer might be able to help here too.
Table.GenerateByPage = (getNextPage as function) as table =>
let
listOfPages = List.Generate(
() => getNextPage(null),
(lastPage) => lastPage <> null,
(lastPage) => getNextPage(lastPage)
),
tableOfPages = Table.FromList(listOfPages, Splitter.SplitByNothing(), {"Column1"}),
firstRow = tableOfPages{0}?
in
if (firstRow = null) then
Table.FromRows({})
else
Value.ReplaceType(
Table.ExpandTableColumn(tableOfPages, "Column1", Table.ColumnNames(firstRow[Column1])),
Value.Type(firstRow[Column1])
),
GetNextLink = (link) =>
let
links = Text.Split(link, ","),
splitLinks = List.Transform(links, each Text.Split(Text.Trim(_), ";")),
next = List.Select(splitLinks, each Text.Trim(_{1}) = "rel=""next"""),
first = List.First(next),
removedBrackets = Text.Range(first{0}, 1, Text.Length(first{0}) - 2)
in
try removedBrackets otherwise null,
GetContents = (url as text) =>
let
content = Web.Contents(url),
link = GetNextLink(Value.Metadata(content)[Headers][#"Link"]?),
json = Json.Document(content)
in
json meta [Next=link],
GetPagedTable = (url as text) => Table.GenerateByPage((previous) =>
let
next = if previous = null then null else Value.Metadata(previous)[Next],
current = if previous <> null and next = null then null else GetContents(if next = null then url else next),
link = if current = null then null else Value.Metadata(current)[Next],
table = if current = null then null else Table.FromList(current, Splitter.SplitByNothing(), null, null, ExtraValues.Error)
in
table meta [Next=link])
At the end of the day you'll call
Source = GetPagedTable( <<your url>>),
HTH, -Lukasz
Hi Lukasz
I am trying to do something similar. I am trying to query the MS Cognitive Services Text Analytics API via the function below. As that API is asynchronous I need to extract the Operation-Location header value in order to retrieve the results. However, the Web.Contents function seems to only return Response Status.
For testing, I have modified the function to return the headers for the response metadata as per your example code but all I get is Content-Type field with a null value. Am I right in summising that it is not possible to extract response header data (except for status) and therefore this sort of processing is only possible if link/location values are retuned as content rather than in headers?
(Source as table) as any =>
let
JsonRecords = Text.FromBinary(Json.FromValue(Source)),
JsonRequest = "{ ""stopWords"": [], ""topicsToExclude"": [],""documents"": " & JsonRecords & "}",
JsonContent = Text.ToBinary(JsonRequest, TextEncoding.Ascii),
Response =
Web.Contents("https://westus.api.cognitive.microsoft.com/text/analytics/v2.0/topics",
[
Headers = [#"Ocp-Apim-Subscription-Key"= "xxxxxxxxxxxxxxxxxxxxxxxxxx",
#"Content-Type"="application/json"],
Content=JsonContent,
ManualStatusHandling={202}
]),
ResponseMeta = Value.Metadata(Response),
ResponseStatus = ResponseMeta[Response.Status],
Output = if ResponseStatus = 200 then Json.Document(Response,1252) else ResponseMeta[Headers]
in
Output
- lukaszp9 years agoPower BI Team
- mllopis9 years agoCommunity Admin
Hi all,
In order to be able to supply an “Api Key” via the credential UI, the user’s M has to look like this:
Web.Contents(URL, [ApiKeyName="mykey"])
If the user enters a key of SECRET, this will produce a final URL of URL?mykey=SECRET
NOTE: this is not supported in cloud refresh.
With respect to other messages on the thread,
- There’s no way to set arbitrary HTTP request headers through the credential system.
- There’s no way to get arbitrary HTTP response headers
Hope this helps.
- Sokon9 years agoAdvocate V
Obviously, there was no OData feed-Connector available in earlier versions of PBID, right?
Because now, there is one - which doesn't mean much, because the error stays the same:
I would expect the build-in editor is able to compose correct API-Queries. If so, what do I do wrong? (Not many options here, right)