Forum Discussion
API calls with Pagination
I almost see the light at the end of the tunnel. I am able to make a connection through an API call to my data source (Okta in this case) and was very excite until I looked at the data and notice...
Hi Anonymous,
Okta's API returns an HTTP response header called "Link" which contains a URL for the next page. It doesn't seem to be currently possible for Power Query to look at the HTTP "Link" response header. I tried the code below, and it does return some HTTP response headers, but not the "Link" header.
response = Web.Contents(url, [Headers = [Authorization = "SSWS " & token]]), headers = Value.Metadata(response)[Headers]
One quick workaround is to use a PowerShell script like:
https://github.com/mbegan/Okta-Scripts/blob/master/saveEventLogs.md
to export logs to a JSON file which can be consumed by Power Query.
An alternative is to write a custom Data Connector since that has the ability to see all HTTP response headers, eg:
https://github.com/gabrielsroka/gabrielsroka.github.io/blob/master/Okta.pq