Web API not pulling all columns

[AmandaMulryan]

Has anyone ever had issues with a paginated web API not pulling all available columns? 

The company providing the data/API has confirmed the data is there, and I can also see it in the front end of the system. 

Here is their documentation for this particular endpoint: 

https://developers.samsara.com/reference/listdrivers 

I used the same Advanced Editor query for all of the API endpoints. 

let
url = "https://api.samsara.com/fleet/drivers",
Token = "#########################",
headers = [Headers=[ #"Authorization" = "Bearer " & Token ]],

FetchPage =
(url) as record =>
let
Source = Json.Document(Web.Contents(url,headers)),
data = try Source[data] otherwise null,
nextCursor = try if Source[pagination][endCursor] = "" then "DONE" else Source[pagination][endCursor] otherwise null,
res = [Data=data, NextPage=nextCursor]
in
res,

Drivers =
List.Combine(List.Generate(
()=>[res = FetchPage(url)],
each [res][NextPage]<> null,
each [res = FetchPage(url & "?after=" & [res][NextPage])],
each [res][Data]))
in
Drivers

Does anyone know why some of the columns wouldn't be showing up? 

[Anonymous]

Hi @AmandaMulryan ,

You can refer to these documents and check if they can help:

Handling paging for Power Query connectors - Power Query | Microsoft Docs

powerbi - Paging REST API results in Power Query - Stack Overflow

powerbi - How to get paginated data from API in Power BI - Stack Overflow

How to do Pagination on a REST API in Power Query (Part 1) - YouTube

Best Regards,

Stephen Tao

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.