Fetch data from Samsara API with all the pages and create a table /dataset in PowerBI

[Anonymous]

Hello folks.

I need to fetch data from Samsara web API .Samsara API’s response has pagination structure.

It gives response in multiple pages.To specify no of pages, it has tag called ‘hasNextPage’. If this flag is true, Value of ‘endCurser’ tag should be assigned to ‘after’ parameter. ‘after’ is input parameter which should be passed with URL.

To fulfill this condition for data fetch, there should be a script/logic that will dynamically compute no of pages and assign ‘endCursor’ value to ‘after’ parameter.

I tried using 

https://developers.samsara.com/docs/pbi-custom-connector#power-bi-desktop-integration-example

this example given on Samsara's documentation. This is really time consuming (connecting a query took 38 minutes) and giving response in nested structure. I tried expanding it. But it took around 35 minutes to apply changes.

Also, it is not displaying all of the rows from source(it has 900+ rows minimum)

Any solution to this?

I tried creating a python script but it has limitations.

The basic script is provided on Samsara's dev portal.

PBI accepts only pandas dataframe. and converting JSON response to Pandas in a loop is causing data loss.

Your help to solve this issue is highly appreciated.

You can help me on either of the approaches that I have tried.

Thanks a lot in advance.

[15woodsjo]

The important part is setting your pagination rules per the Samsara documentation. You will also have to pass an authorization header with your API key.

This is a copy pipeline with an API connection that works:

[willhawkins00]

Hey there!

Have you had any luck resolving this issue? I'm looking into connecting Fabric and Samsara which seems very similar to your problem.