Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

The Power BI DataViz World Championships are on! With four chances to enter, you could win a spot in the LIVE Grand Finale in Las Vegas. Show off your skills.

Reply
Ambhika_07
Frequent Visitor

Pagination in Shopify API

Hi @Everyone I am trying to connect shopify data to Power BI using API. By default 50 rows are fetched and by changing the limit, i am able to get 250.

Tried extracting the headers which contains link directing to page_info. But BI is listing only 3 headers - Content-Type,Date,Server.

When i try using Postman, there are 36 headers which includes the link directing to page_info.

Could anyone help me in getting all headers similar to postman or the link in power BI to accomplish pagination.

Thanks in advance,
Ambhika



1 ACCEPTED SOLUTION
v-tangjie-msft
Community Support
Community Support

Hi @Ambhika_07 ,

 

The only way to get all headers (as opposed to a subset) is to use a custom connector.

Upvote suggestion to get all headers back here: https://ideas.fabric.microsoft.com/ideas/idea/?ideaid=e47d7520-1509-ed11-b5cf-281878de6c19

 

There is a similar thread here,hope it helps.

Solved: Retrieve API response headers - Microsoft Fabric Community

Retrieve header fields like response status from Web.Contents in Power BI and Power Query (thebiccou...

 

Best Regards,

Neeko Tang

If this post  helps, then please consider Accept it as the solution  to help the other members find it more quickly. 

View solution in original post

4 REPLIES 4
ReportGuru
Post Patron
Post Patron

Hi @Ambhika_07 were you able to find a solution? As a workaround, maybe you can try to test your connection with a 3rd party connector, which pulls data from Shopify API and does not have this limitations in terms of rows. I've tried windsor.ai, supemetrics and funnel.io. I stayed with windsor because it is much cheaper so just to let you know other options. In case you wonder, to make the connection first search for the Shopify connector in the data sources list:

 

SHOPIFY-1.jpg

 

After that, you need to follow instructions and install the windsor.ai app from the Shopify App Store:

 

SHOPIFY-1.5.png

 

then on preview and destination page you will see a preview of your Shopify fields

 

SHOPIFY-2.jpg

 

There just select the fields you need. Finally, just select PBI as your data destination and finally just copy and paste the url on PBI --> Get Data --> Web --> Paste the url.

 

SELECT_DESTINATION_NEW.png

v-tangjie-msft
Community Support
Community Support

Hi @Ambhika_07 ,

 

The only way to get all headers (as opposed to a subset) is to use a custom connector.

Upvote suggestion to get all headers back here: https://ideas.fabric.microsoft.com/ideas/idea/?ideaid=e47d7520-1509-ed11-b5cf-281878de6c19

 

There is a similar thread here,hope it helps.

Solved: Retrieve API response headers - Microsoft Fabric Community

Retrieve header fields like response status from Web.Contents in Power BI and Power Query (thebiccou...

 

Best Regards,

Neeko Tang

If this post  helps, then please consider Accept it as the solution  to help the other members find it more quickly. 

Ambhika_07
Frequent Visitor

Hi @v-tangjie-msft 

M query section is clear. But the issue i am facing is , not all response headers are visible in power bi only 3 are visible in Power BI. Where same request in Postman, return 35 response headers. Would be helpful if you let me know on how to retrive all the headers in Power BI.

Thanks in advance,
Ambhika

v-tangjie-msft
Community Support
Community Support

Hi @Ambhika_07 ,

 

To achieve this in Power BI, you could use the Power Query M language to programmatically follow the pagination links.

Please try below steps:

1. Initial API Call: Start by making an initial API call to fetch the first set of data along with the pagination link from the response headers.


2. Parse Response Headers: Extract the "next URL" from the response headers. This will require parsing the headers to locate the URL for the next page of data.


3. Recursive Function: Create a recursive function in Power Query M that takes the "next URL" as an input, fetches the data, and then calls itself with the new "next URL" found in the response headers of the subsequent call. This function should continue to execute until there is no "next URL" found, indicating the end of the data.

 

Here's a simplified example of how you might start to structure this function in Power Query M:

et
    FetchData = (url as text) as table =>
    let
        Source = Json.Document(Web.Contents(url)),
        Data = Source[data],
        NextUrl = Source[headers][next_url], // Adjust this based on actual header structure
        Output = if NextUrl <> null then Table.Combine({Data, @FetchData(NextUrl)}) else Data
    in
        Output
in
    FetchData("YourInitialShopifyApiUrl")

Note: This is a simplified example. You'll need to adjust the logic to match the specific structure of the Shopify API response and ensure you handle authentication headers if required.

 

If you struggle with connecting to Shopify API, there are ready-made solutions for extracting data automatically from Shopify. Here is one of them: https://vidi-corp.com/shopify-power-bi-connector/

 

Best Regards,

Neeko Tang

If this post  helps, then please consider Accept it as the solution  to help the other members find it more quickly. 

Helpful resources

Announcements
Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount!

Feb2025 Sticker Challenge

Join our Community Sticker Challenge 2025

If you love stickers, then you will definitely want to check out our Community Sticker Challenge!

Feb2025 NL Carousel

Fabric Community Update - February 2025

Find out what's new and trending in the Fabric community.