Forum Discussion

Siboska's avatar
Siboska
Helper II
1 year ago
Solved

GraphQL- Shopify Pagination

Hi everyone, I’m building an ADF pipeline to pull all orders from Shopify’s GraphQL API into an Azure SQL database, and I’m stuck on the pagination logic.   What I’ve tried so far 1. Use a Lookup...
Need Help
Pipeline
  • v-prasare's avatar
    v-prasare
    1 year ago

    Hi Siboska 

     

    as per above query since you have mentioned the Copy activity does not contain the response body, so you are not able to access values like needed endCursor and hasNextPage for pagination. Instead, can you please try using a Lookup activity, which does include the full JSON response and allows you to extract the necessary data for paginated requests. please let us know if this helps you resolve your issue

     

     

     

     

    Thanks,

    Prashanth Are

    MS Fabric Community Support

     

    if you find this answer helpful please mark it accept as solution by doing so, this will help community members find asnwers quickly for similar issues.

v-prasare's avatar
v-prasare
Community Support
1 year ago

Hi Siboska 

 

as per above query since you have mentioned the Copy activity does not contain the response body, so you are not able to access values like needed endCursor and hasNextPage for pagination. Instead, can you please try using a Lookup activity, which does include the full JSON response and allows you to extract the necessary data for paginated requests. please let us know if this helps you resolve your issue

 

 

 

 

Thanks,

Prashanth Are

MS Fabric Community Support

 

if you find this answer helpful please mark it accept as solution by doing so, this will help community members find asnwers quickly for similar issues.

  • Siboska's avatar
    Siboska
    Helper II
    1 year ago

    Thanks for the reply. I ended up going with the same solution.

     

    For anyone else reading this: I had to wrap the initial load in an outer Lookup and Copy activity with a date-sorted filter to get the first batch of data. That way, I could use the resulting end cursor inside the Until loop. From my experience, setting the end cursor variable to null or blank isnt allowed.

    Not sure if that’s the "correct" approach, but it worked for me.