topic Re: REST API Pagination not working in Pipelines

REST API Pagination not working

aveekr — Mon, 13 May 2024 21:34:37 GMT

Trying to use the Pagination Capability for REST API connector in Copy Activity in Fabric. But its not working. We are using the AbsoluteURL property. The connector seems to call the API once and not taking into account the pagination config. We are facing this challenge in ADF in Fabric. Any suggestions

Re: REST API Pagination not working

v-cboorla-msft — Tue, 14 May 2024 06:34:07 GMT

Hi aveekr

Thanks for using Microsoft Fabric Community.

If you want to use Value as None then enter the value as $.next.

In case if you want to use Value as Body then enter the value as next.

I see in the screenshot that you are using Body and entering the value as $.next. Try to use the property name without $ symbol and execute the Copy activity.

If the issue still persists, please do let us know. Glad to help.

I hope this information helps.

Thank you.

Re: REST API Pagination not working

aveekr — Thu, 16 May 2024 03:26:21 GMT

We tried both options , it is still not working

"page": {

"count": 108,

"limit": 100,

"links": {

"first": https://api.xxxxxxxxx.com/v2/xxxx?offset=0&limit=100,

"last": https://api.xxxxxxxxx.com/v2/xxxxx?offset=100&limit=100,

"next": https://api.xxxxxxxxx.com/v2/xxxxx?offset=100&limit=100,

"previous": null,

"self": https://api.xxxxxxxxxx.com/v2/xxxx

The next link is in the json body in this format

and we are using page.links.next as the value for absoluteURL

Re: REST API Pagination not working

v-cboorla-msft — Mon, 20 May 2024 12:52:45 GMT

Hi aveekr

Apologies for the delay in response from my side.

Please refer to the documentation which might help you : Implementing Pagination with the Copy Activity in Microsoft Fabric - Microsoft Community Hub

Please do let us know if you have any further queries. Glad to help.

I hope this information helps.

Thank you

Re: REST API Pagination not working

v-cboorla-msft — Tue, 21 May 2024 16:05:02 GMT

Hi aveekr

Thanks.

Re: REST API Pagination not working

v-cboorla-msft — Wed, 22 May 2024 13:21:57 GMT

Hi aveekr

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others .
If you have any question relating to the current thread, please do let us know and we will try out best to help you.
In case if you have any other question on a different issue, we request you to open a new thread.

Thanks.