Using Web Activity's output as a source of Copy Data Activity.

Hi younghoon_kim  👋

Thanks for the extra details in your follow-up thread. I can see the API returns:

"next": "/details?q=..."

…and when you use AbsoluteUrl = $.next, the next call is sent to:

https://serviceportal.telenorconnexion.com/details?... (❌ base path dropped)

instead of:

https://serviceportal.telenorconnexion.com/iot/api/subscriptions/details?... (✅ expected)

That explains the failure you saw. When the API gives a root-relative path (starts with “/”), the Copy activity treats it as domain-root and ignores the base path segment (/iot/api/subscriptions).

Below are three clean ways to get you unblocked—start with Option A if you can make the API return a full link.

A) Easiest (if API can return a full link)

Ask the API (or toggle an option, if it has one) to return an absolute next link:

"next": "https://serviceportal.telenorconnexion.com/iot/api/subscriptions/details?q=..."

Then set Source ▸ Advanced ▸ Pagination rules:

Key: AbsoluteUrl

Value: $.next

That’s the officially supported pattern: AbsoluteUrl can point to the next absolute or relative URL in the response body; JSONPath is used to read it. Also add an EndCondition or MaxRequestNumber to avoid endless loops if the API echoes back the last URL.

B) When the API only returns a root-relative /details?...

Copy activity can’t “prepend” /iot/api/subscriptions/ to a root-relative value in the pagination rules (the value must be a header reference or a JSONPath result—no string concat). That’s why the dynamic @concat(...) you tried errors (“not an ancestor” — pagination rules can’t reference other activities).

Two reliable workarounds:

B1) Dataflow Gen2 (Power Query) paging

Power Query lets you stitch the base path with the relative ‘next’ easily.

Dataflow Gen2 ▸ Blank query ▸ Advanced Editor, paste a template like:

let
Base = "https://serviceportal.telenorconnexion.com/iot/api/subscriptions",
First = "/details?{your params}",
GetPage = (rel as text) =>
let
url = Base & rel,
json = Json.Document(Web.Contents(url)),
rows = json[data],
next = try json[next] otherwise null
in [Rows = rows, Next = next],

Source = Table.GenerateByPage((prev) =>
let rel = if prev = null then First else prev[Next]
in if rel = null then null else GetPage(rel)
),

Result = Table.ExpandListColumn(Source, "Rows")
in
Result

Output to your Lakehouse table.
Docs: Handling paging with Table.GenerateByPage().

B2) Notebook (Python) — full control

If you prefer code, resolve the relative next with urljoin:

import requests, pandas as pd
from urllib.parse import urljoin

base = "https://serviceportal.telenorconnexion.com/iot/api/subscriptions/"
url = urljoin(base, "details?{your params}")
rows = []

while url:
r = requests.get(url, headers={"Authorization": f"Bearer {token}"}, timeout=60)
r.raise_for_status()
js = r.json()
rows.extend(js.get("data", []))
next_rel = js.get("next")
url = urljoin(base, next_rel) if next_rel else None

df = pd.DataFrame(rows)
# write to Lakehouse table/files as you prefer

This precisely fixes the root-relative next issue by always combining it with the correct base path. (General REST + pagination guidance for Copy/REST is here.)

C) Small safety knobs (whichever route you take)

EndCondition / MaxRequestNumber in pagination to prevent endless loops.

Request interval (ms) (e.g., 300–500) if the API rate-limits.

If this solved it, please mark as Solution and give Kudos so others can find it faster 🙏