Parameterized connections in Data pipelines

I know this is an old thread but thought it might be worth posting this.

To make things a little more human friendly you can have a variables library containing the connection_name. 

Pass the connection name from the Variable Library as a parameter to a lightweight Python notebook that resolves the corresponding Fabric connection ID.

A copy data activity can use the notebook exit value as the dynamic connection. So in a notebook you'd have a parameter cell, somthing like this:

connection_name = ''

followed by a code cell, something like this:

import requests
from urllib.parse import quote

connection_name = connection_name.strip()

if not connection_name:
    raise ValueError("connection_name must be provided")

token = notebookutils.credentials.getToken(
    "https://api.fabric.microsoft.com"
)

headers = {
    "Authorization": f"Bearer {token}"
}

base_url = "https://api.fabric.microsoft.com/v1/connections"
url = base_url
matches = []

while url:
    response = requests.get(url, headers=headers, timeout=30)
    response.raise_for_status()

    payload = response.json()

    matches.extend(
        connection
        for connection in payload.get("value", [])
        if connection.get("displayName", "").casefold()
        == connection_name.casefold()
    )

    continuation_token = payload.get("continuationToken")

    url = (
        f"{base_url}?continuationToken="
        f"{quote(continuation_token, safe='')}"
        if continuation_token
        else None
    )

if len(matches) == 0:
    raise ValueError(
        f"Fabric connection not found: '{connection_name}'"
    )

if len(matches) > 1:
    raise ValueError(
        f"Fabric connection name is not unique: '{connection_name}'"
    )

connection = matches[0]

print(f"Connection name: {connection['displayName']}")
print(f"Connection ID:   {connection['id']}")

notebookutils.notebook.exit(connection["id"])

Rather than storing environment-specific connection GUIDs in parameters or a Variable Library, store the human-readable connection name. At runtime, resolve that name against the Fabric Connections API and pass the resulting connection ID to the Copy activity.

That avoids hard-coding a connection ID which could change if the connection is deleted and recreated, while still allowing DEV/TEST/PROD to use different connections through environment-specific Variable Library values.

A Variable Library expression containing the correct connection GUID works when the pipeline runs, but the Copy activity designer may fail to refresh metadata or preview data. Entering the same GUID literally allows the designer to refresh and preview.