Production-scale ADF → Fabric migration: REST API approach vs Microsoft Migration Utility

[dataengineerv2]

I'm automating Azure Data Factory → Microsoft Fabric Data Factory migration using ARM export + Fabric REST APIs instead of the Microsoft migration UI.

Current approach:

• Parse ARM templates (pipelines, datasets, linked services, etc.) using a custom parser.

• Create Fabric resources via REST APIs.

i am  building a reusable ADF→Fabric migration tool (REST-API based, headless) and testing it on a deliberately complex 219-pipeline factory. Our assessment classifies  each pipeline Ready / Needs review / Coming soon / Not compatible. Most land in "Needs review" for two reasons: (1) heavy use of Execute Pipeline (parent/child) which Fabric replaces with the Invoke Pipeline activity, and (2) connections bound to a self-hosted integration runtime, which Fabric needs an On-Premises Data Gateway for. We resolve the IR type from the ARM template (connectVia → runtime.type = SelfHosted) to flag on-prem precisely. Question: what's the most efficient pattern to (a) auto-convert Execute→Invoke with correct child-first deploy ordering, and (b) provision gateway-bound connections at scale once a gateway exists?

[v-kathullac]

Hi @dataengineerv2 ,

Could you please confirm if your issue has been resolved using the suggested approach? This will help other community members facing similar scenarios.
 

Thank you for being part of the Microsoft Fabric Community.

[v-anbandari]

Hi @dataengineerv2 ,

Could you please confirm if your issue has been resolved using the suggested approach? This will help other community members facing similar scenarios.
 

Thank you for being part of the Microsoft Fabric Community.
 

[v-anbandari · Accepted solution]

Hi @dataengineerv2,

Thank you for reaching out to the Microsoft Fabric Community.

Based on the Microsoft migration guidance:

• Convert Execute Pipeline activities to Invoke Pipeline activities and deploy child pipelines before parent pipelines by resolving pipeline dependencies. This ensures all referenced child pipelines already exist.

• For pipelines using a Self-hosted Integration Runtime, Fabric uses an On-premises Data Gateway instead. After the gateway is configured, recreate the required gateway-based connections and update the migrated pipelines to use those connections.

• You can also use the Fabric Pipeline Upgrade PowerShell utility to validate activity mappings and identify unsupported scenarios, even if you're building a custom REST API-based migration tool.

Related Microsoft documentation:

• https://learn.microsoft.com/fabric/data-factory/migrate-from-azure-data-factory

• https://learn.microsoft.com/fabric/data-factory/migrate-pipelines-powershell-upgrade-module-for-azur...

I hope this helps! please feel free to reach out us.

[dataengineerv2]

but powershell migration apporach  did not support parquest based source/sink to migrate ? 

[v-anbandari]

Hi @dataengineerv2 ,

You are correct ,the PowerShell migration utility does not support all scenarios (including parquet), so your REST-based approach is more suitable for complex migrations.

For Execute Invoke conversion, extract pipeline dependencies from ARM templates and deploy pipelines in child-first order, then map them to Invoke Pipeline.

For Self-hosted IR, configure the On-premises Data Gateway, create reusable gateway-based connections, and map ADF linked services to these connections during migration.

The PowerShell utility can be used only for validation of supported and unsupported activities, while your REST approach handles unsupported cases like parquet.

for more details you can refer the below docs:

PowerShell Migration of Azure Data Factory and Synapse Pipelines to Fabric - Microsoft Fabric | Micr...

Migration Planning for Azure Data Factory to Fabric Data Factory - Microsoft Fabric | Microsoft Lear...

Thank you.