Forum Discussion

xgm2109's avatar
xgm2109
Regular Visitor
2 months ago
Solved

Migration from Synapse to Fabric

Hello 

I come to the decision that our company needs to move to Fabric.

This is the current architecture:

Infrastructure

  1. Managed VNet, VNet/subnets, NSGs, Private Endpoints and Private DNS
  2. Azure SQL db.
  3. Self-hosted IR / VM for Oracle extraction
  4. on-premises Gateway VM - used to connect to Oracle OCI
  5. 184 Pipelines
  6. .~430 datasets and 5 triggers
  7. .SSIS IR + SSISDB
  8. Data lake - raw/staging/ containers
  9. Two azure apps which connect to Azure sql db

Current costs around 1600 euro a month.

No spark

  1. How to do this?
  2. What are the tools? are they in GA?
  3. What should I look for?
  4.  
  • Hi Xgm

    Looking at your current Infra and tech stack, i would recommend to review the following mapped Fabric components first :

    Current ComponentFabric Equivalent
    Synapse PipelinesFabric Data Factory Pipelines
    Azure Data LakeOneLake
    Azure SQL DBRemains Azure SQL DB or Fabric Warehouse
    DatasetsPower BI Semantic Models
    TriggersFabric Scheduling
    Self-hosted IRFabric Data Gateway
    Oracle ExtractionFabric Data Factory Connector or On-prem Gateway
    Managed VNet / PEReview carefully (biggest consideration)
    SSIS IRRebuild in Fabric Data Factory / Dataflows / Pipelines

    After reviewing I would propose a phase wise migration approach through assessment first focusing the big risk which includes SSIS dependency and your existing network where you mention

    • Managed VNet
    • Private Endpoints
    • Private DNS
    • NSGs

    with basic questions like  :

    1. Can Oracle only be reached privately?
    2. Is Azure SQL accessible through public endpoint?
    3. Are all sources private-link only?

    Hope this help to you to formulate your roadmap

4 Replies

  • Hi Xgm

    Looking at your current Infra and tech stack, i would recommend to review the following mapped Fabric components first :

    Current ComponentFabric Equivalent
    Synapse PipelinesFabric Data Factory Pipelines
    Azure Data LakeOneLake
    Azure SQL DBRemains Azure SQL DB or Fabric Warehouse
    DatasetsPower BI Semantic Models
    TriggersFabric Scheduling
    Self-hosted IRFabric Data Gateway
    Oracle ExtractionFabric Data Factory Connector or On-prem Gateway
    Managed VNet / PEReview carefully (biggest consideration)
    SSIS IRRebuild in Fabric Data Factory / Dataflows / Pipelines

    After reviewing I would propose a phase wise migration approach through assessment first focusing the big risk which includes SSIS dependency and your existing network where you mention

    • Managed VNet
    • Private Endpoints
    • Private DNS
    • NSGs

    with basic questions like  :

    1. Can Oracle only be reached privately?
    2. Is Azure SQL accessible through public endpoint?
    3. Are all sources private-link only?

    Hope this help to you to formulate your roadmap

  • v-csrikanth's avatar
    v-csrikanth
    Icon for Community Support rankCommunity Support

    Hi xgm2109​ 

    We would like to inquire whether have you got the chance to check the solutions provided by sandeephijam​  in community to resolve the issue. We hope the information provided helps to clear the query. Should you have any further queries, kindly feel free to contact the Microsoft Fabric community.

     

     

  • v-csrikanth's avatar
    v-csrikanth
    Icon for Community Support rankCommunity Support

    Hi xgm2109​ 

    We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. And, if you have any further query do let us know.

    Thank you.

  • maxravi's avatar
    maxravi
    Frequent Visitor

    Current I am in similar situation.

    Yes, this is a very realistic Fabric migration scenario, but I would not recommend treating it as a simple lift-and-shift. With 184 pipelines, ~430 datasets, SSIS IR, SHIR/Oracle connectivity, private networking and Azure SQL dependencies, I would approach it in phases.

     

    1. Start with assessment — don't migrate everything at once

     

    Microsoft now provides a built-in Synapse → Fabric migration assessment. From Synapse, you can use Migrate to Fabric → Get started to assess pipelines and activities. The assessment categorizes workloads as Ready, Needs review, Coming soon, or Not compatible, and the results can be exported to CSV.

     

    For your environment, this should be the first step.

     

    I would create an inventory like:

     

    Current component| Fabric target

    Synapse pipelines| Fabric Data Factory pipelines

    Synapse datasets| Pipeline activity/connection configuration

    Linked Services| Fabric Connections

    ADLS Gen2 raw/staging| OneLake / Lakehouse

    Azure SQL DB| Keep Azure SQL initially or migrate selectively

    SHIR| On-premises Data Gateway

    VNet/Private Endpoints| Fabric networking/VNet Data Gateway where required

    SSIS IR| Redesign/replatform

    SSIS packages| Fabric pipelines / Dataflow Gen2 / SQL / notebooks depending on workload

    Triggers| Fabric pipeline triggers

    Monitoring| Fabric Monitoring Hub

    CI/CD| Fabric Git + Deployment Pipelines

     

    Microsoft's current migration guidance specifically calls out SHIR, VNet IR and SSIS IR as areas that require special handling.

     

    2. The biggest issue in your architecture is SSIS

     

    This is the area I would investigate first.

     

    SSIS IR itself doesn't migrate to Fabric. Microsoft explicitly lists SSIS IR infrastructure migration as out of scope for the Fabric migration experience.

     

    So don't assume:

     

    "Synapse SSIS IR → Fabric"

     

    Instead, classify every SSIS package:

     

    - Simple data movement → Fabric Copy activity

    - SQL transformations → Fabric Warehouse/SQL

    - Mapping transformations → Dataflow Gen2

    - Complex transformations → Notebook/Spark where justified

    - Custom .NET/script logic → redesign depending on requirement

    - Legacy/very complex SSIS → consider keeping temporarily in Azure while you migrate the rest

     

    Since you mentioned "No Spark", that's actually fine. You don't need Spark just because you're moving to Fabric. If your workloads are primarily relational ETL, SQL, Copy activities and Dataflow Gen2, you can build a very effective Fabric architecture without introducing Spark unnecessarily.

     

    3. Oracle OCI connectivity needs a proper POC

     

    Your Oracle path is important:

     

    "Oracle OCI → SHIR/VM → Synapse"

     

    Don't remove the existing VM until you've proven the Fabric connectivity.

     

    Fabric provides On-premises Data Gateway (OPDG) and Virtual Network Data Gateway options. Microsoft's migration guidance recommends replacing SHIR with OPDG and VNet-enabled IR with the Fabric VNet Data Gateway model.

     

    For Oracle OCI, test:

     

    - Oracle driver/connectivity

    - Authentication

    - Network routing

    - DNS resolution

    - Firewall rules

    - Throughput

    - Large-table extraction

    - Incremental loads

    - Parallel copy

    - Failure/retry behavior

     

    I'd make Oracle one of your first POCs.

     

    4. Don't migrate the 430 datasets blindly

     

    This is an important architectural difference.

     

    In Fabric Data Factory, many ADF/Synapse dataset concepts can be simplified because configuration can be defined directly within activities and connections. Microsoft's migration guidance specifically recommends reviewing datasets and moving toward inline activity configuration where appropriate.

     

    So before migrating, identify:

     

    - Which datasets are actually used?

    - Which are duplicates?

    - Which are parameterized?

    - Which are only wrappers around a table/file?

    - Which pipelines depend on them?

     

    You may discover that 430 datasets become significantly fewer Fabric objects after rationalization.

     

    5. Don't migrate Azure SQL automatically

     

    You have two Azure SQL databases/apps.

     

    I would initially leave them where they are.

     

    Migration to Fabric should not automatically mean:

     

    «"Everything must move into Fabric."»

     

    If the Azure SQL DB is an application database used by your two Azure Apps, Fabric Warehouse/Lakehouse may not be an appropriate replacement.

     

    You can initially have:

     

    "Azure Apps → Azure SQL"

     

    and

     

    "Fabric → Azure SQL"

     

    for analytical/integration purposes.

     

    Later, if there is a genuine business/technical reason, evaluate whether some data belongs in Fabric Warehouse/Lakehouse.

     

    6. Use the migration tools, but don't depend entirely on them

     

    There are currently several useful migration paths.

     

    Built-in migration experience is the first option I'd use. It provides assessment, compatibility checking and migration of supported Synapse/ADF pipelines.

     

    For a large estate like yours, also look at the Microsoft.FabricPipelineUpgrade PowerShell module. Microsoft positions this for scripted, bulk and CI/CD-driven migrations.

     

    That could be particularly useful with 184 pipelines.

     

    7. I would execute your migration in this order

     

    Phase 1 – Discovery

     

    Inventory:

     

    - 184 pipelines

    - 430 datasets

    - 5 triggers

    - Linked services

    - SHIR

    - SSIS packages

    - SQL dependencies

    - Oracle dependencies

    - Private endpoints

    - DNS

    - NSGs/firewalls

    - Applications

    - Schedules

    - Failure/retry dependencies

     

    Then classify everything as:

     

    Green – migrate directly

     

    Yellow – requires modification

     

    Red – redesign/retain temporarily

     

    ---

     

    Phase 2 – Build a small Fabric landing zone

     

    Create:

     

    - Fabric workspace(s)

    - OneLake/Lakehouse structure

    - Warehouse if required

    - Connections

    - Gateway/networking

    - Security model

    - Git/CI-CD

    - Monitoring

     

    Don't start with all 184 pipelines.

     

    ---

     

    Phase 3 – Select 5–10 representative pipelines

     

    Choose:

     

    1. Simple Azure SQL → ADLS pipeline

    2. Complex transformation pipeline

    3. Oracle → Azure/OneLake pipeline

    4. Pipeline with dependencies

    5. Pipeline with parameters

    6. Pipeline with trigger

    7. Pipeline involving SSIS

    8. Largest/highest-volume pipeline

     

    This gives you a realistic migration POC.

     

    ---

     

    Phase 4 – Performance and cost comparison

     

    Your current baseline is approximately €1,600/month.

     

    Measure the equivalent Fabric workload for:

     

    - Data ingestion

    - Pipeline execution

    - Storage

    - Warehouse/query consumption

    - Gateway/networking

    - Monitoring

    - Power BI/semantic model workloads if applicable

     

    Don't assume Fabric will automatically be cheaper. Your workload pattern and Fabric capacity sizing will determine that.

     

    ---

     

    Phase 5 – Incremental production migration

     

    I would migrate by business domain rather than trying to migrate all 184 pipelines in one weekend.

     

    For example:

     

    "Domain A → Fabric"

     

    "Domain B → Fabric"

     

    "Domain C → Fabric"

     

    while the remaining workloads continue running in Synapse.

     

    Run old and new pipelines in parallel for a period and compare:

     

    - Row counts

    - Checksums/control totals

    - Runtime

    - Data freshness

    - Failure rates

    - Incremental-load correctness

    - Downstream application impact

     

    Then cut over.

     

    Are the migration tools GA?

     

    This is where I would be careful when planning the project.

     

    The Fabric Data Factory migration/assessment experience is currently documented as a Preview experience in the migration workflow, while Fabric Data Factory itself is the production platform. The built-in assessment explicitly labels the migration entry point as Preview.

     

    Therefore, I wouldn't make the entire production migration dependent on an automated migration tool.

     

    Use the tooling to accelerate assessment and conversion, but keep manual remediation/rebuild as part of your migration plan.

     

    My overall recommendation

     

    Your architecture looks like a good candidate for a phased Fabric migration, particularly because you don't have Spark workloads.

     

    The three things I'd focus on before committing are:

     

    1. SSIS → Fabric redesign

     

    2. Oracle OCI → Fabric connectivity

     

    3. Private networking/security architecture

     

    Everything else—pipelines, datasets, triggers, ADLS, etc.—is comparatively straightforward once those three areas are proven.

     

    I would also take advantage of the new ability to bring/mount an existing Azure Data Factory into a Fabric workspace as an intermediate step. It allows you to get visibility into the existing environment without immediately changing execution; the existing ADF infrastructure continues to run as-is.

     

    So my advice would be:

     

    Don't start by migrating 184 pipelines. Start by assessing all 184, identify the 10–20% that represent the hardest patterns, build a Fabric POC around those, and then scale the migration factory.

     

    That approach will give you a much more reliable estimate of effort, timeline, networking changes and—most importantly—whether the expected €1,600/month cost reduction is actually achievable.

     

    Besides this, I have seen issue with my procedure, I had to update it and also take care of date, date2 datatype