Forum Discussion

Ymatole's avatar
Ymatole
Frequent Visitor
1 year ago
Solved

Lakehouse movement using deployement pipelines

I created deployement pipelines to move object from DEV to PROD. I was able to move lakehouses but not schema's getting moved. Any idea what could be the issue here.I didnt get any error. Movement wa...
  • Ritaf1983's avatar
    1 year ago

    Hi Ymatole 

    The behavior you’re seeing is expected. Deployment pipelines in Fabric move the Lakehouse item (metadata and settings) but do not migrate the schema or underlying data from the SQL endpoint or OneLake. That’s why the deployment shows as successful, but your tables/schemas don’t appear in PROD.

    To move schemas and data you’ll need a separate process:

    • Schema: Export DDL (CREATE TABLE/VIEW, etc.) from DEV and run it in PROD.

    • Data: Copy the Delta/Files from OneLake using a Data Pipeline, Notebook (COPY INTO or Spark CREATE TABLE ... USING DELTA LOCATION ...), or another copy mechanism.

    • Shortcuts: Must be recreated in each environment.

    • Rules: Use Deployment Rules only for connection/config settings, not schema/data.

    In short, deployment pipelines handle the item, but schema and data migration require an additional scripted process.

    If this post helps, then please consider Accepting it as the solution to help the other members find it more quickl

  • Anonymous's avatar
    Anonymous
    1 year ago

    Hi Ymatole,
    Thank you for posting your query in Microsoft Fabric Community Forum. Also, thanks to Ritaf1983  ,for those inputs on this thread. 

    What you are seeing is expected. Deployment pipeline mainly moves the Lakehouse or Warehouse item, but it doesn’t always bring all schemas or table structures. Sometimes a few may come through, but it is not reliable for full schema movement.

    If you want the complete schema and tables in PROD, you will need to move them separately  either by running the DDL (create table scripts), or by copying the data using Data Pipeline/Notebook. Shortcuts also need to be recreated in each environment.

    Kindly refer to the below documentation links for better understanding:
     https://learn.microsoft.com/en-us/fabric/data-engineering/lakehouse-git-deployment-pipelines
    https://learn.microsoft.com/en-us/fabric/cicd/deployment-pipelines/intro-to-deployment-pipelines?tabs=new-ui
    https://learn.microsoft.com/en-us/fabric/data-engineering/lakehouse-schemas
    Hope this helps if you have any queries we are  happy to assist you further.
    Regards,
    Harshitha.