Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
1 year ago
Solved

Dataflows vs Data Pipelines vs Deployment Pipelines

I have 4 Gen2 dataflows in a data pipeline, in a Dev workspace.   Initially the dataflows are the standard Gen2 dataflows, which were in a data pipeline, but won't work in a Deployment Pipeline.  ...
data engineering
  • nilendraFabric's avatar
    nilendraFabric
    1 year ago

    Hello Anonymous 

    I understand this is not ideal

    CI/CD-enabled Dataflows Gen2 cannot be directly added to a Data Pipeline for orchestration ( As you mentioned). Only standard Dataflows Gen2 are supported in Data Pipelines.

    One option is to split or maintain 2 copied - ( Not Ideal) 

    • Use standard Dataflows Gen2 within your Data Pipeline for orchestration.
    • Separately manage CI/CD-enabled Dataflows for deployment purposes until full integration support is available.

    Once CI/CD-enabled Dataflows are available in Data pipelines , delete the Non CICD one 

    Hope this helps 

    Thanks

     

     

v-menakakota's avatar
v-menakakota
Community Support
1 year ago

Hi Anonymous ,

Thank you for helpful respose  nilendraFabric ,Standard Gen2 Dataflows  can be added to a data pipeline for orchestration but won't work in a deployment pipeline.I request you to follow the suggestions provided by Nilendra and let us know if it was helpful.If you still need any assistance feel free to reachout.

Regards,
Menaka.