Forum Discussion
Progrmatically create pipeline
"But there is no way you can manually create a json from scratch that would represents the json otherise created by using the UI."
I think you can: https://learn.microsoft.com/en-us/fabric/data-factory/pipeline-rest-api
Hi smpa01 ,
In addition to the ideas and comments mentioned by the two above, I think you can also try the following:
1. You can create and manage data factory pipelines programmatically using SDKs for .NET, Python, and other languages.2. You can also use the Azure CLI and PowerShell, command-line tools that allow you to manage Data Factory resources through scripts, including creating and updating pipelines.
Read this official documentation below for more information: Use custom activities in a pipeline - Azure Data Factory & Azure Synapse | Microsoft Learn
Best Regards
Yilong Zhou
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Data pipelines in the backend are basically json formats.
You can code the pipelines rather than ui drag /drop in case if you are familiar with the data pipeline json format
Thanks.
Yes, you can feed a json that would create a pipeline. But it is harder to manually create a json that would have all the correct info that is conducive to the desired pipeline.
What I am looking for is a progrmatic way to create pipeline through some python/powershell libraries that already has all the pre-defined pipeline methods. So that I can invoke the methods with correct parameters and promise chain (on success, fail etc) and be able to schedle executing the code.
By programming it, I would then create the pipeline and the correct json. You can edit the pipeine from the code itself.
Otherwise, there is no way to create a json manually with the exact same pipeline in your head.