Forum Discussion
Data Pipeline - DataFlow Gen 2, Variables assignment
Hi divyamalhotra12 ,
Inorder to understand what is Data Pipeline -
List of Data Transformation Options present in Data Pipeline - Data transformation activities - Microsoft Fabric | Microsoft Learn
Basically if you have Dataflow you can publish it inorder to do transformation.
You can schedule the published Dataflow directly without any dependency.
If you want to do prior steps before its execution you can use dataflow activity in data pipeline along with some series of activities.
It completely depends on how do you want to design your pipeline or consume your data flow gen 2.
Hope this is helpful. Please let me know if you have any further queries."If you want to do prior steps before its execution you can use dataflow activity in data pipeline along with some series of activities."
So, this means, I choose the dataflow activity in my pipeline, select the dataflow and then click on the arrow in the dataflow. That will then take me to the dataflow and I can edit, publish it and come back to my pipeline and run it?
Thank you for the response.
I have two more questions.
1. I create a data pipeline. I choose Dataflow activity and select a dataflow that I created. Can I specify some transformations to be performed in the dataflow through data pipeline? Can I see those steps in data pipeline?
2. If I create a pipeline and select 'dataflow' activity and then I select an already published dataflow from my workspace, then what is the output expected once I run the pipeline. I do not understand the purpose of the same.
Hi divyamalhotra12 ,
Inorder to understand what is Data Pipeline -
List of Data Transformation Options present in Data Pipeline - Data transformation activities - Microsoft Fabric | Microsoft Learn
Basically if you have Dataflow you can publish it inorder to do transformation.
You can schedule the published Dataflow directly without any dependency.
If you want to do prior steps before its execution you can use dataflow activity in data pipeline along with some series of activities.
It completely depends on how do you want to design your pipeline or consume your data flow gen 2.
Hope this is helpful. Please let me know if you have any further queries.
Hi divyamalhotra12 ,
We haven’t heard from you on the last response and was just checking back to see if we answered your query.
Otherwise, will respond back with the more details and we will try to help ."If you want to do prior steps before its execution you can use dataflow activity in data pipeline along with some series of activities."
So, this means, I choose the dataflow activity in my pipeline, select the dataflow and then click on the arrow in the dataflow. That will then take me to the dataflow and I can edit, publish it and come back to my pipeline and run it?
Hi divyamalhotra12 ,
You can actually edit your dataflow and publish it.
But what I actually mean is, incase if you have to do prior activities before execution of data flow, you can take help from data pipeline.
Hope this is helpful. Please let me know incase of further queries.Thank you! It is clear to me now.