Forum Discussion
Memorable MemberQUESTION::DATAFLOW GEN 2::Publish vs Save
Does anyone know what is the difference between Publish and close a DFg2 versus just closing the DFg2 and running it later from a pipeline?
In other words, if you close out from a DFg2 without publishing, and run it later (say, 1 day later) from a pipeline, does this mean that this DFg2 will automatically go through its first cylce of publish-->refresh, or not?
And if not, does this mean that when the pipeline runs the DFg2, the data will be stale, ie yesterday's data?
Hi Element115
Thanks for using Fabric Community.Publish vs. Close in Dataflow Gen2:
- Publish: When you publish a Dataflow Gen2, two things happen:
- Validation: The dataflow definition undergoes a validation process to ensure it's error-free and can run successfully. This includes checking for missing connections, invalid transformations, etc.
- Preparation for Refresh: The published dataflow is prepared for the refresh process. This involves generating an orchestration plan and allocating resources for data movement and transformation.
- Close: Closing a Dataflow Gen2 simply saves your current work as a draft and closes the editing window. No validation or refresh preparation occurs.
Running a Closed Dataflow Later:
- Yes, a closed Dataflow Gen2 can be run later from a pipeline. Pipeline does not publish the dataflow for you. So the run will use the last published version.
Hope this helps. Please let me know if you have any further questions.
- Publish: When you publish a Dataflow Gen2, two things happen:
6 Replies
Hi Element115
Thanks for using Fabric Community.Publish vs. Close in Dataflow Gen2:
- Publish: When you publish a Dataflow Gen2, two things happen:
- Validation: The dataflow definition undergoes a validation process to ensure it's error-free and can run successfully. This includes checking for missing connections, invalid transformations, etc.
- Preparation for Refresh: The published dataflow is prepared for the refresh process. This involves generating an orchestration plan and allocating resources for data movement and transformation.
- Close: Closing a Dataflow Gen2 simply saves your current work as a draft and closes the editing window. No validation or refresh preparation occurs.
Running a Closed Dataflow Later:
- Yes, a closed Dataflow Gen2 can be run later from a pipeline. Pipeline does not publish the dataflow for you. So the run will use the last published version.
Hope this helps. Please let me know if you have any further questions.
- Element115
Thanks for the clarification. So just to confirm I understand the Running a Closed Dataflow Later properly:
Let's consider the following example:
0__create DF for the first time (1 M query only).
1__publish --> refresh.
2__open DF and change the M query.
3__close without publishing.
4__pipeline runs DF.
When step 4 occurs, the M query that gets executed is the one from step 1, not the one with the new changes from step 2. Is this correct?
Hi Element115
Yes you are right.
- Publish: When you publish a Dataflow Gen2, two things happen:
