No Publish button on Dataflow Gen 2

[Don-Bot]

I have created a dataflow gen 2 that appends 3 sources together and writes them to a new table within the lakehouse.  However, I only have the option to save and run it.  There is no publish option.  Which seems to prevent me from seeing the dataflow in my pipeline activity.  Any ideas on what I can do?

I have other dataflows that are showing up in the dataflow activity.  Just not this new one.

[lilanthalakmalg]

Yes, the previous "Publish" button has been replaced with "Save" and "Save and Run" options. However, this change applies only to CI/CD-enabled data flows.

That said, there is still an issue. When a data flow is CI/CD-enabled, it cannot be added to a data pipeline. The reason for this limitation is unclear. It is concerning that Microsoft continues to release features in a fragmented manner without comprehensive testing. Similar issues have been observed in multiple cases. If transitioning from another data platform, such inconsistencies can become evident frequently.

[arpost]

@v-tsaipranayand @Don-Bot, this appears to be a possible issue with Fabric Data Factory's ability to see CI/CD-enabled dataflows.

Here's the dataflow in the workspace above:

This is a bummer as this hampers our ability to adopt or use them since Data Factory is the orchestration solution within Fabric. We also really want to be able to deploy our dataflows via deployment pipelines.

Any chance there's an update from Microsoft on when these will be accessible within a data pipeline?

[Don-Bot · Accepted solution]

Hi @arpost , @v-tsaipranay  I found an annoying "fix" to this issue...

This is forcing me to use CI/CD Dataflow when I create a dataflow from a subfolder.  However, if I create a dataflow in the main workspace area and then move it to a folder it still lets me have a regular dataflow that has publish.  

So that is what I did... quit annoying but it worked for me.

[v-tsaipranay]

Hi @arpost ,

Thank you for bringing it to our attention. It is clear that in CI/CD-enabled dataflows, there are some limitations as it is still in preview. Microsoft is still working on it for general availabilty.
If you have any insights regarding new functionalities to share, please use the ideas forum:

Home
Kindly refer the documetation below for your better understanding:

https://learn.microsoft.com/en-us/fabric/data-factory/dataflow-gen2-cicd-and-git-integration#limitat...

Thankyou.

[chrispy]

It might be that you are in a folder. You need to create in the root of the workspace then move to a folder.

[nilendraFabric]

Hello @Don-Bot 

Thanks for getting back with details 

Where are you building these DF Gen2 , in Fabric?

I can see Publish at the right side corner in Fabric

[v-tsaipranay]

Hi @Don-Bot ,

Thanks for reaching out to the Microsoft fabric community forum.

It seems you're having trouble with the publish option for your Dataflow Gen 2. Here are a few steps that might help fix the issue:

1. The publish option may not appear if there are unsaved changes or validation errors in your dataflow. Please ensure that all changes are saved and that there are no errors present.
2. Check the options under the "Save and Run" dropdown. You may find an option for publishing or validating the dataflow.

For an in-depth understanding, check out the official Microsoft documentation on saving and publishing your dataflow.

Dataflow Gen2 with CI/CD and Git integration - Microsoft Fabric | Microsoft Learn

How to save and publish your dataflow - Microsoft Fabric | Microsoft Learn

I hope this should resolve your issue, if you need any further assistance, feel free to reach out.

If this post helps, then please give us Kudos and consider Accept it as a solution to help the other members find it more quickly.

Thank you. 

[v-tsaipranay]

Hello @Don-Bot  ,

I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions. If my response has addressed your query, please accept it as a solution and give a 'Kudos' so other members can easily find it.

Thank you.

[v-tsaipranay]

Hello @Don-Bot ,

I wanted to follow up on our previous suggestions regarding No Publish button on Dataflow Gen 2. We would love to hear back from you to ensure we can assist you further.

If my response has addressed your query, please accept it as a solution and give a ‘Kudos’ so other members can easily find it. Please let us know if there’s anything else we can do to help.

Thank you.

[v-tsaipranay]

Hi @Don-Bot ,

May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.

Thank you.

[nilendraFabric]

Hello @Don-Bot 

What are the options available under the save and run dropdown?

Could you please check if  Dataflow contains unsaved changes or validation errors 

[Don-Bot]

This is below the save and run

I just noticed a differenc between my old Dataflows that have publish and my new ones that don't.  Old ones are just regular Data Flow Gen 2

New DataFlow is a CI/CD preview

I'm not seeing any errors in the new dataflow.  It will save and run successfully.  But I still can't see it to use it in a pipeline.

I'll wait to see if this is truly the answer.  But it forced me to save it as a CI/CD preview because I created this in a folder.  annoying.

https://learn.microsoft.com/en-us/fabric/data-factory/dataflow-gen2-cicd-and-git-integration

[Ameen]

Faced the same issue. This post and suggestions helped me resolve it. Thanks for raising it here.

Just had to avoid checking the preview features box and Publish button got enabled.

[Siva_Sanam]

Hi,

I encountered this problem as well. Here is the solution that worked for me. 

while creating dataflow gen2 just don't check the CI/CD preview option then you can see the publish option and eventually use it in the pipeline. ( If you don't want to repeat your work just export  your old Data flow (which doesn't have the publish option) into a powerquery template and import that into your new Dataflow gen2, it saves a lot of time).

Hope this helps 🙂

[SebasG]

Same issue, but the checkbox isn't enabled on my side 😞