topic Re: No Publish button on Dataflow Gen 2 in Data Engineering

No Publish button on Dataflow Gen 2

Don-Bot — Mon, 03 Feb 2025 19:31:19 GMT

I have created a dataflow gen 2 that appends 3 sources together and writes them to a new table within the lakehouse. However, I only have the option to save and run it. There is no publish option. Which seems to prevent me from seeing the dataflow in my pipeline activity. Any ideas on what I can do?

I have other dataflows that are showing up in the dataflow activity. Just not this new one.

Re: No Publish button on Dataflow Gen 2

nilendraFabric — Mon, 03 Feb 2025 22:38:59 GMT

Hello @Don-Bot

What are the options available under the save and run dropdown?

Could you please check if Dataflow contains unsaved changes or validation errors

Re: No Publish button on Dataflow Gen 2

Anonymous — Sun, 09 Feb 2025 03:33:30 GMT

Hi @Don-Bot ,

Thanks for reaching out to the Microsoft fabric community forum.

It seems you're having trouble with the publish option for your Dataflow Gen 2. Here are a few steps that might help fix the issue:

The publish option may not appear if there are unsaved changes or validation errors in your dataflow. Please ensure that all changes are saved and that there are no errors present.
Check the options under the "Save and Run" dropdown. You may find an option for publishing or validating the dataflow.

For an in-depth understanding, check out the official Microsoft documentation on saving and publishing your dataflow.

Dataflow Gen2 with CI/CD and Git integration - Microsoft Fabric | Microsoft Learn

How to save and publish your dataflow - Microsoft Fabric | Microsoft Learn

I hope this should resolve your issue, if you need any further assistance, feel free to reach out.

If this post helps, then please give us Kudos and consider Accept it as a solution to help the other members find it more quickly.

Thank you.

Re: No Publish button on Dataflow Gen 2

Don-Bot — Tue, 04 Feb 2025 14:23:10 GMT

This is below the save and run

I just noticed a differenc between my old Dataflows that have publish and my new ones that don't. Old ones are just regular Data Flow Gen 2

New DataFlow is a CI/CD preview

I'm not seeing any errors in the new dataflow. It will save and run successfully. But I still can't see it to use it in a pipeline.

I'll wait to see if this is truly the answer. But it forced me to save it as a CI/CD preview because I created this in a folder. annoying.

https://learn.microsoft.com/en-us/fabric/data-factory/dataflow-gen2-cicd-and-git-integration

Re: No Publish button on Dataflow Gen 2

nilendraFabric — Tue, 04 Feb 2025 14:28:22 GMT

Hello @Don-Bot

Thanks for getting back with details

Where are you building these DF Gen2 , in Fabric?

I can see Publish at the right side corner in Fabric

Re: No Publish button on Dataflow Gen 2

Siva_Sanam — Wed, 05 Feb 2025 02:39:07 GMT

Hi,

I encountered this problem as well. Here is the solution that worked for me.

while creating dataflow gen2 just don't check the CI/CD preview option then you can see the publish option and eventually use it in the pipeline. ( If you don't want to repeat your work just export your old Data flow (which doesn't have the publish option) into a powerquery template and import that into your new Dataflow gen2, it saves a lot of time).

Hope this helps 🙂

Re: No Publish button on Dataflow Gen 2

Anonymous — Sun, 09 Feb 2025 03:34:46 GMT

Hello @Don-Bot ,

I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions. If my response has addressed your query, please accept it as a solution and give a 'Kudos' so other members can easily find it.

Thank you.

Re: No Publish button on Dataflow Gen 2

SebasG — Tue, 11 Feb 2025 13:19:47 GMT

Same issue, but the checkbox isn't enabled on my side 😞

Re: No Publish button on Dataflow Gen 2

chrispy — Tue, 11 Feb 2025 14:33:41 GMT

It might be that you are in a folder. You need to create in the root of the workspace then move to a folder.

Re: No Publish button on Dataflow Gen 2

arpost — Tue, 11 Feb 2025 15:09:39 GMT

@Anonymousand @Don-Bot, this appears to be a possible issue with Fabric Data Factory's ability to see CI/CD-enabled dataflows.

Here's the dataflow in the workspace above:

This is a bummer as this hampers our ability to adopt or use them since Data Factory is the orchestration solution within Fabric. We also really want to be able to deploy our dataflows via deployment pipelines.

Any chance there's an update from Microsoft on when these will be accessible within a data pipeline?

Re: No Publish button on Dataflow Gen 2

Anonymous — Thu, 13 Feb 2025 07:42:17 GMT

Hello @Don-Bot ,

I wanted to follow up on our previous suggestions regarding No Publish button on Dataflow Gen 2. We would love to hear back from you to ensure we can assist you further.

If my response has addressed your query, please accept it as a solution and give a ‘Kudos’ so other members can easily find it. Please let us know if there’s anything else we can do to help.

Thank you.

Re: No Publish button on Dataflow Gen 2

Anonymous — Thu, 13 Feb 2025 08:07:44 GMT

Hi @arpost ,

Thank you for bringing it to our attention. It is clear that in CI/CD-enabled dataflows, there are some limitations as it is still in preview. Microsoft is still working on it for general availabilty.
If you have any insights regarding new functionalities to share, please use the ideas forum:

Home
Kindly refer the documetation below for your better understanding:

https://learn.microsoft.com/en-us/fabric/data-factory/dataflow-gen2-cicd-and-git-integration#limitations-and-known-issues

Thankyou.

Re: No Publish button on Dataflow Gen 2

Anonymous — Mon, 17 Feb 2025 04:06:49 GMT

Hi @Don-Bot ,

May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.

Thank you.

Re: No Publish button on Dataflow Gen 2

Ameen — Wed, 19 Feb 2025 20:39:21 GMT

Faced the same issue. This post and suggestions helped me resolve it. Thanks for raising it here.

Just had to avoid checking the preview features box and Publish button got enabled.

Re: No Publish button on Dataflow Gen 2

lilanthalakmalg — Sun, 23 Feb 2025 05:41:26 GMT

Yes, the previous "Publish" button has been replaced with "Save" and "Save and Run" options. However, this change applies only to CI/CD-enabled data flows.

That said, there is still an issue. When a data flow is CI/CD-enabled, it cannot be added to a data pipeline. The reason for this limitation is unclear. It is concerning that Microsoft continues to release features in a fragmented manner without comprehensive testing. Similar issues have been observed in multiple cases. If transitioning from another data platform, such inconsistencies can become evident frequently.

Re: No Publish button on Dataflow Gen 2

Don-Bot — Mon, 24 Feb 2025 17:47:57 GMT

Hi @arpost , @Anonymous I found an annoying "fix" to this issue...

This is forcing me to use CI/CD Dataflow when I create a dataflow from a subfolder. However, if I create a dataflow in the main workspace area and then move it to a folder it still lets me have a regular dataflow that has publish.

So that is what I did... quit annoying but it worked for me.