topic Re: Collaboration in Dataflows? in Data Engineering

Collaboration in Dataflows?

eisaac — Wed, 25 Sep 2024 18:35:01 GMT

Does anyone here have advice (from documentation or from experience) for working with dataflows as part of a dev team?

Ideally, we'd like for more than one person to familiarize themselves with how a dataflow is working. To my understanding, each person would have to take-over the dataflow before being able to see the logic. We'd also like to track changes somehow, but didn't see a straightforward way to do so for dataflows.

For this reason, we've tried favoring notebooks, but the dataflows work really well for coworkers who aren't developers (e.g. tech-minded analysts). We are trying to figure out what options we have to make working with them a bit smoother. Any tips/tricks or features/plans I may not know about would help! Of course, if I'm misunderstanding the intended use/workflow for dataflows, knowing that would help too!

Re: Collaboration in Dataflows?

lbendlin — Wed, 25 Sep 2024 21:18:53 GMT

You are spot on. The need to "take over" is one of the biggest issues with dataflows.

Re: Collaboration in Dataflows?

Anonymous — Thu, 26 Sep 2024 02:38:30 GMT

Hi @eisaac ,

Thanks for the reply from lbendlin .

To collaborate in dataflow, you can share your workspace with your team members by adding them as roles in your workspace.

This way your team members can all access and collaborate on the data flow, and can all see and work on the same data flow without having to take over.

Best Regards,
Yang
Community Support Team

If there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

Re: Collaboration in Dataflows?

eisaac — Thu, 26 Sep 2024 13:39:58 GMT

For the benefit of anyone referencing this post at a future time - this does not appear to be how it works as of today. Clicking to see a DataflowGen2 prompts you to take-over the flow in order to interact with it. If there is another way to view it, I have not found it in the UI or the documentation.

The prompt:

Re: Collaboration in Dataflows?

lbendlin — Thu, 26 Sep 2024 14:28:41 GMT

One thing you can do to mitigate that issue is to regularly export the .JSON and push that into your github repo.

Re: Collaboration in Dataflows?

Anonymous — Fri, 27 Sep 2024 09:24:32 GMT

Hi @eisaac ,

Thanks for the reply from lbendlin .

I can reproduce your error, after I assign the role of workspace to the guest user, that user opens the dataflow with this error message screen you have.

I think this is the difference between inside and outside the organization, when I assign workspace role to a user inside the organization, that user is able to access dataflow normally and take over him.

Can you please try to access dataflow with someone inside the organization.

If you have any other questions please feel free to contact me.

Best Regards,
Yang
Community Support Team

Re: Collaboration in Dataflows?

eisaac — Fri, 27 Sep 2024 14:11:30 GMT

Both I, and the individual who created this flow are priveledged users in the organization's fabric environment. The same behavior can be observed between any two individuals (working on dataflows) within the organization. That, together with the text in the message, strongly imply that this is intended behavior.

I actually can't find any official documentation on this behavior. Every search result is a microsoft community link (or similar).

The question is seeking advice on working with this limitation. Yes, I am able to take over the flow, but the workflow is highly unusual for a development team. The suggestion by Ibendlin to export the .JSON is helpful. It's workable, but the inability for team-members to view flows without wrangling a service account or manually importing the .json flows slows down development and limits knowledge transfer (both decreasing the chance that the organization will actually make the most effective use of Fabric that it could).

Re: Collaboration in Dataflows?

frithjof_v — Sun, 29 Sep 2024 08:58:17 GMT

Please vote for this Idea to highlight the need:

Open Dataflow Gen2 without take over

https://ideas.fabric.microsoft.com/ideas/idea/?ideaid=b3c9448a-5053-ef11-b4ac-00224853e085