topic Best practices for creating connections in Fabric in Data Engineering

Best practices for creating connections in Fabric

Anonymous — Thu, 19 Dec 2024 12:51:29 GMT

Hi,

We are working in a Data Engineering team with multiple developers/analysts using the Microsoft Fabric platform. We are trying to connect to a different source systems from Microsoft Fabric where the data was located in SFTP servers and available through APIs to bring the data into a Lakehouse in the Fabric platform .

When I create a Data Pipeline Connection in my workspace to fetch data from SFTP, it is using my organisation credentials to create this connection and to the SFTP server. The other team members are unable to view or edit the connection unless I share the connection with them.

Is there a way to streamline the process using a Entra Group to create a connection and whoever is part of the group can access the connection by default? And if there are any best practices with regards to creating connections in Fabric, please do let me know.

Re: Best practices for creating connections in Fabric

DataBard — Thu, 19 Dec 2024 18:34:43 GMT

Hi @Anonymous,

You should be able to assign Entra Groups as users to a connection in Fabric.

As for best practices, I'd encourage setting up just a single connection to the source (unless different people need different access to the source, in which case create one connection for each role). Once you have an initial connection established, the connection UI for most components of Fabric should detect the existing connection when you create a new one, and it will redirect you to use the existing connection.

Re: Best practices for creating connections in Fabric

Anonymous — Fri, 20 Dec 2024 08:09:32 GMT

Hi @Anonymous ,

Thanks for the reply from DataBard .

The steps to use the fabric with Microsoft Entra ID are as follows:

In Microsoft Entra ID, create a group that includes all team members who need access to the connection.
When you create a connection in the fabric, assign the Entra group as a user. This way, by default, anyone in the group can access the connection.
Set up SSO for the fabric using a Microsoft Entra ID. This will allow users to automatically log in to the fabric using their Entra account.
This will allow users to automatically log in to the fabric using their Entra account. Use Entra to control who has access to the fabric. You can easily add or remove users from groups as needed.

For more details on using the fabric with Microsoft Entra ID please see:

Tutorial: Microsoft Entra single sign-on (SSO) integration with Fabric - Microsoft Entra ID | Microsoft Learn

As far as best practices for creating connections in a fabric, I have one additional point to make in addition to what DataBard said:

Enable Managed Identity for the workspace to securely access data sources so you don't need to store credentials in code.

Please see the documentation below for more information on this:

Securing Your Data Pipelines: Best Practices for Fabric Data Factory | Microsoft Community Hub

If you have any other questions please feel free to contact me.

Best Regards,
Yang
Community Support Team

If there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

Re: Best practices for creating connections in Fabric

Anonymous — Fri, 20 Dec 2024 09:13:13 GMT

Thank you @v-huijie-msft. This is a great help and it is an appropriate solution I am looking for..