The Get Data with Cloud Connection is a cross-Fabric capability that supports a broad range of data sources—including Azure Blob Storage, PostgreSQL, Azure Key Vault, S3, and many more.
Now available directly within notebooks, this feature empowers users to efficiently and securely access and manage data from diverse sources. Importantly, it provides a secure way to handle data source credentials inside notebooks, ensuring your data connections are both convenient and protected.
This release supports only cloud data sources. For the on-premises data source, please choose Managed Private Endpoint to access the on-premises data.
How to create Fabric Connection inside Notebook
You can create the connection from inside Notebook’s New Connection flow and from the Fabric data source management page.
To create a new connection inside notebook:
- Select the Add connection button under the Connection tab within Notebook Explorer.
- Choose the data source to be accessed and provide the credential detail needed to set up the connection. The supported authentication types include:
- Basic Authentication: Supported for Azure SQL Database and other databases that support basic authentication.
- Account Key Authentication: Supported for REST API data sources that require. Account key authentication.
- Token Authentication: Supported data sources that require token-based authentication.
- Workspace Identity Authentication: Supported for Fabric workspace identity authentication.
- Service Principal Authentication (SPN): Supported for data sources that require SPN-based authentication.
Once the connection is created, it will show under the Current Notebook node, indicating it is linked to the notebook and ready to use.
Create a new connection inside Fabric data source management page:
- Select the toggle named Allow Code-First Artifacts like Notebooks to access this connection (Preview) to enable the connection to be used in notebooks. This toggle can only be set during the creation of the connection and can't be modified later.
2. After creating the connection, it appears under Global permissions, ready to link to the notebook. Selecting Connect in the context menu should link this connection to the notebook.
How to use Fabric Connection inside notebook
Once the Fabric Connection is created and bound to the current notebook, you can generate code snippets to access the data source directly from the notebook.
- Locate the connection in the Current Notebook node.
- Select the ellipsis (...), then Add as code cell from the context menu.
The code retrieves the connection credentials, uses them to set up the data source client, and then runs a query, which you can modify as needed. If the required packages aren't present in the runtime, a preceding code cell with a pip install command appears. Run that cell before running the query.
The following is an example of code snippet generated for an Azure SQL Cosmos DB connection:
from azure.cosmos import CosmosClient
import json
import pandas as pd
connection_id = '9d405da3-3d11-481a-9022-xxxxxxxxxxx' # connection name: "neweventdb qixwang"
connection_credential = notebookutils.connections.getCredential(connection_id)
credential_dict = json.loads(connection_credential['credential'])
key = next(item['value'] for item in credential_dict['credentialData'] if item['name'] == 'key')
endpoint = 'https://userevent.documents.azure.com:443/'
client = CosmosClient(endpoint, credential=key)
databases = list(client.list_databases())
database = databases[0]
database_client = client.get_database_client(database['id'])
containers = list(database_client.list_containers())
container = containers[0]
container_name = container['id']
container_client = database_client.get_container_client(container_name)
query = f"SELECT * FROM {container_name} p"
items = list(container_client.query_items(query=query, enable_cross_partition_query=True))
df = pd.DataFrame(items)
display(df)
Connection permission requirements
- When you run the notebook, a permission check makes sure you have the permissions needed to use the connection. If you don't have permission, the notebook shows an error message. If you share the notebook with other users, they also need the right permissions to use the connection and run the code cell.
- If Workspace Identity or Service Principal Authentication (SPN) is selected as the authentication type, make sure WI/SPN has been assigned the proper permission to access the data source.
To learn more, refer to Fabric connection inside Notebook and share your feedback in the Fabric Data engineering community in Fabric Ideas or Reddit.