Forum Discussion
How can I connect to a SQL Mirror Database?
- 1 year ago
This may not be a real solution to your issue, but WHY do you want to connect a Notebook to a Mirrored SQL database? Typically, you use Notebooks to collect and process non-tabular data sources (Excel, JSON, XML, etc.) and from there dump it into tabular destinations (read: tables).
If data is ALREADY in tables in the mirrored database, why not use T-SQL to access and process it?
- 1 year ago
because you can do things more easily and in a saner way in Python than with the antedeluvian 4GL language that SQL is. That would be my reason #1.
#2 you got a bunch of libs in Python to do data science and data engineering that do not exist in the SQL world.
and on and on it goes.
- 11 months ago
I have solved this issue by creating a User data function. This process is explained here in Microsoft Documentation.
Python programming model for Fabric User data functions (Preview) - Microsoft Fabric | Microsoft Learn
This allows me to make the connection and fetch query results. I am able to get views as well as tables.
As of now, I haven't figured out how to do the stored procedure execution and that is next in line.
my function looks like this -->import pandas as pd import fabric.functions as fn udf = fn.UserDataFunctions() # Select 'Manage connections' and add a connection to a Fabric SQL Database # Replace the alias "<alias for sql database>" with your connection alias. @udf.connection(argName="sqlDB",alias="MYMIRRORDATABASE") @udf.function() def read_from_sql_db(sqlDB: fn.FabricSqlConnection, query: str)-> pd.DataFrame: ''' Description: Read data from Mirror DATABASE using input query. Args: sqlDB (fn.FabricSqlConnection): Fabric SQL database connection. Returns: DataFrame query result. ''' # Establish a connection to the SQL database connection = sqlDB.connect() df = pd.read_sql(query, connection) print(df.head()) return df;
careful though with DFg2 as it is a lot more expensive in terms of CUs than doing the same with either SQL script or PySpark Notebook in a pipeline.