Forum Discussion
Attach multiple lakehouse to a single notebook.
I have a notebook and have attached to a lakehouse. The default lakehouse is called metadata, where I keep details about the job runs. Now, I have created Silver lakehouse and within a notebook at each step I need to insert when the process started n ended and also write to Silver tables. I am getting an error "[SCHEMA_NOT_FOUND] The schema `silver` cannot be found. Verify the spelling and correctness of the schema and catalog.
If you did not qualify the name with a catalog, verify the current_schema() output, or qualify the name with the correct catalog."
Hi V-yubandi-msft , sorry for coming back late on this, I was on leave and not feeling well. The issue is resolved now. Was able to attach multiple lakehouses to the notebook which resolved the issue
15 Replies
- tayloramySuper User
KrishnaMoola you can add multiple lakehouses to a notebook.
In your notebook, click the add data items button:
then you can choose an existing datasource and add another lakehouse:
THe default lakehouse is still the one with the pin beside it, so to reference the other lakehouse you need to use the full identifier:
GatewayLogs.dbo.table1, not just dbo.table1If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.
- KrishnaMoolaRegular Visitor
I have given GatewayLogs.dbo.table1 but still no luck, so have created a shortcut to the metadata lakehouse and this works. I am not sure if this is correct way to do
- V-yubandi-msftCommunity Support
Hi KrishnaMoola ,
Thanks for the update. Creating a shortcut will work because it exposes the Silver table inside your default lakehouse, so Spark is able to resolve it. However, this is more of a workaround.
The recommended approach is to attach the Silver lakehouse as a Data item in the notebook. Once both lakehouses are attached, the tables in Silver should be recognised directly using the standard namespace format.
So your shortcut isn’t incorrect but attaching the Silver lakehouse as a data item is cleaner and keeps the table where it belongs.
If you still can’t reference the tables after attaching the Silver lakehouse, please let us know and we’ll help investigate further
- AntoineWSuper User
You have bronze as default lakehouse, you can use the abfs path to get on silver lakehouse schema and inster data.
- KrishnaMoolaRegular Visitor
Hi Antonie, I have tried using the abfs path but no luck
- KevinChantSuper User
This implies a naming convention error. What notation do you get if you drag and drop the tables into your notebook?
- Gpop13Advocate IV
Hi KrishnaMoola ,
Are both of the lakehouses schema enabled?
May be this will help?
Lakehouse schemas in notebook
When you look at a schema enabled lakehouse in the notebook object explorer, you see tables are in schemas. You can drag and drop table into a code cell and get a code snippet that refers to the schema where the table is located. Use this namespace to refer to tables in your code: "workspace.lakehouse.schema.table". If you leave out any of the elements, the executor uses default setting. For example, if you only give table name, it uses default schema (dbo) from default lakehouse for the notebook.Referencing nonschema lakehouses
When you set a schema-enabled lakehouse or no lakehouse as the default in your notebook, Spark code uses schema-enabled referencing for tables. However, you can still access lakehouses without enabled schemas within the same code by referencing them as "lakehouse.table."Additionally, it’s possible to join tables from different types of lakehouses.
SQL
SELECT *
FROM workspace.schemalh.schema.table as schematable
INNER JOIN workspace.nonschemalh.table as nonschematable
ON schematable.id = nonschematable.id;
To help transition and refactor existing code to schema-enabled lakehouses, four-part naming is supported for lakehouses without schemas. You can reference tables as "workspace.lakehouse.dbo.table", where "dbo" serves as the schema name—even though it doesn’t exist yet in a lakehouse that isn’t schema-enabled. This approach lets you update your code with no downtime before enabling schema support in your lakehouses.- KrishnaMoolaRegular Visitor
Hi there, I tried to attach my notebook to Silver lakehouse but still the tables in silver notebook are not recognised. I tried giving the workspace.lakehouse.schema.table but no luck.
Is there any other way ?
- KrishnaMoolaRegular Visitor
I created a shortcut to the table in Silver lakehouse but not sure if this is the correct approach, can anyone suggest on this ?
- V-yubandi-msftCommunity Support
Hi KrishnaMoola ,
we wanted to check in as we haven't heard back from you. Did our solution work for you? If you need any more help, please don't hesitate to ask. Your feedback is very important to us. We hope to hear from you soon.
- chetanhiwaleResolver I
Hi KrishnaMoola
In Fabric, the spark catelog only keeps the default lakehosue tables and schemas. Thus you are not able to access the other lakehouse(silver lakehouse) tables. The most appropriate option would be to use ABFS path.
Other than that if you are using spark.sql you can use first convert the dataframe in VIEW and then perform sql operations.
Hope this helps. Let me know if you have any futher question , we can discuss !! - V-yubandi-msftCommunity Support
Hi KrishnaMoola ,
Kindly let us know if your issue is resolved or if you need any additional information.
Thank you.
- KrishnaMoolaRegular Visitor
Hi V-yubandi-msft , sorry for coming back late on this, I was on leave and not feeling well. The issue is resolved now. Was able to attach multiple lakehouses to the notebook which resolved the issue
- V-yubandi-msftCommunity Support
Thank you for the update, and no worries at all.Glad to hear the issue is resolved and that attaching multiple lakehouses to the notebook addressed the problem.
Please feel free to reach out if you need any further assistance.
Thank You.
- Swetha_25Advocate I
Because only the default attached Lakehouse is used automatically in a Fabric notebook, Spark can’t find the silver schema in the other Lakehouse , use fully qualified table name (LakehouseName.silver.table) or switch the catalog/schema explicitly.