Forum Discussion
cannot write to lakehouse with schema enabled 3 part naming
Hi kely
My previous reply is based on my testing results, without any documentation supporting. So I'm not sure whether it's expected behavior, or it's a limitation, or it might be improved in the future.
If your notebook is currently using a 'bronze' lakehouse as the default lakehouse, using saveAsTable with 'goldlakehouse.schema.table' is probably not valid. Usually switching the default lakehouse of a running notebook will need to terminate the session first. If I understand it correctly, one session can have only one default lakehouse. So at least one lakehouse needs to use the ABFS path in the notebook.
As of now, I recommend using ABFS paths and save function to write the dataframe in your case. This is more reliable.
Best Regards,
Jing
Hi kely
My testing result is somewhat different. When I attach a schema-enabled lakehouse to the notebook and use saveAsTable function to save a dataframe to the default lakehouse, it works fine with both 3 part naming and 2 part naming.
However when the destination schema-enabled lakehouse is not attached as the default lakehouse of the notebook, it failed with an error "AnalysisException: Couldn't find a catalog to handle the identifier LHschema.raw.newTableOne."
It seems when using saveAsTable, it always tries to look for the schema and table in the default lakehouse. With 3 part naming, saveAsTable treats the first part as a schema within the default lakehouse. However actually it should be the name of another lakehouse. This leads to the failure.
Best Regards,
Jing
Community Support Team
thx Anonymous !
So if I am in a notebook connected to a 'bronze' lakehouse, and I am writing to a 'gold' lakehouse that is schema enabled... is this not a valid use case?
i.e. When I write my gold table i would use goldlakehouse.schema.table?
Hi kely
My previous reply is based on my testing results, without any documentation supporting. So I'm not sure whether it's expected behavior, or it's a limitation, or it might be improved in the future.
If your notebook is currently using a 'bronze' lakehouse as the default lakehouse, using saveAsTable with 'goldlakehouse.schema.table' is probably not valid. Usually switching the default lakehouse of a running notebook will need to terminate the session first. If I understand it correctly, one session can have only one default lakehouse. So at least one lakehouse needs to use the ABFS path in the notebook.
As of now, I recommend using ABFS paths and save function to write the dataframe in your case. This is more reliable.
Best Regards,
Jing
