Forum Discussion

danitskofabric's avatar
danitskofabric
Frequent Visitor
1 year ago
Solved

Fabric Default Schema Lakehouse

So for a Notebook i use this to define the defaultLakehouse: %%configure -f { "defaultLakehouse": { "name": "test" } }   Here "test" is a Lakehouse without activating Lakehouse-Sc...
data engineering
lakehouse
notebook
  • v-achippa's avatar
    v-achippa
    1 year ago

    Hi danitskofabric,

     

    Currently %%configure with only the lakehouse name works only when schemas are disabled.
    When schemas are enabled, fabric requires both the lakehouse Id and workspaceId for binding, so fully portable notebooks need to fetch these Id’s dynamically at runtime. As of now there is no way to avoid Id’s completely, it is a current limitation of schema enabled lakehouses.

     

    I recommend submitting your detailed feedback and ideas through Microsoft's official feedback channels. Feedback submitted through these channels is frequently reviewed by the product teams and can contribute to meaningful improvements.

    Fabric Ideas - Microsoft Fabric Community

     

    Thanks and regards,

    Anjan Kumar Chippa

v-achippa's avatar
v-achippa
Icon for Community Support rankCommunity Support
1 year ago

Hi danitskofabric,

 

When lakehouse schemas is enabled, %%configure with only name: test does not create the internal binding the spark session needs. We need to include the lakehouse id and workspace id in the default lakehouse block and run that %%configure as the very first cell. Please follow below steps:

  • Get the lakehouse id and workspace id
  • Make sure the first cell of the notebook is the configure cell. If you already executed other spark work, restart the session and make the following the first cell. %%configure must run before spark is initialized.
  • Put this as the very first cell and paste the lakehouse-Id and Workspace-Id you got:
    %%configure -f
    {
    "defaultLakehouse": {
    "name": "test",
    "id": "lakehouse-Id",
    "workspaceId": "Workspace-Id"
    }
    }
  • And then run this, it will work
    spark.sql("DESCRIBE TABLE test.dbo.publicholidays").show()

 

Thanks and regards,

Anjan Kumar Chippa

danitskofabric's avatar
danitskofabric
Frequent Visitor
1 year ago

Hi v-achippa,

Thank you for your reply. Yes, I figured that out, but what I’m trying to achieve is that I don’t have to specify any IDs, so the same code works in different workspaces and automatically uses the lakehouse from the current workspace. Unfortunately, this seems only to work for lakehouses without schema enabled.

 

  • v-achippa's avatar
    v-achippa
    Icon for Community Support rankCommunity Support
    1 year ago

    Hi danitskofabric,

     

    Currently %%configure with only the lakehouse name works only when schemas are disabled.
    When schemas are enabled, fabric requires both the lakehouse Id and workspaceId for binding, so fully portable notebooks need to fetch these Id’s dynamically at runtime. As of now there is no way to avoid Id’s completely, it is a current limitation of schema enabled lakehouses.

     

    I recommend submitting your detailed feedback and ideas through Microsoft's official feedback channels. Feedback submitted through these channels is frequently reviewed by the product teams and can contribute to meaningful improvements.

    Fabric Ideas - Microsoft Fabric Community

     

    Thanks and regards,

    Anjan Kumar Chippa