Forum Discussion

TGTun's avatar
TGTun
Frequent Visitor
4 months ago
Solved

Schema not appeared in Lakehouse SQL Endpoint UI tree

Hi Team,

 

We created new schema from Notebook connected with Lakehouse and all delta tables under new schema from Notebook pyShark SQL. All created tables with new schema are visible in Lakehouse but in SQL endpoint UI tree is not visible. We checked in information_schemas, new schema with tables are available there. We can see also in SSMS connectivity. It only happened in the specific workspace. We have Dev and Test workspace and created the same with Notebook. No issue on viewing in SQL endpoint UI tree. 

 

Becoz of this, we couldn't create Semantic Model with those tables or views via that SQL endpoint.  Tried below options

1. Manually force refresh in SQL Analytic endpoint page. 

2. Refresh SQL endpoint metadata via REST API from here > Items - Refresh Sql Endpoint Metadata - REST API (SQLEndpoint) | Microsoft Learn

    All those tables are returned status with "NotRun" so no changes since the last sync

3. Sanity check in sys.schema and sys.tables filtered by schema name (schema and table list are returned as expected)

 

Looking forward to your advice.

Thanks in advance. 

TG tun

  • Hi TGTun ,

    Thanks for reaching fabric community, will happy to assist.

     

    This is a known metadata sync issue between the Lakehouse and the SQL Analytics Endpoint. The fact that your schema/tables are visible in information_schema, sys.schema, and SSMS but NOT in the SQL Endpoint UI tree confirms it's a UI metadata cache problem, not a data or permissions issue.

    Here's how to fix it step by step:

     

    Step 1 Force Metadata Refresh via Notebook (Most Effective)

    Run this in a Fabric Notebook attached to your Lakehouse:

    # Refresh individual tables in the custom schema
    spark.sql("REFRESH TABLE your_schema.your_table_name")
    
    # OR refresh the entire catalog
    spark.catalog.refreshTable("your_schema.your_table_name")

    You can also force update table metadata by re-saving the table to trigger a sync: Microsoft Community

    df = spark.read.table("your_schema.your_table_name")
    df.write.mode("overwrite").saveAsTable("your_schema.your_table_name")

     

    Step 2 Force Metadata Refresh via T-SQL in SQL Endpoint

    Open the SQL Endpoint query editor and run:

    ALTER TABLE your_schema.your_table_name REFRESH METADATA;

    This forces a resync between OneLake and the SQL endpoint, re-registering the table schema. Microsoft Community

     

    Step 3 Trigger Internal Metadata Rebuild (Proven Workaround)

    Writing a table into the new schema using a notebook causes the new schema and the new table to appear in the SQL endpoint UI. Microsoft Community So if Step 1 & 2 don't work, try writing any small dummy/real table into the affected schema via notebook this forces Fabric to rebuild the schema metadata tree.

     

    Step 4 REST API Metadata Refresh

    Since your REST API returned "NotRun" (no changes since last sync), the endpoint believes it's already up to date. Try forcing a full sync using the Fabric REST API:

     

    POST https://api.fabric.microsoft.com/v1/workspaces/{workspaceId}/lakehouses/{lakehouseId}/sqlEndpoints/refreshMetadata

     

    You can also use a Python script via sempy.fabric to fetch SQL Endpoint properties, trigger a metadata refresh, and monitor the sync status until it completes. obvience

     

    Why This Happens

    The SQL endpoint metadata cache sometimes doesn't refresh automatically, especially after schema changes or delayed commit operations. The Lakehouse view shows the data because it reads directly from the file system, while the SQL endpoint relies on cached metadata. Microsoft Community

     

    The fact that it works in your Dev workspace but not Test suggests a workspace-level metadata desync this is a platform-side inconsistency, not something you did wrong.

     

    If Nothing Works

    • Raise a Microsoft Support ticket this is a platform-level metadata sync failure that may need backend intervention
    • Provide them: Workspace ID, Lakehouse ID, Activity ID from your error logs

    Fix Priority Order

    Step

    Action

    1st

    REFRESH TABLE in Notebook

    2nd

    ALTER TABLE REFRESH METADATA in SQL Endpoint

    3rd

    Write a table into the schema via Notebook

    4th

    REST API full metadata sync

    Last

    Raise Microsoft Support ticket

     

    Hope this resolves it! Please mark as Accepted Solution if it helps others.

     

     

     

  • TGTun's avatar
    TGTun
    4 months ago

    Hi Lodha_Jaydeep, 

     

    It worked after manually refreshed. We dropped and recreated some views in SQL endpoint that is likely issue on different schema with same view name.  

     

    Appreciate on the steps to resolve. 

     

    Thank you so much.

    TG tun

2 Replies

  • Hi TGTun ,

    Thanks for reaching fabric community, will happy to assist.

     

    This is a known metadata sync issue between the Lakehouse and the SQL Analytics Endpoint. The fact that your schema/tables are visible in information_schema, sys.schema, and SSMS but NOT in the SQL Endpoint UI tree confirms it's a UI metadata cache problem, not a data or permissions issue.

    Here's how to fix it step by step:

     

    Step 1 Force Metadata Refresh via Notebook (Most Effective)

    Run this in a Fabric Notebook attached to your Lakehouse:

    # Refresh individual tables in the custom schema
    spark.sql("REFRESH TABLE your_schema.your_table_name")
    
    # OR refresh the entire catalog
    spark.catalog.refreshTable("your_schema.your_table_name")

    You can also force update table metadata by re-saving the table to trigger a sync: Microsoft Community

    df = spark.read.table("your_schema.your_table_name")
    df.write.mode("overwrite").saveAsTable("your_schema.your_table_name")

     

    Step 2 Force Metadata Refresh via T-SQL in SQL Endpoint

    Open the SQL Endpoint query editor and run:

    ALTER TABLE your_schema.your_table_name REFRESH METADATA;

    This forces a resync between OneLake and the SQL endpoint, re-registering the table schema. Microsoft Community

     

    Step 3 Trigger Internal Metadata Rebuild (Proven Workaround)

    Writing a table into the new schema using a notebook causes the new schema and the new table to appear in the SQL endpoint UI. Microsoft Community So if Step 1 & 2 don't work, try writing any small dummy/real table into the affected schema via notebook this forces Fabric to rebuild the schema metadata tree.

     

    Step 4 REST API Metadata Refresh

    Since your REST API returned "NotRun" (no changes since last sync), the endpoint believes it's already up to date. Try forcing a full sync using the Fabric REST API:

     

    POST https://api.fabric.microsoft.com/v1/workspaces/{workspaceId}/lakehouses/{lakehouseId}/sqlEndpoints/refreshMetadata

     

    You can also use a Python script via sempy.fabric to fetch SQL Endpoint properties, trigger a metadata refresh, and monitor the sync status until it completes. obvience

     

    Why This Happens

    The SQL endpoint metadata cache sometimes doesn't refresh automatically, especially after schema changes or delayed commit operations. The Lakehouse view shows the data because it reads directly from the file system, while the SQL endpoint relies on cached metadata. Microsoft Community

     

    The fact that it works in your Dev workspace but not Test suggests a workspace-level metadata desync this is a platform-side inconsistency, not something you did wrong.

     

    If Nothing Works

    • Raise a Microsoft Support ticket this is a platform-level metadata sync failure that may need backend intervention
    • Provide them: Workspace ID, Lakehouse ID, Activity ID from your error logs

    Fix Priority Order

    Step

    Action

    1st

    REFRESH TABLE in Notebook

    2nd

    ALTER TABLE REFRESH METADATA in SQL Endpoint

    3rd

    Write a table into the schema via Notebook

    4th

    REST API full metadata sync

    Last

    Raise Microsoft Support ticket

     

    Hope this resolves it! Please mark as Accepted Solution if it helps others.

     

     

     

    • TGTun's avatar
      TGTun
      Frequent Visitor

      Hi Lodha_Jaydeep, 

       

      It worked after manually refreshed. We dropped and recreated some views in SQL endpoint that is likely issue on different schema with same view name.  

       

      Appreciate on the steps to resolve. 

       

      Thank you so much.

      TG tun