Lakehouse Tables Not Appearing in SQL Endpoint Schema

[BCBSTnathan]

Hello everyone,

I’m using notebooks to create several lakehouse tables from CSV files. The tables are being created successfully in the lakehouse, but they are not appearing under the lakehouse SQL endpoint schema.

Has anyone encountered this issue before or have any suggestions on what might be causing this? Any help or guidance would be greatly appreciated!

Thank you in advance!

[BCBSTnathan · Accepted solution]

Thank you all for your support! I was able to successfully create the delta tables, and they are now visible in the SQL endpoint. I must admit, the issue was due to blank spaces in the column names, and a blank line at the end of the files. Once I removed those, everything worked OK!

Through this process, I also added code to catch exceptions and handle blank spaces, which will help similar issues in the future. 

A special thank you to @v-huijiey-msft and @AndyDDC  -- I appreciate you both for checking in and providing support. Thank you!

[v-huijiey-msft]

Hi @BCBSTnathan ,

Thanks for the reply from AndyDDC .

If you have enabled the preview schema feature of lakehouse, then this is a known issue. Delta tables may not appear in the SQL analysis endpoint when the preview schema feature of lakehouse is enabled.

There is currently no workaround for this issue, but the team is actively working on a fix.

Known issue - Tables aren't visible in the SQL analytics endpoint in schema enabled lakehouse - Micr...

This is a test I did when I did not enable the preview schema feature of lakehouse, loaded the csv file to the table using the following statement, switched to SQL endpoint view, and you can see that it works fine.

# Create a new table
df.write.format(“delta”).saveAsTable(“data”)

I hope this is helpful to you!

Best Regards,
Yang
Community Support Team

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

[BCBSTnathan]

Hi @v-huijie-msft  and @AndyDDC Thank you both for your responses. I confirmed that the Lakehouse preview option was unchecked, and the delta format was set using the SaveAsTable command. Despite these settings, the tables still aren’t appearing under the Lakehouse SQL endpoint immediately.

Interestingly, some tables appeared today after a couple of days. I ran the Spark notebook again to see if it would resolve the issue, but there was no change. I plan to check the Lakehouse SQL endpoint again tomorrow to see if the tables appear. If tables are delayed by several hours or more before showing up, this behavior wouldn’t be acceptable for a production environment.

I appreciate any additional insights or suggestions you might have on addressing this delay. Thanks again for your help!

[v-huijiey-msft]

Hi @BCBSTnathan ,

Here are a few suggestions that might help:

Sometimes, there can be a delay in the synchronization between the Spark catalog and the SQL endpoint. This might explain why some tables appeared after a couple of days.

There could be a schema mismatch issue. Make sure that the schema in which the tables are created matches the schema expected by the SQL endpoint.

Someone here has had a similar problem to you, please see if that helps:

Solved: Delta Tables in LakeHouse missing in SQL Endpoint - Microsoft Fabric Community

Best Regards,
Yang
Community Support Team

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

[AndyDDC]

Hi @BCBSTnathan @can you share how you're creating the tables in your Notebook? Only Delta tables are supported in the SQL Endpoint.  EG if you create external tables directly over CSV, you won't see those tables in SQL Endpoint.

Also, is your Lakehouse enabled for the new schema public preview? EG you can create additional schemas inside the Lakehouse? If so then currently SQL Endpoint is not supported so you won't see any tables appearing.