Forum Discussion
write_deltalake with Python Notebook is creating an "Unidentified" folder in my lakehouse.
Hello joshuaking
Here is what I am thinking could happened.
when writing a Delta table using Python functions like `write_deltalake` or Polars’ `write_delta`, the underlying data files are correctly created and queryable, yet the lakehouse metadata may not recognize or register the table properly. This can lead to the table being shown under an “Unidentified” folder
Writing data to a specific path does not always automatically update the lakehouse catalog.
try using ddl command like in your notebook
`CREATE TABLE student USING DELTA LOCATION '/lakehouse/your_schema/your_path/student';`
to inform the system about the logical table nameif this helps please accept the solution and give kudos
Yes it is automatic in Fabric
When files in Delta format (Parquet + `_delta_log` folder) are placed in the managed area of a Lakehouse, Fabric automatically registers them as tables in the catalog.
In most thr cases properly structured delta file from Unidentified folder eventually moves to Table
Hello joshuaking
Here is what I am thinking could happened.
when writing a Delta table using Python functions like `write_deltalake` or Polars’ `write_delta`, the underlying data files are correctly created and queryable, yet the lakehouse metadata may not recognize or register the table properly. This can lead to the table being shown under an “Unidentified” folder
Writing data to a specific path does not always automatically update the lakehouse catalog.
try using ddl command like in your notebook
`CREATE TABLE student USING DELTA LOCATION '/lakehouse/your_schema/your_path/student';`
to inform the system about the logical table name
if this helps please accept the solution and give kudos
I'll give this a try and see what happens. While I'm working on it, do you know if the lakehouse catalog updates on its own periodically? If not, what do you assume would be the best way to register new tables automatically?
Yes it is automatic in Fabric
When files in Delta format (Parquet + `_delta_log` folder) are placed in the managed area of a Lakehouse, Fabric automatically registers them as tables in the catalog.
In most thr cases properly structured delta file from Unidentified folder eventually moves to Table