topic Couldnt load table error - but can preview data fine in Pipelines

Couldnt load table error - but can preview data fine

DanielAmbler — Thu, 27 Jun 2024 14:18:11 GMT

Hi,

After writing a dataframe to the Lakehouse as a table

MyTable.write.format("delta").mode("overwrite").saveAsTable("MyTable") (N.B - also tried using save( full ABFSS path)

- The table is created fine, displays in the notebook source explorer and lakehouse explorer

- I can preview the data in lakehouse explorer

however

- When trying to expand the table columns in Lakehouse Explorer - the table displays a red cross with an error 'Couldnt load table' - the message is 'Server Error' ' The requested Lakehouse operation failed' (details below)

- Cannot see the table via the SQL endpoint.

Where do I begin troubleshooting?

Server Error.
Message : The requested Lakehouse operation failed. A retry of the operation may help.
Error Code : LakehouseOperationFailed
Activity Id (Failed Request) : a2d819e4-3c43-4ce3-8556-79ad718ce623
Client Session Id : ad30364c-9c09-4754-9048-64f5ee65025c
Activity ID (Initial Request) : 40fb6e11-fdf6-45cc-a794-f5ff89d99362
User session Id : e2c056cd-1317-41a5-9f33-c1ea87dfffc9

Re: Couldnt load table error - but can preview data fine

v-cboorla-msft — Fri, 28 Jun 2024 09:23:20 GMT

Hi DanielAmbler

Thanks for using Microsoft Fabric Community.

While you mentioned using "delta" format for writing, ensure the data format in the table is compatible with Lakehouse Explorer's preview functionality. When you write your DataFrame to the Lakehouse as a Delta table, there might be a mismatch between the data types in your DataFrame and the data types expected by Lakehouse Explorer for displaying the table schema.

If a column in your DataFrame contains data of a type not natively supported by Lakehouse Explorer for previewing, it could cause issues when expanding the table columns. For example, complex data types like nested structures might not be easily displayed. Before writing to Delta format, inspect the data types of each column in your DataFrame.

For more details please refer : Data types - Microsoft Fabric | Microsoft Learn

I hope this information helps.

Thank you.

Re: Couldnt load table error - but can preview data fine

DanielAmbler — Fri, 28 Jun 2024 14:48:50 GMT

Hi v-cboorla-msft - Thanks for the reply. I had a complex explicit schema defined with quite a few columns that had to be aliased in the final dataframe.

It was an aliased column that had a name of "" (i.e blank) that was causing the issue.

It wasnt a data type in the end but your response did prompt me to have a good look at the dataframe with some fresh eyes and gave me some ideas for building some basic sanity checks for the future.

Re: Couldnt load table error - but can preview data fine

v-cboorla-msft — Fri, 28 Jun 2024 16:06:18 GMT

Hi DanielAmbler

Glad that your query got resolved.
Please continue using Fabric Community for further queries.

Thank you.