Data Display Issue in Warehouse

[tegrus-coutinho]

I'm encountering an issue with data display in my table in the Warehouse. I perform daily incremental loads to this table. However, I noticed that the load from the day before yesterday seems to be missing, which is unusual since I checked the data pipeline and it executed without any issues.

Upon investigating, I accessed the lakehouse and confirmed the presence of the .parquet files corresponding to the missing load. However, these data are strangely not appearing in my table in the Warehouse.

I would like to know if anyone has experienced a similar issue and how it was resolved. I appreciate any guidance or assistance you can provide in advance.

[Joshrodgers123 · Accepted solution]

I have seen the same thing. Support told me there is a sync issue between the delta logs in the lakehouse and the sql endpoint. I have many instances where the sql endpoint does not update and shows an older version of the delta tables. They said the only way to fix it is to manually rename the delta table and rename it back.

[fabricator1]

Do you mean Data Warehouse or the SQL Analytics Endpoint in the Lakehouse?

Did you try accessing the data in Data Warehouse and SQL Analytics Endpoint and Power BI, and see if it is the same situation everywhere or if it is only missing in the Data Warehouse?

[tegrus-coutinho]

In both SQL Endpoint and Power BI, the data is not appearing. I was only able to see the .parquet file within the Lakehouse.

[frithjof_v]

I guess I would try to look in the Delta log files.