Forum Discussion
Microsoft EmployeeLakehouse error: "An internal error has occurred while applying table changes to SQL"
Hi RishikeshPashte,
The issue here is that the EventStream writes are succeeding but SQL endpoint queries not reflecting the latest data, this looks like a metadata sync issue between the Spark engine and the SQL analytics endpoint in microsoft Fabric.
Create a new Lakehouse or new table and use a Fabric notebook to manually copy the data using Spark. This ensures that the metadata is clean and synchronized correctly with the SQL endpoint. Once done, refresh the SQL endpoint and re run the query, this should now reflect the latest data.
If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it!
Thanks and regards,
Anjan Kumar Chippa
Community SupportHi @RishikeshPashte,
We wanted to kindly follow up to check if the solution I have provided for the issue worked.
If my response addressed, please mark it as "Accept as solution" and click "Yes" if you found it helpful.
Thanks and regards,
Anjan Kumar Chippa
Microsoft EmployeeHey v-achippa ,
I created a new table, dumped the previous data in the new table and again set up an EventStream for the new table but, it's again giving me the same error as mentioned above
Also, when I query the table via the SQL endpoint it doesn't show me the latest data whereas, when I query the table using a Fabric notebook it fetches the lata data from the table. Please find the screenshots for the same below:
Queried using SQL Endpoint:
Queried using Fabric notebook:
- v-achippa
Hi RishikeshPashte,
The issue here is that the EventStream writes are succeeding but SQL endpoint queries not reflecting the latest data, this looks like a metadata sync issue between the Spark engine and the SQL analytics endpoint in microsoft Fabric.
Create a new Lakehouse or new table and use a Fabric notebook to manually copy the data using Spark. This ensures that the metadata is clean and synchronized correctly with the SQL endpoint. Once done, refresh the SQL endpoint and re run the query, this should now reflect the latest data.
If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it!
Thanks and regards,
Anjan Kumar Chippa
- v-achippa
Hi @RishikeshPashte,
As we haven’t heard back from you, we wanted to kindly follow up to check if the solution I have provided for the issue worked.
If my response addressed, please mark it as "Accept as solution" and click "Yes" if you found it helpful.Thanks and regards,
Anjan Kumar Chippa