Warehouse data not refreshing when pulling from Lakehouse

Hi marius1106,

Based on my understanding, I recommend first copying the raw data into a Warehouse table and then applying T-SQL transformations using views or additional logic. This approach helps avoid delays associated with the SQL endpoint and ensures that you are working with the most recent data in a stable environment that is optimised for transformations.

Please note that Notebooks may fail, as T-SQL Notebooks rely on the Lakehouse SQL analytics endpoint, which can experience metadata synchronisation delays. These delays may affect timely access to data, even for existing tables.

Other alternatives, such as introducing wait or delay logic, may provide some relief but are not entirely reliable. Similarly, using a view over Lakehouse tables is a potential solution, but it may also be impacted by the same metadata sync issues.

If you find this response helpful, we kindly request you to mark it as the accepted solution and consider giving kudos. This will help other community members who may have similar queries.

Thank you.