Forum Discussion

dzav's avatar
dzav
Advocate III
1 year ago
Solved

SQL endpoint sync issues

I've been experiencing 3 different issues related to the SQL endpoint not being refreshed quickly enough or at all when the undelying delta table is updated with new data. 1) A pipeline will fail wh...
Pipelines
  • dzav's avatar
    dzav
    1 year ago

    frithjof_v , thank you! Looks like that was the optimal solution. I now how the following pattern:

    1) Load the bronze lakehouse

    2) A copy task to load the bronze warehouse from the bronze lakehouse

    3) A stored procedure to copy from the bronze warehouse to the silver warehouse

     

    I don't know how scalabale this is because we have over a hundred tables from Synapse Link and adding tables to a copy task is a bit onerous, but for now this seems to be working.

dzav's avatar
dzav
Advocate III
1 year ago

Well, this is still not working. Despite using this statement to wait until the sql analytics endpoint table is created:

WHILE OBJECT_ID([lakehouse].dbo.[table]) IS NULL

 

Subsequent queries still fail with:

Failed to complete the command because the underlying location does not exist. Underlying data description: table 'dbo.table', file 'https://onelake.dfs.fabric.microsoft.com/guid/guid/Tables/table/part-guid.snappy.parquet'.

 

This makes no sense.