Imjkrao's avatar
Imjkrao
Icon for Advocate I rankAdvocate I
1 year ago
Status:
New

Mirror DB's SQL Endpoint to be refreshed using the Fabric API Endpoi -Refresh Sql Endpoint Metadata

Mirror DB's SQL Endpoint METADATA to be refreshed using the Fabric API Endpoint -Refresh Sql Endpoint Metadata

Currently, when files are delivered to the landing zones of the mirrorDB, a script is run against the SQL endpoint to move data to the warehouse using a script activity or a notebook activity in the data pipeline. The pipeline fails and throws the following error for a few of the tables in the mirrorDB, unable to find the files for the required data.

 

Notebook execution failed at Notebook service with http status code - '200', 
please check the Run logs on Notebook, additional details - 'Error name - DatawarehouseExecuteSqlError, 
Error value - Failed to complete the command because the underlying location does not exist. 
Underlying data description: table 'dbo.Tablename', file 'https://onelake.dfs.fabric.microsoft.com/a40942ee-3dfa-47ed-9ff0-8215f19a5773/ee162d20-8423-411d-b49b-c2ae0187db94/Tables/dbo/Tablename/part-00000-fde0ba3c-4315-402a-97fa-4cc3ccb4da0e.c000.zstd.parquet'. Data warehouse trace id: a27050b9-28fc-4cf4-9722-91e4a42e0b94' :


I believe there is a delay in the sync between mirrordb vs mirrordb's sql endpoint. It would be great if you could make mirrorDB's SQL endpoint available to refresh through the Fabric API Endpoint -Refresh Sql Endpoint Metadata.

Currently, when I try to refresh the metadata, I get the following error in Postman.

 

 

{
    "requestId": "8a5cbfe5-02ed-4525-a9a7-8e9544631586",
    "errorCode": "EntityNotFound",
    "message": "The requested resource could not be found"
}


Thank You 
Jeevan

 

 

data factory
Data Factory | Mirroring
Data Factory | Pipelines
fabric platform

1 Comment

  • Imjkrao's avatar
    Imjkrao
    Icon for Advocate I rankAdvocate I
    11 months ago

    It is now possible to refresh the mirrorDB's SQL Analytics endpoint using the REST API. This also supports LRO.

     

    conn = http.client.HTTPSConnection("api.fabric.microsoft.com")
payload = json.dumps({
  "timeout": {
    "timeUnit": "Seconds",
    "value": "60"
  }
})
headers = {
  'Content-Type': 'application/json',
  'lro_wait': 'False'
}
conn.request("POST", "/v1/workspaces/{WorkspaceId}/sqlEndpoints/{sqlendpointId}/refreshMetadata", payload, headers)

Recent ideas