Forum Discussion
Lakehouse SQL endpoint
As my issue is ETL / pipeline related, neither of that helps.
What I did now as a soloution until MS releases the API which frithjof_v mentions, is to use the "unsupported script" to update SQL endpoints written by somone at MSFT found here:
What I did, if someone is interested to:
- Rewrote the script to be a bit more robust, with some error handling etc.
- Created it as a wheel, uploaded in a shared lakehouse, call it "own modules".
- Shortcuted into "own modules" in every lakehouse where a notebook updates the data & a sql endpoint needs a refresh.
- At end of each notebook which updates lakehouses, i run the script with error handling & logging, so if the unsupported APIs in the script gets changed/stops working, it won't affect the pipeline run, but I will know from logs.
- Wait for the official API to come.
Best regards,
Alexander
Hi,
I was about to create a topic on what I believe is a similar issue.
In my pipeline, I have an audit step that utilizes SQL endpoints for auditing. However, these checks often fail falsely because the queries seem to be based on outdated data. They do not reflect updates that have just occurred in the pipeline. For example, if Table X initially has 100 records and the pipeline inserts 50 more, the table count still shows 100 instead of 150.
I believe this is related to the issue being discussed. Is the only solution coming up the one refered about API update trigger?
//Alexander
Hi p_da ,
AlexanderPowBI shared a potential workaround. If possible, please review the response and give it a try to see if it resolves the issue
Thank You.