Data Warehouse is inaccessible overnight

[coverturfLSH]

Hi All,

Documenting my issue here in hopes of anyone offering a possible solution or me sharing a solution if Microsoft can assist. I've already submitted a ticket (TrackingID#2405160010002716) with them.

The problem I'm having is in a "Premium Capacity" workspace. There are dataflow gen2, semantic model, lakehouse, and warehouse objects in the workspace. Starting yesterday morning, My warehouse has unable to "start" and I am unable to query or view tables in the warehouse web/local application. The issue seems to be isolated to this specific workspace as I am able to view data warehouses in other workspaces. Also, the lakehouse tables are viewable (at the moment). 

The various errors I have received are an infinite "Loading" screen (30 min+), error pop-up "We couldn't get your warehouse's updated schema", error pop-up "Internal error, we failed while handling this action: [Datamart Model] Load datamart schema failed.

When using the developer tools to track the network activity, it was failing on the "authentication" step. 

As mentioned, I've escalated the issue with Microsoft support, but if anyone has had the issue before or can offer suggestions I'm open to anything at this point.

Thanks,

[zhihaowu208]

@coverturfLSH , I have the same issue.
It all started after the may 2024 update. For me, after 48 hours, it is still an issue. Can you explain how you submitted the ticket? Maybe that ticket has gave someone a headsup to fix something. Which I need now 😛 

[coverturfLSH]

Hi @zhihaowu208,

In PowerBI, there's a help & support icon in the upper right. Then select "Get Help". That will take you to this webpage https://support.fabric.microsoft.com/en-US/support. 

On the support page, I selected "Synapse Data Warehouse > Get Support"

Type in a description of your problem. Then select the "Support" tab at the top.

Best of luck, as of right now still no explanation of what caused the issue and what fixed it. I am curious though, have you been running "Notebooks" in the workspace that has the warehouse failure?  

[zhihaowu208]

Thanks @coverturfLSH , I was able to submit a ticket on Monday. So far issue is not resolved by Microsoft Unified Support.

And answer to your question regarding to using notebook in the workspace. Yes, I do. 

[coverturfLSH]

@zhihaowu208 Hope they can find an answer for you. 

The reason I asked about the Notebook was I got this error in one of my dataflows. 

A long story short, in my searching about this error I came across a post that said that Notebooks weren't properly terminating sessions after the scripts were run. This was essentially creating a new session for each notebook that was run. My hunch is if you/I ran your script X number of times in a given time period, there were too many "active sessions" and it wouldn't let you authenticate any more "new sessions" aka opening the warehouse. 

Have you been continuing to run Notebooks in the last 48 hour period while you've been unable to access the warehouse? I mostly stopped after I got my first warehouse timeout and it fixed itself (or sessions logged out?) after the 48 hour period. 

[zhihaowu208]

I see what you mean. Unfortunately, I don't have any open sessions as far I can see in the Spark active sessions.

I have turned off all schedule process since last week, still no avail.

[coverturfLSH · Accepted solution]

The workspace "self-healed" itself after about 48 hours from the initial issue. No explanation of what caused the problem or how it was resolved. 

[Anonymous]

Hi @coverturfLSH ,

Thanks for using Fabric Community and sharing the Support ticket Number.

Please allow some time, so team can check and provide a resolution.

In case if you got a resolution please do share with the community as it can be helpful to others .