Multi-geo capacity error

[dbeavon3]

Long ago we had accidentally set our default Power BI region to West US, even though the business resides in the eastern part of the USA. 

Recently we corrected the problem by moving some premium workspaces to East US capacity.  Our Power BI admin moved the workspaces, and mine was among them.

However now I'm facing an interesting error whenever I interact with my datasets (to refresh them, or to connect to them via XMLA endpoints):

{"error":{"code":"LargeModelMovedAcrossRegions","pbi.error":{"code":"LargeModelMovedAcrossRegions","parameters":{"regionName":"West US"},"details":[],"exceptionCulprit":1}}}

Technical Details:
RootActivityId: a7ce0692-6932-4416-97de-dc451cb0eeee
Date (UTC): 10/19/2021 1:40:00 AM (Microsoft Analysis Services)

The message is not appearing in any google searches.  

I've tried the obvious things, like deleting and republish the dataflow and dataset.  Eventually I was also able to refresh my data flow and dataset.  Some of the places where I used to get the error are now fixed.

However other things are still broken, like using the XMLA endpoint to connect from SSMS or DAX studio.  When I try to connect to my datasets with those tools, I still see the error message.

I'm not sure how we dug ourselves so deeply into a hole.  As far as I was told, we just migrated the workspaces per the documentation.  Any tips would be appreciated!

[dbeavon3 · Accepted solution]

I ended up having to delete just about everything in that workspace.  Then I reloaded it all back in again manually.

Connecting from SSMS (via the XMLA endpoint) is probably the best way to verify when your workspace is kosher again.   If that connection fails then you have probably not deleted enough of the old cruft yet.  I'm guessing here ... but I think that after the GEO migration, half of the underlying contents may get left behind in the old storage, without any easy way to move it, clear it, or refresh it.

That error ("LargeModelMovedAcrossRegions") was pretty persistent and couldn't shake it without quite a lot of effort.  Of course there is nothing in the portal U/I to suggest *which* artifacts in the workspace are the offending ones.  If anyone else encounters this, I suggest you resign yourself to the fact that you will need to clear everything out of your workspace and then load it all in again.  I think the folks who designed this GEO migration were content to give us a half-baked solution.

As with many things in Power BI, some of the ideas for the product are pretty good, but the execution is not so good (to put it very lightly).  On top of that, the tech support is not very good either. </venting>

[dbeavon3 · Accepted solution]

I ended up having to delete just about everything in that workspace.  Then I reloaded it all back in again manually.

Connecting from SSMS (via the XMLA endpoint) is probably the best way to verify when your workspace is kosher again.   If that connection fails then you have probably not deleted enough of the old cruft yet.  I'm guessing here ... but I think that after the GEO migration, half of the underlying contents may get left behind in the old storage, without any easy way to move it, clear it, or refresh it.

That error ("LargeModelMovedAcrossRegions") was pretty persistent and couldn't shake it without quite a lot of effort.  Of course there is nothing in the portal U/I to suggest *which* artifacts in the workspace are the offending ones.  If anyone else encounters this, I suggest you resign yourself to the fact that you will need to clear everything out of your workspace and then load it all in again.  I think the folks who designed this GEO migration were content to give us a half-baked solution.

As with many things in Power BI, some of the ideas for the product are pretty good, but the execution is not so good (to put it very lightly).  On top of that, the tech support is not very good either. </venting>

[KilianG]

I had ta similar issue -  mine was caused by the more or less accidentally setting a workspace to large dataset format.
Unfortunately setting the workaüace(s) to small dataset format again, forced meto rebuild the whole pipeline, since i had to recreate the workspaces.
This was very annying since the dashboards werde embedded and i had to reconfigure the embedding too.
An alert on choosing "large dataset format" (or the geo region) showing that this step can't be undone (under most circumstances) would have been a nice feature.

[lbendlin]

If you have a Pro license you should consider raising a Pro ticket at https://powerbi.microsoft.com/en-us/support/pro/

[dbeavon3]

Thanks Ibendlin, I am already working thru another support ticket.  It is taking much, much longer than it should.  I hoped someone had already encountered this message, and that might save me the trouble of working for several days with the PBI tier 1 support (... capturing irrelevant Fiddler traces in my browser, and what-not). 

[lbendlin]

Sorry to hear that - we had really good support experience via Pro tickets so far.  Hope yours gets the attention it deserves. Let me know if you want me to help push the ticket.  (I am interested in the outcome as we have a similar setup and may run into these issues too).

[dbeavon3 · Accepted solution]

I opened a Pro ticket and they said that this error is a customer responsibility.

Here are relevant docs:
https://learn.microsoft.com/en-us/fabric/admin/service-admin-premium-multi-geo?tabs=power-bi-premium...

"It's possible to create and maintain large-storage format semantic models in remote regions to meet data residency requirements. However, you can't move storage format semantic models to another region. Moving large-storage format semantic models from the region where they were created results in reports failing to load the semantic model.

Move the large-storage semantic model back to its original region to make it available. If you must move such a model, deploy it as if it was a new model, and then delete the old model from the undesired region.
"

Basically they want customers to redeploy and possibly lose metadata in the process.

What is really annoying is that even something like an XMLA clear command won't work, over XMLA endpoints.  That should clear out the data, but for this particular scenario, it won't work for some reason.

My next attempt is to use SSMS to generate an XMLA-alter statement for the whole dataset, and try to re-run that against the existing one.  Worth a shot.

[dbeavon3]

Another update.  None of the xmla endpoint commands are possible on a corrupted model.  Even an attempt to generate a CREATE script fails like so:

{"error":{"code":"LargeModelMovedAcrossRegions","pbi.error":{"code":"LargeModelMovedAcrossRegions","parameters":{"regionName":"East US"},"details":[],"exceptionCulprit":1}}}

... seems like they could have at least allowed us to retrieve schema before we blow one of these models away.

Obviously this scenario wasn't worthy of too much attention from Microsoft.  They allowed customers to bear full responsibility for any fall-out that comes after moving from one region to another.  It ain't pretty.