Issue - Recovering sources (Warehouse, etc.) from Azure Dev Ops in a blank Workspace

[vivien57]

Hello everyone,

The situation is the following, I have a workspace which is connected to git, in which the synchronization is up to date.

Then, without using the deployment pipelines, I want to pull the contents of this workspace (which is therefore on Azure Dev Ops, cf picture) into an empty workspace.

I map the git repo with the empty workspace and the elements start to be created ... then I get an error

What's strange is that the error is “Datamart Not FOund” even though I don't have a datamart in the Workspace.

Isn't this a Warehouse dataset by default, for which it can't make the link? Is it the fact that it can't recreate the folders, as in the error I encountered here?

https://community.fabric.microsoft.com/t5/General-Discussion/Issue-Deployment-pipeline-and-folders-d...

Have you encountered this error?

Thanks in advance for your feedback,

Vivien

[rocketporg]

@vivien57 @v-nikhilan-msft 

I get the same error, it looks like warehouses aren't working properly with deployment pipelines or git branching. You can reproduce this error in several ways for example:-

a) create empty workspace

b) connect it to a git repo

c) create an empty warehouse

d) try the 'branch out to new workspace' option under source control

e) receive the error 'datamartnotfound'

Also, you can also get the same error if you:-

a) clone an existing Fabric git repo with a warehouse in it

b) create an empty workspace

c) link the workspace to the cloned repo

d) wait for it to try and 'sync' the items in the repo and it fails on the first time it tries to sync a warehouse... same error 'datamartnotfound'

Note, if you remove the warehouse(s) from the repo/workspace then all the above examples work fine. I've tried this several times with different workspaces and happens every time...

[rocketporg]

A small update @vivien57 @v-nikhilan-msft 

Looks like the Fabric devs have just deployed some changes to Git integration in Fabric as you can now choose between a repo in Azure DevOps or GitHub. So, I tried cloning the repo with a warehouse in it (same repo as used above) and connected it to a new workspace, this time I don't get the 'datamartnotfound' error, but instead it clones the warehouse artifacts then flags them as 'uncommitted', despite them being in the repo!?! Oh dear... note that only warehouse artifacts seem to be affected, see the screen shot below...

As expected, Microsoft are continually updating/fixing Fabric and this looks like a small update released in the last day or so. So maybe, 3rd time lucky and they might finally fix warehouses and Git integration completely. I'll keep checking every now and then to see if it gets fixed...

[vivien57]

@rocketporg Great, thank you very much for your feedback

[rocketporg]

Hi @vivien57 / @v-nikhilan-msft 

As per the July Fabric update, I've tried this again. They've said there are new features and improvements with the warehouse and CI/CD etc... So, it looks like its getting better, but still not totally fixed (we'll not for me, you might be ok now).

I now get an error 'Variables are not allowed in the CREATE TABLE statement' when it tries to sync/clone a warehouse artifact.

I suspect this is Fabric getting upset with the following code in one of my stored procedures where I'm using a CTAS statement. Even though this proc works perfectly fine and the code is valid (the variables are created many lines before this little bit of code). I'd imagine that the engine (or rather the thing that is parsing the code whilst syncing it from Git into the workspace) is maybe getting 'confused' with trying to actually use a variable in a CREATE table statement (which will error and won't run) as opposed to a CTAS statement. Like I said, this proc works and the code runs fine:-

DROP TABLE IF EXISTS [dbo].[DataSourceStaging];
CREATE TABLE [dbo].[DataSourceStaging] AS
SELECT
[Dataset] = @dataset
,[Subset] = @subset
,[Url] = @url
,[DataFormat] = UPPER(@format)
,[HasHeaderRow] = @hasHeaderRow
,[JoinColumns] = @joinColumns	

So maybe your problem is fixed, but mine are still there unfortunately 😉

I think Fabric warehouse maybe needs some sort of 'internal' build and validation check mechanism, like when you have a Visual Studio database project that can build and package as a DACPAC (i.e. that you could then use to deploy via DevOps and CI/CD). Especially if people are going to use Fabric deployment pipelines instead of DevOps CI/CD pipelines to deploy all artifacts...

[v-nikhilan-msft]

Hi @vivien57
Thanks for using Fabric Community.
Apologies for the issue you have been facing. I am not able to repro this issue at my end. As the issue is somewhat similar to the issue mentioned here: https://community.fabric.microsoft.com/t5/General-Discussion/Issue-Deployment-pipeline-and-folders-d...
I request you to wait for the support team to resolve this.

Thanks