Forum Discussion
Dataflow Gen2 CI/CD Enabled – Refresh Fails for Non-Owner with Admin/Contributor Access
Hi team,
I created a Dataflow Gen2 in Microsoft Fabric with the CI/CD option enabled. When I trigger a refresh as the owner, it completes successfully. However, when a non-owner user with Admin or Contributor access tries to refresh it, the operation fails with the following error:
emails_table WriteToDataDestination: There was a problem refreshing the dataflow: 'Data source credentials are missing or invalid. Please update the connection credentials in settings, and try again.'. Error code: 999999. (Request ID: b2f44e29-a797-43c0-b080-b8272cb85ce4)
But the non-owner was able to successfully refresh the same dfgen2 without ci/cd enabled.
Is this a known issue with CI/CD-enabled Dataflow Gen2? Are there any workarounds or permission settings we should adjust?
Thanks in advance!
Hello Bhargava05,
When CI/CD (Git integration) is enabled in a Fabric workspace:
The Dataflow Gen2 definition is versioned and stored as JSON in the Git repository.
Connection credentials, however, are not stored in Git (for security reasons).
The credentials are instead stored per user in the Fabric service layer.
👉 This means only the user who originally authenticated the data source (the dataflow owner) has valid credentials linked to that Git-bound dataflow instance.
Solutions :
Option 1 – Re-authenticate the data source as the non-owner
Have the non-owner open the Dataflow Gen2 in Edit mode.
Go to Settings → Connections.
Select the data source → click Edit credentials.
Enter their own credentials and save.
🔹 Drawback: each admin/contributor must do this individually (credentials are not shared).
**Option 2 – Use a Service Principal (recommended)
If your source supports Azure AD App/Service Principal authentication (SQL, Lakehouse, OneLake, etc.):
Create a service principal in Entra ID.
Grant it access to the data source.
In the Dataflow connection → choose “Sign in with Service Principal”.
✅ Result: credentials are now tenant-level, not user-level, so any workspace admin can trigger refreshes successfully.
Hope it can help you !
Best regards,
Antoine
6 Replies
Hello Bhargava05,
When CI/CD (Git integration) is enabled in a Fabric workspace:
The Dataflow Gen2 definition is versioned and stored as JSON in the Git repository.
Connection credentials, however, are not stored in Git (for security reasons).
The credentials are instead stored per user in the Fabric service layer.
👉 This means only the user who originally authenticated the data source (the dataflow owner) has valid credentials linked to that Git-bound dataflow instance.
Solutions :
Option 1 – Re-authenticate the data source as the non-owner
Have the non-owner open the Dataflow Gen2 in Edit mode.
Go to Settings → Connections.
Select the data source → click Edit credentials.
Enter their own credentials and save.
🔹 Drawback: each admin/contributor must do this individually (credentials are not shared).
**Option 2 – Use a Service Principal (recommended)
If your source supports Azure AD App/Service Principal authentication (SQL, Lakehouse, OneLake, etc.):
Create a service principal in Entra ID.
Grant it access to the data source.
In the Dataflow connection → choose “Sign in with Service Principal”.
✅ Result: credentials are now tenant-level, not user-level, so any workspace admin can trigger refreshes successfully.
Hope it can help you !
Best regards,
Antoine
I can see why this is an accepted solution. But it's not working in my tennant. When configuring a lakehouse connection in a Dataflow Gen2 (CI/CD) (same workspace) , the only option to choose from is "organizational account".
I have a service principal with access to the lakehouse. The service principal is used (and working) in semantic models connecting to the same lakehouse.
When trying to make a new connection under Manage Connections and Gateways > New > Cloud > connection type: Lakehouse I only have the option "OAuth 2.0" available.Help or suggestions are welcome!
Hi arjaano ,
Thank you for reaching out to the Microsoft Community Forum.
As mentioned in your previous response, while configuring a lakehouse connection in a Dataflow Gen2 (CI/CD) (same workspace) , the only option to choose from is "organizational account". Thank you for your information, it will help other community members.
Set up your Lakehouse connection - Microsoft Fabric | Microsoft Learn
Regards,
Dinesh
Hi Bhargava05 ,
Thank you for reaching out to the Microsoft Community Forum.
Hi AntoineW , Thank you for your prompt response.
Hi Bhargava05 , Could you please try the proposed solution shared by AntoineW ? In additon to that , I have added the Microsoft community thread. Please refer below link.
Solved: Re: Dataflow Gen2 Fails to refresh - '_WriteToData... - Microsoft Fabric Community
I hope this information helps. Please do let us know if you have any further queries.
Regards,
Dinesh
Hi Bhargava05 ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. And, if you have any further query do let us know.
Regards,
Dinesh
Hi @Bhargava05 ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. And, if you have any further query do let us know.
Regards,
Dinesh
