Dataflow - Databricks - Deployment Pipeline

HI,johnpjustus 

1.When you deploy a data flow from a development environment to a user acceptance test environment or production environment, the data flow cannot be refreshed. We think this may be related to your authentication, as certain authentication methods and connection details may not be automatically transmitted or persist in different environments without re-establishment.

 

2.For Azure Databricks, the types of authentication supported by Dataflow Gen2 include, but are not limited to, username/password, personal access token, and Microsoft Entra ID.Given this, the behavior you encounter is somewhat expected due to the nature of secure connections and authentication tokens or credentials, which are environment-specific and may require re-authentication when moving between different workspaces or environments.

The following is the relevant documentation, I hope to help you:

Authentication for Azure Databricks automation - overview - Azure Databricks | Microsoft Learn

Dataflow Gen2 refresh - Microsoft Fabric | Microsoft Learn

Microsoft Entra Connect: Pass-through Authentication - Microsoft Entra ID | Microsoft Learn

3. When the data is being recreated, you will also be asked to configure a refresh to ensure that the refresh does not fail to access the data due to authentication.The official documentation mentions that "whenever a data stream is created, you are prompted to refresh the data for the data stream. The data flow needs to be refreshed before it can be used in the semantic model of Power BI Desktop or referenced as a linked table or computation table."

Here is a link to the relevant documentation:

Configure and consume a dataflow - Power BI | Microsoft Learn

 

Best Regards,

Leroy Lu

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.