topic GetDataflowDatasourcesAsAdmin returning multiple connections for dataflow in Developer

GetDataflowDatasourcesAsAdmin returning multiple connections for dataflow

BobLang — Fri, 05 Jan 2024 21:03:43 GMT

I have 2 connections in my Gateway that are identical, except for the connection name.

I have a dataflow that uses one of those connections. When I open the dataflow and go to the Manage Connections screen and edit the connection, the connection name is one of the connections in the Gateway.

When I use the GetDataflowDatasourcesAsAdmin API for this dataflow, the results include both gateway connections. I was expecting to see just the connection that is linked to dataflow.

We have a bunch of duplicate connections in the Gateway that I want to eliminate. I was hoping the API would help me see which connections are being used for each dataflow.

Is there a better option to identify lineage from a gateway connection to the dataflows?

Thanks,

Re: GetDataflowDatasourcesAsAdmin returning multiple connections for dataflow

Anonymous — Fri, 12 Jan 2024 01:56:58 GMT

Hi @BobLang ,

Thanks for reaching out to us with your problem.The GetDataflowDatasourcesAsAdmin API is designed to return a list of data sources for the specified dataflow. This includes both cloud and on-premise data gateway sources. Base on your description, it seems that you're experiencing an issue where it's returning multiple connecitons for a single dataflow. Could you please provide the related screeshots with the connections(mask the sensitive info)?

In addition, you can refer the following official documentation to get the related info by the feature "Lineage View". In Lineage View, you can see the lineage relationships between all the artifacts in a workspace, and all its external dependencies . It shows connections between all workspace artifacts, including connections to dataflows, both upstream and downstream .

Data lineage - Power BI | Microsoft Learn

Best Regards

Re: GetDataflowDatasourcesAsAdmin returning multiple connections for dataflow

BobLang — Fri, 12 Jan 2024 16:37:40 GMT

My use case is that over time, my developers have created numerous duplicate connections in our gateway. I am going through an exercise to clean up the gateway. I am trying to find the linage from the Connections down to the dataflows.

When using the GetDataflowDatasourcesAsAdmin API, I the API is pulling all duplicated connections.

Here is an example: The gateway has 2 connections that are identical except for the name.

In the dataflow, the connection is showing that it is linked to the left connection that is illustrated in the screenshot above.

But, when I use the API to pull the datasources for this dataflow, the API is pulling both connections:

Using the Datasets GetDatasourcesAsAdmin API works great with the dataset connections. It pulls the actual connection used in the dataset.

The current lineage view in Power BI does not effectively meet my use case for 2 reasons:

1. I would have to go through 150+workspaces to look at that lineage view.

2. When you use the linage view, it does not shows the name of the gateway connection - only the connection details. As you can see in the screenshot below of the lineage for example, the connection string is displayed, not the name of the connection, So, I can't see which connection in the gateway is actually being used.

Hope these better illustrates the challenge that I am facing with GetDataflowDatasourcesAsAdmin api.

Thanks,