Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Don't miss out! 2025 Microsoft Fabric Community Conference, March 31 - April 2, Las Vegas, Nevada. Use code MSCUST for a $150 discount. Prices go up February 11th. Register now.

Reply
DennesTorres
Impactful Individual
Impactful Individual

Can't find dataflow

Hi,

After creating a dataflow, I'm adding a dataflow activity to my pipeline, but for some reason the pipeline can't find the dataflow.

The image below illustrates the problem. What am I missing? The dataflow is in the workspace.

 

DennesTorres_0-1707394033476.png

 



Kind Regards,

Dennes

2 ACCEPTED SOLUTIONS

Hi @DennesTorres 
We haven’t heard from you on the last response and was just checking back to see if you can provide the details asked above. This helps us in understanding the issue and help you better.
Thanks

View solution in original post

Hi,

Your troubleshooting steps and some investigation with the support managed to solve the problem.

The solution: Dataflows are not visible to the Dataflow activity in pipelines before they are published.

Well, it seems to make sense, except for one detail: When the dataflow is "Published" it is Executed. Sometimes, like on the situation I have in hands, I had no intention to execute the dataflow outside the orchestration of the pipeline.

Luckly, it was possible to workaround and execute, but I keep imagine situations where it wouldn't make sense.

It seems a pending improvement.

 

Kind Regards,

 

Dennes

View solution in original post

11 REPLIES 11
avilasebastian
Regular Visitor

Hi, 

 

I had the same problem yesterday and still having it today, so waiting is not the solution. The dataflow has been published many times and locates in the same workspace as the pipeline, but during the creation of the element in the pipeline does not list me dataflows in any workspace.

 

 

avilasebastian_0-1732213112590.png

 

thank you !...

 

Sebastian A.

goflse
Frequent Visitor

I have the same issue in our Fabric workspace. A pipeline is being cretaed but I can't find any of the dataflows even if they have been published.
Any other things to look at? It's working for another customer though.

v-nikhilan-msft
Community Support
Community Support

Hi @DennesTorres 
Thanks for using Fabric Community.
The issue you’re encountering where the pipeline can’t find the dataflow might be due to a few reasons:

Dataflow Configuration: Ensure that you’ve correctly configured the dataflow activity in your pipeline.           Double-check that you’ve selected the correct dataflow from the dropdown list in the activity settings.

Workspace and Permissions: Make sure you’re working in the correct Microsoft Fabric-enabled workspace (not the default “My Workspace”). 

This might be an intermittent issue also. Please wait for sometime and retry. If the issue still persists please do let us know. Hope this helps.

Hi,

It's not during execution, the error happens when selecting the dataflow in the dropdown, so I can't select it.

Yes, the dataflow is in the same workspace as the pipeline.

I remember, from data factory, some name requirements on the dataflows, so I tried to rename it, removing spaces and leaving everything lowercase, but it didn't solve the problem. I still think it's some not so obvious requirement not fulfilled.

 

Kind Regards,

 

Dennes

Hi @DennesTorres 
Is it the same in all the workspaces? Or this is happening only in this workspace? Is the Dataflow published successfully? If so, how recently?
Try creating another dataflow and check. Is the Dataflow published recently?
Please let me know if the issue persists. 

Hi @DennesTorres 
We haven’t heard from you on the last response and was just checking back to see if you can provide the details asked above. This helps us in understanding the issue and help you better.
Thanks

Hello !!

 

so, microsoft made contact with me and we saw the issue in a call.

The solution for me was to enable the Gen1 dataflow in the admin portal. Microsoft team detected the error with the logs of the request while refreshing the dataflow list noticing an unauthorized statment. 1 minute after clicking the enable button the pipeline was able to list the dataflows.
Hope you can fix it the same way !

 

Sebastian A.

Hello .. thank you so much - problem solved.

Hello @v-nikhilan-msft 

this issue still persists and it's causing my firm some big issue during a BC Go-Live in combination with MS Fabric Power BI Reports.

The Dataflow is supposed to run every 5 minute to load BC Data into Fabric, but it's not visible in the drop-down list of the data pipeline's dataflow activity.

 

I have tried all the mentioned steps several times without luck:

  • Dataflow Gen2 is published
  • Workspace is fabric enabled (even 3 different workspaces)
  • creating new dataflow didn't solve the problem

I have used dataflows in pipelines across several projects and never faced anything like this.

Please let me know of any Tenant Level Setup necessary to enable this - this is supposed to be a core feature and besides the fact that Microsoft has caused us a huge amount of shame facing our customers by not enabling git integration and other basic data engineering features for dataflows, such occurences make customers second guess if MS Fabric was the right way to go.

 

Please fix these things before implementing any new fancy features to MS Fabric, it's making life hell for us.

 

Thank you for your understanding

Hi,

 

- I haven't tested in different workspaces.
- I haven't published the dataflow. Published=run . The execution will only be complete when done from the pipeline.
- There are 3 dataflows in the workspace. None of them appear in the pipeline, on the dropdown.

Kind Regards,

 

Dennes

Hi,

Your troubleshooting steps and some investigation with the support managed to solve the problem.

The solution: Dataflows are not visible to the Dataflow activity in pipelines before they are published.

Well, it seems to make sense, except for one detail: When the dataflow is "Published" it is Executed. Sometimes, like on the situation I have in hands, I had no intention to execute the dataflow outside the orchestration of the pipeline.

Luckly, it was possible to workaround and execute, but I keep imagine situations where it wouldn't make sense.

It seems a pending improvement.

 

Kind Regards,

 

Dennes

Helpful resources

Announcements
Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount! Prices go up Feb. 11th.

JanFabricDE_carousel

Fabric Monthly Update - January 2025

Explore the power of Python Notebooks in Fabric!

JanFabricDW_carousel

Fabric Monthly Update - January 2025

Unlock the latest Fabric Data Warehouse upgrades!

JanFabricDF_carousel

Fabric Monthly Update - January 2025

Take your data replication to the next level with Fabric's latest updates!