The pipeline metadata activity currently only allows finding metadata on Lakehouse Tables and Files, and Warehouse Tables. It would be great to add the ability to retrieve names and GUIDs for all Fabric artifacts it can source.
For an example of how this could help - I have 24+ Gen2 dataflows that I use to load data in parallel. Currently I have to make a pipeline by adding 24 individual dataflow activities and selecting which dataflow to run for each activity.
If the Get Metadata activity would allow me to return dataflow IDs, I could automate this with a single get metadata activity, followed by a ForEach acvitity with embedded dataflow.
Get Metadata should be able to find:
Workspace Names and IDs
Dataflow Names and IDs
Nootbook Names and IDs
Dataset Names and IDs (for when the eventual "trigger dataset refresh" becomes available)
etc.
Basically, anything that is a valid Pipeline source or target should be discoverable through Get Metadata.
Thanks,
Scott
1 Comment
- fbcideas_migusrNew MemberStatus added:New
Recent ideas
Allow us to rename fabric data agents published to m365
If we use deployment pipelines to promote fabric data agents between dev, test/UAT, and prod fabric workspaces, we need to keep the name of the fabric agent the same in each workspace. If we want to ...PeterDaniels3 hours agoAdvocate IIINew179Views0likes2CommentsMake workspace and item session persistence optional
Description The new persistent session behavior in Microsoft Fabric should be optional rather than forced. Currently, Fabric remembers the workspaces and items that were open in my previous session...TeemuMultanen4 hours agoAdvocate INew293Views47likes2CommentsSupport Encrypted Sensitivity-Labeled Excel Files in Power Query
Description Currently, Power Query Online and Power Query in Excel are unable to access encrypted Excel files. Excel files with sensitivity types other than Public or Non-Business can be encrypted a...ewarstdhyjugkhi4 hours agoMicrosoft EmployeeNew19Views6likes0Comments