Dynamic Batch Count in Data Factory Pipelines (FOREACH ACTIVITY)
Currently, in Data Factory, the ForEach activity requires a static batch count when executing in parallel mode. However, this fixed value does not always align with dynamic workloads, leading to inefficiencies. A more flexible approach would be to allow the batch count to be dynamic, adjusting automatically based on the number of items in the loop or available system resources.
For example, instead of manually setting a batch count of 10, users could define a dynamic value based on the total number of items, ensuring optimal parallel execution without overloading the system. This could be achieved by allowing expressions, pipeline parameters, or metadata-driven configurations to determine batch size at runtime. lets get this up-voted!!
1 Comment
- JannickVanGielRegular Visitor
Agreed. This is a must-have feature, which was already available in Azure Data Factory.
Recent ideas
Justification Prompts When Downgrading or Removing Sensitivity Labels in Power BI Desktop
Description Currently, organizations can configure Microsoft Purview sensitivity label policies to require users to provide a justification when removing a label or lowering its classification. T...vvargasv2 hours agoMicrosoft EmployeeNew3Views0likes0CommentsMatrix: column totals as a bar chart docked to the edge
What's missing Data bars already work on the Total row and column through conditional formatting, so row totals can be read as length. Column totals cannot: they render as a horizontal bar inside a ...Sayurivalente4 hours agoRegular VisitorNew77Views6likes1CommentAllow Bookmarks to Affect Multiple Report Pages
Description: I would like to request the ability for a single Power BI bookmark to affect and maintain a state across multiple report pages, rather than being limited to a single page. For example,...Josegonzalez214 hours agoNew MemberNew18Views4likes1CommentFeature Proposal: Expandable/Collapsible Page Sections for Reports on Same Semantic Model
Subject: Feature Proposal: Expandable/Collapsible Page Sections for Reports Using the Same Semantic Model Dear Microsoft Power BI Team, I would like to propose an enhancement to the Power BI report...Dr7awb4 hours agoNew MemberNew29Views1like1Comment