data factory | dataflow
296 TopicsSupport for Ingesting Purview Encrypted files
We would like to use Dataflow Gen2 to ingest files that have been encrypted by Microsoft Purview Information Protection controls, i.e. sensitivity labels. Currently, Power Query Online (Dataflow) and Power Query in Excel are unable to access encrypted Excel files. Since Excel files labeled with sensitivity types other than "Public" or "Non-Business" are encrypted, they aren't accessible through Power Query Online (Dataflow). This appears to be a current product limitation related to the underlying Power Query experience used by Dataflow Gen2. Encrypted files protected through Microsoft Purview Information Protection are not currently supported as source files for Dataflow ingestion scenarios. As described by an MS engineer, "Where encrypted workbooks could not be processed by Dataflows, removing the encryption resolved the issue". If possible, we would like this to be mechanism to be developed.54Views1like1CommentAdd Search Functionality to Power Query "Queries" Pane
See Attached Image. Quick Overview When working on large Power BI models, it’s easy to end up with 50, 100, or even more queries in Power Query. These come from all kinds of sources and transformation steps, and the list gets long fast. At that point, just finding the query you’re looking for becomes a real hassle. Right now, the only option is to scroll through the list manually—and in big projects, that’s just not practical. Why This Would Help Models grow quickly: staging queries, reference layers, intermediate logic—all stack up fast. Scrolling through a huge list every time to find a query breaks flow and wastes time. A simple search/filter option in Queries Pane would make it way easier to work efficiently, especially under deadlines. The Ask Would love to see a search bar added at the top of the Queries pane in Power Query. Just something lightweight to help quickly jump to a query by name. That’s it—nothing to do with loading data or transformations, purely about improving authoring experience. I've included a screenshot as well to help visualize the idea.2.1KViews141likes37CommentsAdd Better Error Details and Preview in Data Factory Data Flow
Please improve Data Flow by showing clear error messages with step-by-step suggestions. Add a faster data preview, automatic validation, and better performance insights before running pipelines. These improvements will help users find problems quickly, reduce development time, and make Data Factory easier for beginners and experienced users alike.13Views0likes0CommentsFabric > Dataflow Gen 2 > Destination: Allow this to be a dataflow parameter like table name can be
I recently disovered parameters in dataflow which was a big unlock in organization/efficiency, but not being able to specify the detination dynamically (outside of very end table name) nullifies that features' usefulness by having to make at least one per lakehouse because of this. Please update the Destination UI to allow a Parameter to be selected for the lakehouse (including folder/schema/dbo) to be populated by a dataflow parameter.172Views2likes4CommentsNotify Subscription Owners About Invalid Email Recipients in Power BI
Power BI should automatically notify subscription owners whenever one or more recipient email addresses are invalid or undeliverable. Currently, report subscriptions may silently fail for specific recipients without providing clear feedback to the person who created and manages the subscription. A notification should identify which email addresses could not receive the subscription and explain the reason, such as an invalid address, deleted user account, or external recipient restriction. This would allow subscription owners to quickly update the recipient list instead of assuming that reports are being delivered successfully. Providing proactive notifications for failed deliveries would improve reliability, reduce support requests, and ensure that important reports reach their intended audience. It would also enhance the overall user experience by making subscription management more transparent and easier to maintain.15Views0likes0CommentsShow exact row-level error details and failed data preview in Fabric Dataflow Gen2 & Pipeline failur
Right now when a Fabric Pipeline or Dataflow Gen2 fails, we only get generic error messages like: "Data conversion error", "Column not found", "Expression.Error" To find the root cause, we have to manually check lakhs of rows. This wastes 1-3 hours every time for every Fabric developer. What we need: Please add detailed error diagnostics that include: 1. Exact location: Pipeline Activity Name > Dataflow Name > Step Name > Column Name > Row Number 2. Failed value: What was the actual bad value that caused the error 3. Sample data preview: Show top 10 failed rows directly in the error details pane, with option to download as CSV 4. Suggested fix: AI suggestion like "Column 'Amount' has text 'N/A'. Consider replacing with 0 or null" Why this matters: This is the #1 time-wasting issue in Fabric today. In ADF and SSIS we had better debugging. For Fabric to be adopted by enterprises, debugging must be 10x faster. This will save thousands of hours across all Fabric users every week.27Views0likes0CommentsDataflows as code
Please let me just define a Dataflow as code? The UX is terrible and cumbersome. If I need a simple Dataflow sure. But even then I'd rather submit a YAML file. That way I could do real diffs, have proper pre and post commit hooks and testing and linting. Anything that you can do through the UI, should be doable as a code first experience.18Views0likes0CommentsOpen Dataflow Gen2 without take over
Please make it possible to open a dataflow gen2 without taking it over. I wish we could open a dataflow gen2 even if it is "owned" by someone else in the workspace. I think that the user who opens the dataflow gen2 should need to apply their own - or shared - data source connections in order to see any data in the query steps. Otherwise, they would be able to use the owner's connections to interact with the data sources, and I don't think that's a good idea. Because the owner might not want other workspace members to use his/her connections. So I think that the user who opens the dataflow gen2 should need to apply their own - or shared - data source connections in order to see any data in the query steps. I think it should be possible to open the item and start editing it, and not having to formally "take over" until you are ready to publish the changes. Or some times you just want to open it for inspection without making any changes. And if you do start to edit the dataflow, but then realize you aren't able to authenticate to all the data sources, then it should be possible to cancel so that the dataflow remains unchanged and the previous connections keep working. Would be nice to have: before a user saves (publishes) changes to the dataflow, there could be a validation step to check that the user has authenticated properly to all of the item's data sources and data destinations, so that they won't cause the dataflow to stop working. If the validation fails, the dataflow should keep running with the previous connections and the changes should be undone.5.6KViews98likes11CommentsDataflow Gen2: auto mapping doesn't re-evaluate parameterized destination table columns at runtime
Using a parameter for the destination table appears supported, but it isn't — unless the destination table's columns are identical to what was set during the preview at publish time. The source query re-evaluates per run, but the auto destination mapping stays locked to the published preview schema. So if the parameterized table has different columns, the refresh fails: The column '<name>' wasn't found. Error code: 104100 / EntityUserFailure This blocks using one parameterized dataflow for multiple tables with different schemas. Request: Re-evaluate the destination table columns at runtime so parameterized destination tables work when columns differ.22Views0likes0Comments