data factory | dataflow
307 Topics📉 How Would You Investigate a 20% Drop in Revenue?
A data analyst shouldn’t stop at identifying what happened. The real questions are: Where did the decline occur? → Why did it happen? → What caused it? → What should we do next? By analyzing regions, products, customers, time periods, and key business metrics, raw data can reveal the root cause behind the numbers. Good data analysis turns numbers into decisions. 📊 #DataAnalysis #BusinessIntelligence #PowerBI #MicrosoftFabric #DataAnalytics6Views0likes0CommentsAllow CDC and Watermark Tables in the Same Copy Job Without Falling Back to Watermark
Enterprise databases commonly contain a mix of tables. Some tables support change data capture, while others need watermark-based incremental processing. When CDC-enabled and non-CDC-enabled tables are included in the same Copy Job, each table should be able to retain the change detection method best suited for that table. Please allow replication mode to be configured independently for each table in a Copy Job. CDC-enabled tables should continue using CDC, while other tables can use a configured watermark column or full copy. Each table should maintain its own checkpoint, reset state, monitoring information, and error status. This would avoid creating multiple Copy Jobs for the same source simply because different tables have different change-tracking capabilities. It would also make Copy Job easier to operate for large enterprise source systems containing hundreds of tables.3Views0likes0CommentsFabric Copy Jobs desperately need basic editability after creation.
Requiring users to recreate an entire Copy Job just to change something as fundamental as the source connection or connection URL is unnecessarily restrictive and creates significant maintenance overhead. A Copy Job should allow users to: Change the source connection Change the destination connection Edit connection details Repoint a job to a replacement connection Preserve existing mappings, scheduling, and other configuration when changing connections Having to rebuild a working job because one connection property needs to change makes Copy Jobs much less useful for production data workflows. Please add the ability to edit or replace connections on an existing Copy Job.10Views0likes0CommentsAdd Search Functionality to Power Query "Queries" Pane
See Attached Image. Quick Overview When working on large Power BI models, it’s easy to end up with 50, 100, or even more queries in Power Query. These come from all kinds of sources and transformation steps, and the list gets long fast. At that point, just finding the query you’re looking for becomes a real hassle. Right now, the only option is to scroll through the list manually—and in big projects, that’s just not practical. Why This Would Help Models grow quickly: staging queries, reference layers, intermediate logic—all stack up fast. Scrolling through a huge list every time to find a query breaks flow and wastes time. A simple search/filter option in Queries Pane would make it way easier to work efficiently, especially under deadlines. The Ask Would love to see a search bar added at the top of the Queries pane in Power Query. Just something lightweight to help quickly jump to a query by name. That’s it—nothing to do with loading data or transformations, purely about improving authoring experience. I've included a screenshot as well to help visualize the idea.2.2KViews143likes38Comments
Portable Library of Queries (Functions)
Currently, a custom connector requires a Data Source and a Authentication record associated to a function in order to be Published or exposed in the 'Get Data' dialog/window. This is perfectly fine for data sources, but if I want to create a custom connector that creates data using native M functions, like. List.Generate or List.Dates, then I shouldn't have to define a Data Source nor an Authentication which complicates things and is an unnecessary step for the end user. Here are 2 videos showcasing 2 functions that do not need any sort of authentication, but due to the way custom connectors currently need to be constructed, they get prompted for credentials as an "Anonymous" user: https://youtu.be/76W1Cu1qSfw https://youtu.be/Q06NKl8A4eI The idea: to remove the need for a DataSource/Authentication record for custom connectors that act as library of functions so they can be published on the 'Get Data' dialog and not only shared as environment variables.1.7KViews147likes15CommentsAllow me to resize Power Query step configuration dialogs
Feedback initially received via: Share your thoughts on the New Power Query experience in Power BI Desktop (Preview) | Microsoft Fabric Community Ask: allow the dialogs that allow me to reconfigure a step to be resizable and provide a better user experience to read fields with high density of content like Native Query fields51Views1like1CommentPower Query Schema View - width of column "Name"
While working with the Schema view in Dataflow, the width of column "Name", which lists the name of the columns in the query, is fixed. Due to this it only shows initial about 19 characters of the column name. When there are multiple columns with commone first ~20 characters, it becomes difficult to know the right column and the only way to see full column name is to hover mouse pointer over the row with the column name. Due to this the utility of the schema view is greatly reduced, as a simple task of correcting the Type takes a long time. One needs to hover over each column to see the full name and accordingly change the Type. Please make the width of the column "Name" in Schema view adjustable, or make it enough to show at least 40 characters of the column name. This will enable users to quickly change the Type of required columns.399Views4likes3CommentsBuilt a free tool to inventory & risk-rate legacy SSIS packages before a Fabric migration
Hi all, Sharing something I built that might save some of you the painful "open every .dtsx in SSDT one by one" phase of an SSIS → Fabric migration. The problem it addresses: most teams sitting on a legacy SSIS estate have dozens to hundreds of undocumented packages and no clear starting point. Before any actual migration work, someone has to answer: what does each package do, which components map cleanly to Fabric, and which need a full rewrite? What it does: Parses .dtsx package XML and extracts control flow tasks, data flow components, connection managers, variables, and precedence logic Maps each component against a Fabric equivalent — Data Flow Task → Dataflow Gen2, Execute SQL Task → Script/Stored Procedure Activity, Script Task → Notebook Activity, Foreach Loop → ForEach Activity, and so on Gives each package a Low/Medium/High risk rating based on concrete signals (Script Task usage, Fuzzy Lookup, deprecated CDC/Attunity components, nesting depth, dynamic connection strings) Flags every component individually as auto-mappable or needs-manual-redesign, not just the package as a whole Across a batch of packages, produces a portfolio rollup: risk tier counts, the blockers showing up across the most packages, deprecated components flagged separately, and a suggested migration order What it's built as: a Claude Skill (Anthropic's Claude AI), with the actual XML parsing done by a plain Python script with zero dependencies — not an LLM guessing at package structure. It's designed to run fully offline, no live Azure/Fabric API access assumed, though it flags anywhere that access would improve accuracy (resolving a dynamic connection string, for example). What it deliberately doesn't do yet: this covers the Inventory/Classify phase only. It won't draft actual Fabric pipeline JSON or claim a package is production-ready — every future draft artifact gets an explicit DRAFT/NEEDS REVIEW label. I'd rather it under-claim than over-claim. Repo's on GitHub with a sample package and example report included, so you can see the actual output before running it against your own packages: https://github.com/HBBH11/ssis-fabric-migration-assistant Genuinely interested in feedback from people who've done these migrations for real — particularly whether the risk heuristics hold up against messier packages than my test case, and whether the Fabric equivalence mappings match what you've found in practice. Happy to take suggestions on what a Phase 2 (draft pipeline generation) should prioritize too.16Views0likes0CommentsFabric REST API endpoint for dataflow transactions should GET *sortable* results for a time-range
Dataflow refresh APIs seem to be the buggiest; the list of transactions on a GET for Fabric-dataflow, are not sorted by the latest. OData query options such as $top, $filter, $orderby do not work with this specific endpoint. Endpoint accepts the parameter but ignores it. Endpoint implements its own pagination mechanism. If GET ../transactions returns 10 rows, I can't assume it will have the latest refresh in those 10. This has been confirmed by running at different times of the day.8Views0likes0CommentsFeature Request: Search functionality in Power Query Queries pane
Power Query becomes increasingly difficult to navigate as the number of queries grows. In real-world Power BI projects, it is common to have 50, 100, or even 200+ queries across multiple folders or groups. Currently, locating a specific query often requires manually scrolling through the Queries pane. This becomes time-consuming, especially when query names are similar or when working with large enterprise Power BI models. I would like to suggest adding a Search box at the top of the Queries pane. For example, if I search for "Purchase", Power Query could immediately filter and display queries such as: Purchase Report Purchase Report Oct 25 - Mar 26 RT Pending Purchase Orders Purchase Summary This would make it much easier to locate queries without manually scrolling through a large list. It would be even more useful if the search supported partial query names and worked across query groups or folders. An optional keyboard shortcut such as Ctrl + F could also be provided to quickly focus the query search box. This may look like a small UI enhancement, but it would provide a significant productivity improvement for Power BI developers and data analysts working with large Power Query projects. Thank you for considering this suggestion.27Views1like0Comments