Visibility into Source Table Metadata Currently, there is no available option to view metadata or properties (such as data source name, original location, or table lineage) for a Shortcut table. Providing this level of visibility is critical for data governance, auditing, and root cause analysis. Especially in environments where multiple shortcuts and data sources are interconnected. This information would be invaluable for both developers and administrators managing enterprise data at scale.   Improved Stability and Reliability of Shortcut Integration We've observed some recurring instability or performance issues when using Shortcuts in production environments. Enhancing the underlying reliability and offering better diagnostics or logging would go a long way in increasing user confidence and adoption of this feature. ... View more

There should be a secure way to pass secrets or keys when making an API call (e.g. web.contents) in Power Query M.   Suggestions:   A) Make it possible to enter and edit secrets in Power Query, similar to how we enter and edit data source credentials in Power Query. Make these secrets encrypted and redacted. The secrets should be available to use as variables in the M code. When entering a secret, we would give it a name, which we can reference as a variable in the M code. We would only see the name, but the value would not be visible (the value would be redacted).   B) Make an Azure Key Vault connector for Power Query, that will make it possible to fetch secrets from AKV and assign the secret to a variable in the M query. This variable can be passed as part of the header in an API call.   C) Make it possible to redact some variables in M, so that we can use secret variables without the value being visible in the Power Query user interface.    The main point is that we want a way to use secured and hidden secrets as part of API calls in Power Query. ... View more

its well known that intellisense has always been broken with things like ListList.Contains or pressing . after typing certain prefixes does not correctly show options, or typing null enter puts null.type in. Also a huge issue is no intellisense in Dataflows both need fixing ... View more

A table is a table. It doesn't matter if it's a New table or an Existing table.   I don't understand why we need to toggle between New and Existing table in data destination.   We should simply be able to choose an existing table or insert a new table name. Easy as that. They should be treated the same. No need to toggle between New and Existing mode.   If we write to an Existing table, we should be able to add new columns directly in the Dataflow and this should be written to the selected destination table.   If we choose Automatic mode, this should be possible also for Existing tables. Same as for New tables. It's just a table.   Doesn't matter if it's a New table or an Existing table. They should be treated the same. ... View more

Basically, we should be able to edit the final output (data destination) columns directly in the Dataflow Gen2, without needing to go to the Warehouse itself to add the new columns (using T-SQL).   This should be possible to do in the Dataflow user interface both for the Append and Replace options.   The settings in the dataflow outputs should be applied to the destination (Lakehouse, Warehouse) without needing to go into the Warehouse or Lakehouse to add the columns.   This should work regardless of writing to a New or Existing table.    This should just work.   We should be able to choose which columns to write to the destination, and add more columns to an existing table, directly in the Dataflow user interface, without having to go to the Lakehouse or Warehouse to make any changes. ... View more

I would like to request a feature enhancement for Microsoft Fabric Dataflows to support multiple Date and Date/Time formats, similar to what is currently available in Power BI Desktop.   Power BI Desktop datetime format options   In Power BI Desktop, we can easily parse and interpret various date and datetime formats within a column. However, in Fabric Dataflows, the formatting options are limited and inflexible.   It would be extremely helpful if Dataflows could allow flexible configuration of multiple formats for date and date/time columns. This feature would greatly improve usability, especially when handling data from diverse sources where date formats are inconsistent. ... View more

Orchestrating a refresh of a Dataflow Gen2 with CI/CD and Git support isn't possible in Fabric data pipelines. We request you to bring this feature in Fabric so that users can integrate these dataflow gen 2 with CI/CD and Git integration in data pipeline. ... View more

When refreshing a Dataflow Gen2 via Data Pipeline or API, make it possible to select which table(s) to refresh.   Such an option will provide great flexibility.   Some tables need to be refreshed more often than others.   And sometimes we wish to cherry-pick a single table to refresh only that table. ... View more

Please make it possible to open a dataflow gen2 without taking it over. I wish we could open a dataflow gen2 even if it is "owned" by someone else in the workspace. I think that the user who opens the dataflow gen2 should need to apply their own - or shared - data source connections in order to see any data in the query steps. Otherwise, they would be able to use the owner's connections to interact with the data sources, and I don't think that's a good idea. Because the owner might not want other workspace members to use his/her connections. So I think that the user who opens the dataflow gen2 should need to apply their own - or shared - data source connections in order to see any data in the query steps. I think it should be possible to open the item and start editing it, and not having to formally "take over" until you are ready to publish the changes. Or some times you just want to open it for inspection without making any changes. And if you do start to edit the dataflow, but then realize you aren't able to authenticate to all the data sources, then it should be possible to cancel so that the dataflow remains unchanged and the previous connections keep working. Would be nice to have: before a user saves (publishes) changes to the dataflow, there could be a validation step to check that the user has authenticated properly to all of the item's data sources and data destinations, so that they won't cause the dataflow to stop working. If the validation fails, the dataflow should keep running with the previous connections and the changes should be undone. ... View more

Is it possible to create a utility dataflow to define all reusable functions and reference that utility in any dataflow that requires them? Similar to how scripting works in JavaScript or Python.   Currently, the only alternative is to rewrite the same function separately in each dataflow, since there is no way to define it once and reuse it across dataflows. This makes the process extremely cumbersome and inefficient, as the functions have to be manually duplicated every time.     ... View more

Enable to modify the mashup document of a Dataflow Gen2 by passing a variable from a pipeline before an execution of the Dataflow ... View more

For low code users, it would be awesome if there was a Data Pipeline activity that can be used to run Optimize on a table in a Lakehouse (or all tables in a Lakehouse).   For example, when using Copy Activity or Dataflow Gen2 to append data to a Lakehouse table, the tables need to get optimized (compacted) after x runs. But there is no automated, low-code way to do it. So users forget to optimize the table in the destination Lakehouse. ... View more

If you create a Gen2 dataflow and set the data destination to "DWH" you can create a table in the DWH. If you later modify the dataflow to add a column, this column doesn't appear in the DWH. When you publish the report or when you run the dataflow THERE IS NO INDICATION OF A PROBLEM - everything looks good.   Please provide some indication that there is an issue so that the developer can fix it. ... View more

Deployment pipelines should allow for rules in the different environment/stages for DataflowGen2 CICD items. Ideally specifying differenent (paramaterized) sources and destinations in each stage, for each DataflowGen2 CICD item. ... View more

• Data Pipelines in Microsoft Fabric are built on Azure Data Factory. When a pipeline writes data to OneLake or other destinations, it leverages ADF's managed infrastructure and integration runtime, which handles connectivity differently and does not rely on the same DNS resolution path as DF.  • Dataflows Gen 2, are built on Power Query and run using the on-premises data gateway infrastructure, it executes the queries from the environment where the gateway is installed. The network configuration and DNS resolution of the gateway host (in this case, your GCP VM) directly impacts the ability to connect to external services like SQL Server." Currently, For Dataflow DNS team need to open direct conenction (*.datawarehouse.pbidedicated.windows.net, *.datawarehouse.fabric.microsoft.com) via DNS exception. Large companies does not allow to open this connection instead they ask Product to work with proxy setting on server to open.  Requesting Power BI team to have similar setting like Datapipeline to Dataflow. ... View more

I would love a tool that converts Dataflows to Spark Notebook. ... View more

Enable an experience that allows the end-user to setup incremental refresh for Dataflows Gen2 in Data Factory for Microsoft Fabric ... View more

Dataflows are supported by the Deployment pipelines, but there is no Git integration for dataflows. With the introduction of Git integration of workspaces, it would be fantastic if Dataflows could be stored and traced and reviewed in Git same like it is possible now for datasets and reports. ... View more

Problem: The auto-save feature in Gen 2 Dataflows automatically saves changes as you develop the dataflow. While generally helpful, this can be problematic in certain situations: Experimental Changes: When making significant changes or experimenting with different transformations, auto-save can create unwanted versions or interfere with the development process. Accidental Errors: If you accidentally introduce an error or break the dataflow, auto-save might save the broken state before you can revert the changes. Version Control: Auto-save can make it more difficult to manage versions of your dataflow, especially if you're using Git integration. Proposed Solution: The feature request is for Microsoft to add a setting (e.g., a checkbox or toggle in the Dataflow editor) that allows users to enable or disable auto-save. When auto-save is disabled, changes would only be saved when the user explicitly clicks a "Save" button. Benefits of Implementing this Feature: Improved Control: Developers would have more control over when their changes are saved. Reduced Risk of Errors: It would be easier to experiment with changes without the risk of auto-saving a broken state. Better Version Control: It would simplify version management, especially when using Git integration. ... View more