Fabric customers rely on a shared metadata contract across ingestion, modeling, analytics, and AI. Descriptions for tables and fields exist across multiple Fabric engines, including Delta, SQL Warehouse, Semantic Models, and KQL, but today these descriptions are typically defined after ingestion and do not consistently propagate downstream.   This idea proposes enabling table and column descriptions to be authored in Dataflows during the ELT process and automatically propagated into Delta table and column comments, Fabric SQL and Warehouse metadata, Semantic Model field descriptions, and the OneLake Catalog. By setting descriptions at ingestion time, Fabric can establish a single authoritative metadata foundation that travels with the data across all Fabric experiences.   This ensures consistent discovery, understanding, and trustworthy AI-assisted analytics, while creating a durable metadata layer that benefits governance, self-service BI, Copilot experiences, and AI workflows without requiring duplication or post hoc documentation.   Delta and OneLake (Foundational Layer) Delta supports native table and column comments, making it a natural system of record for metadata.   Example concepts: Table-level comments describing dataset purpose and scope Column-level comments describing business meaning and usage Ask in this idea: Allow Dataflows to set Delta table and column comments during write operations so metadata is stored with the Delta files. --- Fabric SQL Engine (Lakehouse SQL Endpoint) Fabric SQL can read metadata from Delta and expose it through SQL metadata views and extended properties. Supported concepts Table descriptions Column descriptions These descriptions are visible in SQL tooling and data exploration experiences and can be surfaced to Copilot. Ask in this idea: Inherit Delta comments authored by Dataflows and prevent metadata loss unless explicitly edited.   Fabric Data Warehouse Fabric Data Warehouse supports first-class metadata similar to SQL Server and Synapse. Supported concepts Table descriptions Column descriptions Schema descriptions Ask in this idea: When Dataflows write to Warehouse-backed tables, descriptions should flow automatically and not require re-authoring.   Semantic Models (Power BI) Semantic Models rely heavily on field descriptions for usability and Copilot effectiveness. Table descriptions Column descriptions Measure descriptions These descriptions are shown in field wells and are critical for self-service users. Ask in this idea: Auto-populate and keep Semantic Model descriptions in sync with OneLake metadata unless intentionally overridden.   KQL Database in Fabric KQL supports documentation at both the table and column level. Table-level documentation Column-level documentation This metadata improves query autocomplete, exploration experiences, and AI-assisted querying. Ask in this idea: When Dataflows land data into Eventhouse or KQL databases, apply descriptions automatically and treat them as part of schema evolution.   OneLake Catalog OneLake Catalog should treat table and column descriptions as first-class metadata. Surface descriptions consistently across Lakehouse Surface descriptions across SQL and Warehouse Surface descriptions across Semantic Models Surface descriptions across KQL Key ask: Dataflow-authored descriptions should flow into OneLake once and become the canonical metadata source that downstream tools consume. ... View more

We need support for Service Principal authentication when creating Lakehouse connections.   Right now, Lakehouse connections can only be created using a user account. As a result, Dataflow Gen2, Pipeline copy activity, and Copy Job cannot use a Service Principal (SPN) when connecting to a Fabric Lakehouse as a source or destination.   This is an issue for us. We want to avoid user-based connections because: We need environment-specific identities (dev/test/prod) to prevent accidental cross-environment writes (e.g., dev pushing data to prod). Tying connections to personal user accounts creates fragility - things break when someone leaves the project or the organization. Service Principal support for Lakehouse connections is required to ensure secure, reliable, and environment-isolated authentication.   The screenshot shows that only user authentication is available, not service principal authentication. ... View more

Hi, I generally use the mirroring feature to get data from an on-premise supported source at bronze level of the medallion architecture. In order to perform some data transformations and put the related data in a next lakehouse or warehouse I could use a dataflow gen2 that doesn't allow to select a mirrored database as a source. Moreover, also inside a pipeline it isn't possible to select a mirrored database as a source. I think that using a mirrored database as a source could be a good thing to have! Many thanks ... View more

its well known that intellisense has always been broken with things like ListList.Contains or pressing . after typing certain prefixes does not correctly show options, or typing null enter puts null.type in. Also a huge issue is no intellisense in Dataflows both need fixing ... View more

Hello,   We use Dataflows Gen 2 to build our data transformations, and use Snowflake as our database. We run all our transformations on one single Snowflake database and try to utilize query folding as much as possible to minimize data movement. Inherently though, query folding only supports source side of the operations, and Dataflows brings the transformed, final set of data from source to its memory, only to write it back to the very same destination.   If Dataflows can distinguish it is running over a single data store/database for a set of operations, it can convert all logic to SQL (or data store's native language) and let actual execution completely happen on the database itself, preventing an unnecessary dat journey to its memory and back to the database itself.   In the past, when cloud was not a thing, we used Informatica for complete query folding (they call it push query) , and that improved ETL performance drastically.   Query folding is a great feature, and if it can cover complete DML execution on the database itself, that can bring the performance to next level.   Thank you. ... View more

When I modify the data desination table name in a Git repository and pull the changes into Fabric, running the dataflow from Fabric results in the old table name being used. It's only possible to change the desination table name through the Fabric UI, not the Git code.  ... View more

When Power BI executes queries against SQL Server, it connects using the Mashup Engine or Mashup Engine (%HostContext%)) as program_name. In a crowded environment, it can be difficult to identify which semantic model is responsible for activity or issues on the SQL Server side. The desired outcome is to be able to run sp_whoisactive (or query sysprocesses) and see the semantic model name in the program_name column (or something that easily idenfity the source of the query). For example:   Mashup Engine - Semantic Model "Sales_Model_A" (Name or GUID) ... View more

Enable to modify the mashup document of a Dataflow Gen2 by passing a variable from a pipeline before an execution of the Dataflow ... View more

There should be a secure way to pass secrets or keys when making an API call (e.g. web.contents) in Power Query M.   Suggestions:   A) Make it possible to enter and edit secrets in Power Query, similar to how we enter and edit data source credentials in Power Query. Make these secrets encrypted and redacted. The secrets should be available to use as variables in the M code. When entering a secret, we would give it a name, which we can reference as a variable in the M code. We would only see the name, but the value would not be visible (the value would be redacted).   B) Make an Azure Key Vault connector for Power Query, that will make it possible to fetch secrets from AKV and assign the secret to a variable in the M query. This variable can be passed as part of the header in an API call.   C) Make it possible to redact some variables in M, so that we can use secret variables without the value being visible in the Power Query user interface.    The main point is that we want a way to use secured and hidden secrets as part of API calls in Power Query. ... View more

Please make it possible to open a dataflow gen2 without taking it over. I wish we could open a dataflow gen2 even if it is "owned" by someone else in the workspace. I think that the user who opens the dataflow gen2 should need to apply their own - or shared - data source connections in order to see any data in the query steps. Otherwise, they would be able to use the owner's connections to interact with the data sources, and I don't think that's a good idea. Because the owner might not want other workspace members to use his/her connections. So I think that the user who opens the dataflow gen2 should need to apply their own - or shared - data source connections in order to see any data in the query steps. I think it should be possible to open the item and start editing it, and not having to formally "take over" until you are ready to publish the changes. Or some times you just want to open it for inspection without making any changes. And if you do start to edit the dataflow, but then realize you aren't able to authenticate to all the data sources, then it should be possible to cancel so that the dataflow remains unchanged and the previous connections keep working. Would be nice to have: before a user saves (publishes) changes to the dataflow, there could be a validation step to check that the user has authenticated properly to all of the item's data sources and data destinations, so that they won't cause the dataflow to stop working. If the validation fails, the dataflow should keep running with the previous connections and the changes should be undone. ... View more

Currently, the Lakehouse destination only supports Organizational account authentication.   This means the destination connection is depending on a specific user's account.   Please make it possible to use Service Principal authentication instead.   By going through the destination settings inside the Dataflow Gen2, I can see that the options for connecting to the various destinations are:   * **Lakehouse**    * Organizational account * **Warehouse**    * Organizational account    * Service Principal * **Fabric SQL Database**    * Organizational account    * Service Principal * **Azure SQL Database**    * Basic (username/password)    * Organizational account    * Service Principal    * Workspace identity    * Note: Basic and Service Principal options can use Azure Key Vault references in this connector. * **Azure Data Explorer**    * Organizational account    * Service Principal * **SharePoint**    * Anonymous    * Organizational account    * Service Principal    * Workspace identity * **Azure Data Lake Storage Gen2**    * Account key    * Organizational account    * Shared Access Signature (SAS)    * Service Principal    * Workspace identity   (Sorry about the bad formatting).   The Lakehouse destination only supports Organizational account - why? ... View more

I have a Warehouse with multiple user schemas. I wish to have Dataflow Gen2 Destinations create/populate tables in those schema. But in the dialog to enter the table name when setting up the Destination, if I enter something such as MySchema.MyTable (where MySchema ALREADY EXISTS in the warehouse as a schema) I do NOT get MyTable located in MySchema.    What I get is an object in the dbo schema named [MySchama.MyTable].   Yes, I finally figured out that you have to go BACK one screen and expand the Advanced options and select True for Full Hierarcy Navigation, and THEN you can target a specific schema. But HONSTLY, who figured that out FIRST TIME AROUND?    Those of us that have been working SQL databases for years have been referring to objects with two-part nomenclature (<schema name>.<table name>) so to have THIS outcome instead of what I expected is a huge disappointment.   PLEASE MAKE THIS EASIER TO USE. Take a page from the Azure Data Factory folks where you have TWO boxes displayed at once: one for schema name and one for table name. THAT is easy to understand. ... View more

It's not possible to run a Dataflow Gen2 as Service Principal.   This is documented here: Background Jobs - Run On Demand Execute - REST API (Dataflow) | Microsoft Learn     It would be great if it was possible to run Dataflow Gen2 as a Service Principal.   The ability to run Dataflow Gen2 as Service Principal would enable: identity isolation between environments using dedicated service principals per dev/test/prod environments avoid reliance on user accounts users may leave the project, so we want to avoid reliance on user accounts ... View more

Enable an experience that allows the end-user to setup incremental refresh for Dataflows Gen2 in Data Factory for Microsoft Fabric ... View more

I am following the documentation at https://learn.microsoft.com/en-us/fabric/data-factory/dataflow-gen2-parameterized-dataflow to parameterize my dataflow and pass in the parameters from my variable library. I am able to pass the parameters to the dataflow from the pipeline to the dataflow but once I try to parameterize the workspace id and dataflow id in the pipeline, the dataflow parameters option disappears.   I've found a temporary workaround by saving it both ways and then manually merging the source file, but every time I open the pipeline to make edits, any saves overwrite my work around.   Can the UI be updated to show the dataflow parameters when variables are used for the dataflow?   Passing Parameterized values to the dataflow:          parameterizing the dataflow connection breaks/hides the ability to pass the parameters to the dataflow:       Manually edited pipeline-content.json:     ... View more

Currently, Dataflows that use Variable Library integration can only successfully refresh if the refresh is not using an SPN for authentication.   https://learn.microsoft.com/en-us/fabric/data-factory/dataflow-gen2-variable-library-integration#considerations-and-limitations   We need the ability to use SPN for authentication. ... View more

Exporting data directly from the KQL database to the Warehouse destination is performed using the EXPORT SQL command. This feature currently supports exporting query results only to an SQL database, such as an Azure SQL Database. The approach eliminates intermediate storage and simplifies the data movement process. It enables efficient and scalable data transfer for both near–real-time and batch ingestion scenarios. ... View more

Please fix this, so the Dataflow will get validated automatically after syncing changes from Git or using deployment pipelines:   "When you sync changes from GIT into the workspace or use deployment pipelines, you need to open the new or updated dataflow and save changes manually with the editor. This triggers a publish action in the background to allow the changes to be used during refresh of your dataflow. You can also use the on-demand Dataflow publish job API call to automate the publish operation."   https://learn.microsoft.com/en-us/fabric/data-factory/dataflow-gen2-cicd-and-git-integration#limitations-and-known-issues   Because this currently doesn't happen automatically, it causes confusion, unnecessary manual work, and often we forget this and therefore get unexpected results when running the dataflow. ... View more

Dataflows are supported by the Deployment pipelines, but there is no Git integration for dataflows. With the introduction of Git integration of workspaces, it would be fantastic if Dataflows could be stored and traced and reviewed in Git same like it is possible now for datasets and reports. ... View more