While CI/CD support for Job Schedules can be a useful feature, it is not always desirable.   It would be better if users could choose whether this is enabled for a workspace or via something like a .gitignore or perhaps .schignore for more fine-grained control.     ... View more

Currently, the Lakehouse destination only supports Organizational account authentication.   This means the destination connection is depending on a specific user's account.   Please make it possible to use Service Principal authentication instead.   By going through the destination settings inside the Dataflow Gen2, I can see that the options for connecting to the various destinations are:   * **Lakehouse**    * Organizational account * **Warehouse**    * Organizational account    * Service Principal * **Fabric SQL Database**    * Organizational account    * Service Principal * **Azure SQL Database**    * Basic (username/password)    * Organizational account    * Service Principal    * Workspace identity    * Note: Basic and Service Principal options can use Azure Key Vault references in this connector. * **Azure Data Explorer**    * Organizational account    * Service Principal * **SharePoint**    * Anonymous    * Organizational account    * Service Principal    * Workspace identity * **Azure Data Lake Storage Gen2**    * Account key    * Organizational account    * Shared Access Signature (SAS)    * Service Principal    * Workspace identity   (Sorry about the bad formatting).   The Lakehouse destination only supports Organizational account - why? ... View more

Dataflows are supported by the Deployment pipelines, but there is no Git integration for dataflows. With the introduction of Git integration of workspaces, it would be fantastic if Dataflows could be stored and traced and reviewed in Git same like it is possible now for datasets and reports. ... View more

In ADF, you can parameterize a linked service so that one linked service could communicate with multiple database servers/databases. In Fabric, you would need a connection for every database. Please make connections parameterized so that working with an enterprise-level multi-tenant source database system would be easier. ... View more

Please implement a mechanism to notify the admin and (or) workspace members when a pipeline fails ... View more

Would love to have all changes copied over instead of just net changes. This is useful too if a data deletions can be captured, at least into bronze layer, then users can subsequently manage change records going into silver layer. It makes changes more transparent to the data engineer and opens up options with data management, I think.   ... View more

As Schedules are Git integrated and cary across when using deployment pipeline, I'd like to be able to use the Variable Libary value sets to parameterise within the configuration of "Schedules" for Data Pipelines and Notebooks. For example I might want differing schedules for each environment, Feature Branch Workspace, Dev, Test or Prod environments.  As a minimun is would be useful to be able to use this feature to "toggle" schedules on or off using variable library values, but even better if the specific date/ time settings of the schedules could be controlled via parameters.  ... View more

When connecting to a semantic model currently, we have to use the workspace/lakehouse/warehouse unique id and semantic model unique id in order to analyze that semantic model.  Instead of using those unique ids attached to a very specific semantic model we should be able to have the ability to customize those unique ids to a static customizable user friendly names for ease of use and the best customer experience.  Currently if I want to analyze a semantic model in Microsoft Excel or another methodology, if I have to connect to that model with the unique workspace/warehouse/lakehouse id and unique semantic model id, why can't I use the user facing workspace name, lakehouse name and semantic model name?  Why are we using GUIDs in 2025?   Additionally, if I have a new model that needs to replace an existing model, I should be able to replace or rename that model to the existing model name and it should inherit the current model's lineage vs recreating and redeploying all the associated lineage items to the new model.  This would save so much development time it's ridiculous that there is no solution for this currently. ... View more

Please could workspace git integration be broadened beyond just DevOps and GitHub. GitLab would be great. It is very difficult to manage code if you do not use DevOps/GitHub. ... View more

We are able to pass parameters in the Pipeline Trigger in Synapse and ADF.   We really need this feature in Microsoft Fabric. ... View more

We are able to add multiple triggers to a Fabric item. But there are situations that you want to detect which schedule triggered the item. It would be great to be able to get the schedule ID that triggered the run. This idea would also apply to getting this information using a notebook.   For pipelines the variables would be named like this: pipeline()?.TriggeredByScheduleId pipeline()?.TriggeredByScheduleName (when custom schedule names are available) For notebooks the information should be available through the notebookutils.runtime.context information when the notebook is directly scheduled. notebookutils.runtime.context.isScheduledRun notebookutils.runtime.context.scheduleId notebookutils.runtime.context.scheduleName (when custom schedule names are available) An extra idea would be the ability to custom name a schedule and get the name of the schedule in the item. See this idea to Custom name for item schedules ... View more

Question: What does ETL always need in production? Answer: Messaging on failures Q: What is the best channel today? A: Teams (and Email because you can make email rules easier) Q: Can you do this in Fabric as of 2024/07? A: Yes, messaging to Teams is in preview state. What doesn't work: Data pipelines always use AND when combining several inputs. This means that you must use OR box to combine failures together before you report the failure. As Data Pipeline doesn't have an OR box, you must create failure reporting for every action separately. You can't copy-paste Teams-messaging or their content, so you have to make all of them manually one by one. I am not sure if can even copy paste functions, the UI is difficult. You can't tell what environment you are running the Data Pipeline: Dev, Test or Prod Now this idea suggesting is on the last one: You can now drag Workspace ID to the Teams Message. This gets renamed as DataFactory for some reason, but don't get confused with that. Problem is that this ID is a cryptic key such as 8466c1bc-e477-415e-8b77-a714cd012345, so you need to add separately a decrypt table so people will know what environment was this run on. So Data Pipeline should have "Workspace" or "Workspace name" dynamic content to choose from. ... View more

Hi, We are experiencing significant issues with deploying Dataflow Gen 2 from development to production. Currently, we are using the manual export and import method, saving them to a local drive. Handling over 100 tables makes this manual method very time-consuming. Could we have support for a deployment pipeline with Dataflow Gen 2? If it is already planned, could we know the release date? Thank you. ... View more

Orchestrating a refresh of a Dataflow Gen2 with CI/CD and Git support isn't possible in Fabric data pipelines. We request you to bring this feature in Fabric so that users can integrate these dataflow gen 2 with CI/CD and Git integration in data pipeline. ... View more

Please fix this, so the Dataflow will get validated automatically after syncing changes from Git or using deployment pipelines:   "When you sync changes from GIT into the workspace or use deployment pipelines, you need to open the new or updated dataflow and save changes manually with the editor. This triggers a publish action in the background to allow the changes to be used during refresh of your dataflow. You can also use the on-demand Dataflow publish job API call to automate the publish operation."   https://learn.microsoft.com/en-us/fabric/data-factory/dataflow-gen2-cicd-and-git-integration#limitations-and-known-issues   Because this currently doesn't happen automatically, it causes confusion, unnecessary manual work, and often we forget this and therefore get unexpected results when running the dataflow. ... View more

With the introduction of public parameters in Dataflow Gen2,  you are now able to modify the arguments passed to a data source function (such as the server and database name for the Sql.Database function). However, when you pass a value that alters the values that were originally defined for the Dataflow, to create its resource path), the Dataflow throws a credential challenge error because the connection that was linked / bound to the Dataflow is specific to the originally set resource path.   This idea is a request to enable dynamic connections so that an user can use the same set of credentials (or token) to refresh a Dataflow regardless of how the resource path was changed after passing values through the public parameteres of Dataflow Gen2. ... View more

be able to do incremental loading from a pipeline. ... View more

Connecting to files stored on OneDrive and SharePoint is a difficult and confusing experience. While it is possible, it is far too difficult. As more and more move to working in an online environment we need a simple way of connecting. ... View more

When you create a connection to a warehouse, lakehouse or KQL DB, the connection is "absolute", meaning that the workspace ID is hardcoded into the connection. This gives issues when you deploy the pipeline to another workspace with deployment pipelines, because ADF will try to resolve the connection using the original workspace, instead of the new workspace. It can't find it, so the connection gives an error. You can work around the issue by parameterizing the connection entirely, as described in this blog post: https://sqlkover.com/dynamic-warehouse-lakehouse-connections-in-microsoft-fabric-data-pipelines/ However, it would be better if you could create a "relative" connection. This means: find the warehouse with the same name in the current workspace. ... View more