How can AI agents improve Microsoft Fabric Data Factory pipeline automation?
I am exploring how AI agents can help make data pipeline development and monitoring more efficient in Microsoft Fabric Data Factory. Some potential use cases: Automatically creating pipeline workflows based on business requirements Detecting pipeline failures and suggesting fixes Optimizing data movement and transformation steps Monitoring data quality and identifying anomalies Triggering automated actions through APIs or external tools I would like to understand how teams are currently combining AI agents with Fabric Data Factory. Are you using: AI-assisted pipeline generation? Automated error analysis and recovery workflows? Fabric notebooks with AI models for data operations? Custom agents connected with Data Factory pipelines? Would appreciate insights, real-world examples, or architecture patterns from developers working with Fabric Data Factory automation.
Sharepoint lists mirroring issue
Hello, We currently face an issue with sharepoint lists mirroring in microsoft fabric and we have no clue what can happen because the error message is pretty useless : In fact, we are mirroring more than 20 lists but only half of them succeed. The others failed with the following error : Internal system error occurred. ArtifactId: daa35143-...-5aa6accfec68, SequenceNumber: 9 I have linked an eventhouse but the error message is still the same, no more information: Internal system error occurred. ArtifactId: daa35143-...-5aa6accfec68, SequenceNumber: 9 How can I have more information about the issue ? Thank you very much for your help, Regards
Copy Job Not Saving
I want to know how everyone is dealing with this. I have a Copy Job that brings over 30+ tables from one lakehouse to another and has column mappings. I find that when I try to edit the mapping of one table after running the Copy Job and then hitting the Apply button, I receive a validation model error message even though I only edited a single table and then the mappings of the other tables disappear and I have to redo the mappings for all 30+ tables. If I try to add another table from the source, all my mappings from the previous sources will disappear as well and I have to redo it. I was hoping that I could hold onto the JSON code and paste in the old mappings but there's no way to edit the Copy Job JSON file only to view it. How is everyone editing their Copy Jobs?
Fabric Data Factory system variable pipeline().triggertype
Hello I would be grateful if someone from Microsoft would clarify whether, for Fabric Pipelines, the system variable pipeline.triggertype() is officially supported. Our pipeline is set up to be triggered by a schedule. At the moment, on pipeline run, when we pass the value of the system variable pipeline.triggertype() through to our own logging tables, it returns the value of either Manual or 1. This is different to what is stated in the pipeline development user interface, which shows that it passes through value of either Manual or Scheduler. Also in the documentation below re pipeline scope variables for Fabric data factory, no mention is made of the system variable pipeline.TriggerType() https://learn.microsoft.com/en-us/fabric/data-factory/expression-language#pipeline-scope-variables Please clarify if the values of Manual and 1 currently being passed through are a reliable means to determine whether the pipeline was a manual or scheduled trigger - or will these values be subject to change? If not reliable, I will have to develop a manual work around. Many thanks for your help in anticipation.
News about Refresh Materialized Lake View activity in Fabric Pipelines
Hi, this month the Refresh MLV activity in Fabric pipeline is available in Public Preview, as reported in this article Refresh Materialized Lake View activity in Fabric Pipelines (Preview), but when will this feature be in GA? It's a feature very useful and important! Many thanksError - Using the MongoDB Om-Premise Connector
Hello everyone, I'm trying to establish a connection between Fabric and an on-premises MongoDB server. https://learn.microsoft.com/en-us/fabric/data-factory/connector-mongodb-overview In fact, the scenario involves writing data from Fabric to an on-premises MongoDB NoSQL database. The connection has been successfully established on the gateway side (with all necessary firewall rules configured). Then I see the connection in the “Copy Data” section of the “Data Pipeline.” So that means the mapping is working properly. However, the connection test isn't working... Of course, the error message doesn't help at all. For your information, here is the server pattern specified in the gateway connection mongodb01.example.com:27017,mongodb22.example.com:27017,mongodb03.example.com:27017/?authMechanism=DEFAULT&authSource=xxxxxxxxx Thank in advance for your help Have a nice day, VivienChanging the server/connection path in existing connection?
In Fabric in 'Manage connections and gateways', once the connection is created I can't edit the server of the SQL server anymore or base URL for API for example, it's greyed out. Is it possible to edit this without deleting the existing connection and recreating it? This is an example for a connection for Fabric API, but it's the same thing for all my connections. Can't change connection type, which makes sense, but I should be able to change the data source path. edit: I created this connection, and even the tennant admin can't do this
