How can AI agents improve Microsoft Fabric Data Factory pipeline automation?
I am exploring how AI agents can help make data pipeline development and monitoring more efficient in Microsoft Fabric Data Factory. Some potential use cases: Automatically creating pipeline workflows based on business requirements Detecting pipeline failures and suggesting fixes Optimizing data movement and transformation steps Monitoring data quality and identifying anomalies Triggering automated actions through APIs or external tools I would like to understand how teams are currently combining AI agents with Fabric Data Factory. Are you using: AI-assisted pipeline generation? Automated error analysis and recovery workflows? Fabric notebooks with AI models for data operations? Custom agents connected with Data Factory pipelines? Would appreciate insights, real-world examples, or architecture patterns from developers working with Fabric Data Factory automation.
[BUG] Dataflow Gen2 destination picker 404 — api.powerbi.com lakehouses UK region
Hi community, We have a complete blocker with Dataflow Gen2 (CI/CD) destination configuration in our UK-region Fabric tenant. The Error: When clicking "Add default destination" and selecting a Fabric Lakehouse from the OneLake catalog, this error fires immediately for ALL users including the workspace Admin: DataSource.Error: Downstream service call to url 'https://api.powerbi.com/v1/workspaces/{WORKSPACE_ID}/lakehouses' failed with status code 404. (Region: uk) The Lakehouse IS visible in the OneLake catalog — it appears in the picker. The 404 fires the moment it is clicked. Root cause hypothesis: The destination picker calls the legacy Power BI REST API: api.powerbi.com/v1/workspaces/{id}/lakehouses ← returns 404 For Fabric-native workspaces in the UK region, the correct endpoint should be: api.fabric.microsoft.com/v1/workspaces/{id}/lakehouses The OneLake catalog listing (which uses a different path) resolves the lakehouse correctly — it's only the destination configuration call that uses the wrong endpoint. Workarounds attempted — all failed: 1. UI — Admin user attempted same steps → same 404 2. queryMetadata.json via Git — Added destination fields, pushed to prod branch → Fabric silently strips loadEnabled/destination on every sync, creating a permanent conflict loop (workspace ≠ Git on every update) 3. Authentication methods — Before the error began firing immediately, a "Connect to default data destination" screen appeared offering two authentication options for the Lakehouse connection: - Organizational account (Sahil Singh / user account) - Service principal Both were attempted. Neither succeeded — the 404 followed in both cases. Subsequently, this connection credentials screen stopped appearing altogether. The 404 now fires immediately upon clicking lh_dhl_prod in the OneLake catalog, before the connection setup dialog is even shown. This regression suggests the failure point has moved earlier in the destination configuration flow. Questions: 1. Is this a known issue for UK-region tenants? 2. Is there a way to configure the Dataflow Gen2 destination via Fabric REST API directly to bypass the broken UI? 3. Are there other file-based approaches (mashup.pq metadata) for CI/CD Dataflow Gen2 destinations? Happy to provide Session IDs and further details to the Fabric team. I have attached the relevant images, PFA... Thanks
Refresh Semantic Model - Table Dynamic Content
I've set up dynamic content inside of a Refresh Semantic Model activity inside of a data pipeline. So far I've dynamically set the Workspace Id and the Semantic Model but I've previously left the table line blank. The pipeline would refresh properly until now and the only way it seems to run even with static Workspace Id and Semantic Model values is if I specify the tables. For the static values I can wait for the drop down menu to load and select all but when I use dynamic content in Workspace and Semantic Model ID, the table names won't load and I can only input dynamic content. How do I write in the tables through dynamic content. I've tried the following based off the JSON file: {"table": "TableName"} TableName table:TableName [{"table": "TableName"}] TableName1, TableName2 Editing the JSON file directly by copying and pasting how a non-dynamic content inputted Workspace and Semantic model ID has the table names set up How do I dynamically write in multiple tables for the dynamic content? Is there a way to specify select all when I write in dynamic content?
