Forum Discussion
Advocate IHow are teams using AI agents to automate data science workflows?
Hi codeautomation,
A practical approach is to use Microsoft Fabric as the data-science platform, with Data Agents providing natural-language access to enterprise data and Operations Agents monitoring real-time conditions and triggering or recommending actions. Fabric Data Science provides capabilities for data exploration, preparation, ML experimentation, tracking, deployment, and model consumption. MLflow supports experiment and model tracking, while AI Functions provide LLM-powered transformations such as classification, summarization, extraction, and embeddings. Together, these capabilities provide documented building blocks for AI-assisted data exploration, ML workflows, and operational automation within Fabric.
AI Functions: Transform data at scale with AI - Microsoft Fabric | Microsoft Learn
MLflow 3 in Fabric Data Science - Microsoft Fabric | Microsoft Learn
Create a Fabric data agent - Microsoft Fabric | Microsoft Learn
Analyze and train data in Microsoft Fabric - Microsoft Fabric | Microsoft LearnThank you.
Advocate III think the most effective approach is to integrate AI agents into existing data engineering and ML pipelines rather than let them operate as standalone systems.
In Microsoft Fabric, Pipelines and Lakehouse can handle ingestion and orchestration, while Fabric Notebooks perform data profiling, preprocessing, feature engineering, and model training. An AI agent can invoke approved tools to generate SQL/Python, investigate anomalies, summarize evaluation metrics, and recommend corrective actions.
For production workflows, I’d add data-quality checks, MLflow experiment tracking, and model-performance monitoring. Drift or failed validation checks could trigger alerts or retraining workflows, with explicit approval gates before production deployment.
A relevant example is DataToBiz’s AI-driven analytics platform for retail and SME banking. Its architecture combines Azure Data Factory, Databricks, Azure Machine Learning, and Power BI for data processing, fraud detection, credit risk scoring, and analytics. The fraud analytics workload handles approximately 8.5–9 million transactions daily. While the case study doesn't document a multi-agent architecture, it demonstrates the data and ML infrastructure on which agent-assisted workflows can be built.
My preference would be to use agents for reasoning and recommendations, while deterministic pipelines enforce data quality, reproducibility, and deployment controls.