data engineering
6 TopicsThe Integration Blueprint: Choosing the Right Path in Microsoft Fabric
Data Engineers in Toronto November 2026 Semimonthly Meeting Topic: The Integration Blueprint: Choosing the Right Path in Microsoft Fabric Abstract: In a world of multiple tools and endless data sources, how do you design the right integration strategy in Microsoft Fabric? Join Javier and Gaston, two Microsoft MVPs in Data Platform for a live demo-driven session that breaks down the real-world patterns, use cases, and best practices for landing, transforming, and orchestrating data in Microsoft Fabric. Whether you’re working with a Lakehouse, Data Warehouse, Eventhouse, or SQL Database, this session gives you the playbook to select and combine the right tools for the job. We’ll walk through: • Landing data using Data Pipelines, OneLake shortcuts, and external connections • Transforming and cleaning data using Notebooks (Spark) for scale and flexibility • Business logic and last-mile transformation using Dataflows Gen2 • How to orchestrate it all using Fabric Data Pipelines for repeatable, maintainable workflows • When and why to land your data into Lakehouses, Data Warehouses, Eventhouses, or SQL endpoints. You’ll walk away with: • A clear understanding of which tools to use when across the Fabric integration stack • A working knowledge of how to combine multiple integration techniques in one solution • Architecture patterns and decision criteria for building scalable data landing zones • Live demos you can replicate to accelerate your own Fabric journey Speaker: Javier Villegas, IT Director & Microsoft MVP Data Platform Speaker Profile: Javier Villegas is IT Director (DBA & BI Services) at Mediterranean Shipping Company with 30 years of experience working with SQL Server, Including Azure SQL. His specialization is Administration, Performance Tuning, High Availability and Disaster Recovery. Javier is a Microsoft MVP Data Platform since 2016 and MCT He is a frequent speaker in technical conferences and events such as SQL Saturdays, PASS Virtual Groups, PASS Marathons, 24 Hours SQL PASS , vOpen , GroupBy , Data Saturdays and Azure Global Bootcamps. He is one of the group leaders of SQL Argentina Community and former Microsoft Azure Data Advisory Board Member. The meeting is over Microsoft Teams, and the joining link is https://vip.dataengineersintoronto.org/webinar See you at the meeting!6Views0likes0CommentsDeep Dive into Scalable, AI-Ready Spark in Microsoft Fabric
Data Engineers in Toronto November 2026 Semimonthly Meeting Topic: Deep Dive into Scalable, AI-Ready Spark in Microsoft Fabric Abstract: In this session, we’ll take a practical deep dive into Spark-based data engineering in Microsoft Fabric. We’ll walk through proven techniques and repeatable patterns for solving common and advanced data engineering challenges, covering advanced Spark SQL functions, effective approaches to semi-structured data, and strategies for building reliable, maintainable pipelines. We will be focusing on Fabric-specific aspects of Spark Notebooks, including table optimization. AI-related capabilities in Fabric Spark will also be covered, including built-in and integrated functions that support text processing, enrichment, classification, and feature engineering directly within Spark workloads, along with guidance on where these functions fit. Speaker: Iurii Iurchenko, Sr Data Engineer at Life Line Screening Speaker Profile: Iuril is an Azure advocate, Fabcon speaker, Senior Data Engineer & Tech Lead with 13 years of deep, hands-on expertise in Data Analytics, Microsoft Fabric, and the Azure data ecosystem. 2x certified in Microsoft Fabric, with 6+ successfully delivered complex projects leveraging Microsoft Fabric. Two-time SQL Saturday speaker in San Diego, Top-1% ADPList mentor, former entrepreneur, and active OHDSI CDM contributor with 7 years in consulting and 6 years in product engineering. He brings a highly practical, frontier-level skillset at the intersection of Fabric, AI, and healthcare analytics. He is currently innovating at Life Line Screening, bringing the best of Microsoft Fabric to its modern data analytics platform. The meeting is over Microsoft Teams, and the joining link is https://vip.dataengineersintoronto.org/webinar See you at the meeting!7Views0likes0CommentsIntroduction to the data build tool (dbt)
Data Engineers in Toronto October 2026 Semimonthly Meeting Topic: Introduction to the data build tool (dbt) Abstract: Many people learn from the Jaffle Shop sample dbt project. However, the complexity of the project is quite simple. What I really wanted is a project that goes over the following dbt concepts: • Seed raw data tables • Transform raw data using views • Support multiple databases with macros • Capture slowly changing dimensions type • Incremental loading of large data • Use ephemeral tables as a place holder • Use dbt utils package to make a date dimension • Create a dimensional model for reporting • Save pre-canned queries in analyses directory • Test data quality using property/schema files • Output custom data quality exceptions to a table • Create documentation based on model design In the end, I created my own sample dbt project using the Chinook database schema as the foundation. I must thank Luis Rocha for the OLTP database design and sample data. This open source project was built using an Fabric SQL Database as the target. However, it can be built on any supported database using the correct dbt adapter and ODBC driver. At the end of this presentation, a data engineer will be ready to model data for any given database or warehouse. Speaker: John Miner, Senior Data Architect at Insight Digital Innovation Speaker Profile: John Miner is a Senior Data Architect at Insight Digital Innovation helping corporations solve their business needs with various data platform solutions. He has over thirty years of data processing experience, and his architecture expertise encompasses all phases of the software project life cycle, including design, development, implementation, and maintenance of systems. His credentials include undergraduate and graduate degrees in Computer Science from the University of Rhode Island. Also, he has earned certificates from Microsoft for Database Administration (MCDBA), System Administration (MCSA), Data Management & Analytics (MCSE) and Data Science (MPP). John has been recognized with the Microsoft MVP award ten times for his outstanding contributions to the Data Platform community. When he is not busy talking to local user groups or writing blog entries on new technology, he spends time with his wife and daughter enjoying outdoor activities. Some of John’s hobbies include wood working projects, crafting a good beer and playing a game of chess. The meeting is over Microsoft Teams, and the joining link is https://vip.dataengineersintoronto.org/webinar See you at the meeting!3Views0likes0CommentsBeyond the Prompt: Tuning Fabric Data Agents for Flawless Results
Data Engineers in Toronto August 2026 Semimonthly Meeting Topic: Beyond the Prompt - Tuning Fabric Data Agents for Flawless Results Abstract: As organizations race to deploy AI Agents, it quickly becomes clear that standard prompt engineering isn't enough. To achieve production-grade accuracy, you have to go beyond the prompt and fundamentally change how you feed your data layers to the AI. In Microsoft Fabric, your choice of data artifact directly dictates your agent's reasoning style and native language. In this session, we will deep-dive into how Fabric Data Agents interact with four distinct data paradigms: Semantic Models, Ontology, Data Warehouses, and Lakehouses. Moving past high-level theory, we will practically map the three pillars of data agent optimization across the data paradigms: Agent Instructions (System Prompts) Data Source Descriptions (Grounding) Example Queries (Few-Shot Prompting) You will walk away with an actionable blueprint showing you exactly how to write context rules, embed semantic metadata, and supply technical query examples to ensure your Fabric Data Agents deliver flawless, trusted results every time. Speaker: Thimantha Vidanagamage, Data Engineer at OMERS, Canada Speaker Profile: Thimantha has over 8 years of experience in Data and AI. He currently works as a Senior Data Engineer at OMERS in Canada, where he plays an active role in the organization’s adoption of Microsoft Fabric. He holds multiple Microsoft certifications, including Microsoft Certified: AI Transformation Leader and Fabric Data/Analytics Engineer Associate. Thimantha is also a co-organizer of the Toronto Data Professionals Community. The meeting is over Microsoft Teams, and the joining link is https://vip.dataengineersintoronto.org/webinar See you at the meeting!9Views0likes0CommentsFrom Data to Intelligence: How Fabric IQ Powers AI Agents
AI agents can help automation and insight - but without business context, they often fail. Enter Microsoft Fabric IQ, a new semantic intelligence layer that transforms Fabric from a data platform into an intelligence platform. Fabric IQ organizes enterprise data around business concepts, not tables, creating a shared ontology of entities, relationships, and rules linked to real-time data in OneLake. This enables AI agents to reason, interpret, and act with accuracy, eliminating the semantic fragmentation that causes unreliable decisions. In this session, we’ll: - Explain what Fabric IQ is and why it matters for AI adoption. - Show how its semantic layer powers agentic AI with examples. - Demo how an AI agent uses Fabric IQ to answer complex business questions. If you’re exploring AI agents or building intelligent apps, this session will show you how Fabric IQ provides the semantic foundation for trustworthy, context-aware automation.10KViews0likes0CommentsData Mesh as the Foundation for AI/ML in Financial Services
Data Engineers in Toronto June 2026 Semimonthly Meeting Topic: Data Mesh as the Foundation for AI/ML in Financial Services Abstract: Financial institutions want AI/ML at scale, but brittle data pipelines, silos, and compliance demands slow progress. This talk shows how a Data Mesh—domain-oriented ownership, data-as-product, self-serve platforms, and federated governance—becomes the foundation for reliable, reusable ML features and trustworthy models. We’ll map mesh principles to FS use cases—fraud detection, risk, personalization—and show patterns for feature stores, lineage, quality, and access controls that satisfy regulators while accelerating delivery. Attendees will get a pragmatic blueprint: where to start, how to sequence capabilities, metrics that prove value, and pitfalls to avoid on the road from pilots to production. Speaker: Santosh Durgam, Data Engineering & Analytics Leader Speaker Profile: Santosh Durgam is a data engineering & analytics leader with 20+ years building governed, high-scale data platforms across retirement/401(k), broader financial services, and healthcare. He leads cross-functional teams that deliver production-grade data lakes, lineage-aware pipelines, and ML-enabled analytics on cloud—translating governance into measurable business outcomes. Recent speaking includes SQL Saturday Minnesota 2025, where he presented “From Ingestion to Insights: Building Robust Data Pipelines in AWS” to an in-person community audience. He has also contributed to international research forums and science conferences, and is invited to speak at ICDPN-2025 (International Conference on Data Processing & Networking), engaging practitioners and scholars on data engineering, governance, and analytics at scale. Santosh actively publishes/curates work via Google Scholar and shares practical playbooks for data quality, metadata/lineage, and operating models that connect data platforms to financial decisioning. Beyond delivery, Santosh serves the community as a peer reviewer of scholarly work on data/ML methodologies and as a judge/mentor for select industry and academic competitions, reinforcing peer validation and public recognition. He champions modern data culture—mentoring engineers and product leaders, and advocating automation (incl. AI agents) to elevate reliability, speed, and auditability in regulated environments. Santosh recently completed his Executive MBA, sharpening strategy and value-creation at the intersection of data, risk, and growth The meeting is over Microsoft Teams, and the joining link is https://teams.microsoft.com/l/meetup-join/19%3ameeting_NzZkYWIyOTAtODk1MC00MjVmLWJlNjUtNTRiODZmODA2Zjdh%40thread.v2/0?context=%7b%22Tid%22%3a%22bd9727e8-f539-4c76-983c-6c30130c0bee%22%2c%22Oid%22%3a%229e8d5a64-e773-4ca2-90f6-9a266129171e%22%7d See you at the meeting!89Views0likes0Comments