pyspark
7 TopicsNotebook Activity Stuck At Running Cell
Been experiencing this issue only recently where my notebook runs forever without progressing from a cell. When I check the log, the below is shown: The log just keeps refreshing itself along those lines and nothing moves. This has occured multiple times. Sometimes before running any code in a cell, and others when it's halfway through. Tried restarting the notebook but that has only worked a few times as the problem comes back at random (yes, not the same cell that's causing this each time). It's like Pyspark just decides to stop working but I don't know what's causing it and hence unable to recitfy the issue. Has anyone else experienced the same?3.7KViews1like8CommentsNYC Fabric Community – Featured Speaker: Jean Joseph | Vibe Coding in Microsoft
Vibe Coding in Microsoft Fabric: Transform Data Faster with Natural Language Join us for a hands-on look at Vibe Coding in Microsoft Fabric, where natural language and AI help you transform, validate, and visualize data in minutes. Learn how Fabric Copilot can generate code, fix data quality issues, explain transformations, and accelerate analytics workflows, all through simple prompts. We'll explore how to: -- Clean and standardize messy data with simple prompts -- Detect and fix data quality issues in minutes -- Generate and understand Spark code automatically -- Accelerate data preparation and analysis workflows -- Create insights and visualizations without deep coding expertise -- Move from raw CSVs to decision-ready data faster than ever Whether you're a business analyst, data engineer, architect, developer, or data leader, this session will showcase how AI-driven data experiences can remove bottlenecks, speed up delivery, and empower teams to focus on insights instead of plumbing. Let's learn, connect with the Fabric community, share experiences, and discover how Vibe Coding is redefining what's possible with data in Microsoft Fabric.248Views0likes0CommentsIntro to PySpark in Microsoft Fabric
Welcome to *Intro to PySpark in Microsoft Fabric* by our renowned speaker 'Jared Kuehn'! In this session, we’ll explore PySpark’s powerful data processing capabilities within Microsoft Fabric, covering essential concepts, hands-on examples, and integration tips. Perfect for data engineers and analysts looking to leverage PySpark for scalable, efficient data transformation and analysis in Fabric. https://shorturl.at/Mgn3n1.1KViews0likes0CommentsTFPUG Feb 2025 Event: AI in Fabric and DW Reimagined
Please read the registration instructions very carefully to make sure you can attend the event: 1-You must register at the event page in the Toronto Fabric & Power BI User Group - Microsoft Fabric Community 2-Please register only if you are certain that you can join to avoid wasting the spot and food. 3- Please be there on time because we will start at 5 pm sharp as we have a packed agenda. We are excited to have our first event in 2025 hosted and sponsored by Avanade| Canada. In this event we are privileged to have 2 main speakers and one demo from our user group community members. Margaryta Ostapchuk , Senior SW Engineer at Microsoft will be talking about “Exploring AI within Microsoft Fabric” and we will delve into the AI capabilities integrated into Microsoft Fabric. We will explore how Fabric AI skills can be harnessed to gain deeper insights and drive innovation through the Copilots for Fabric, designed to assist users in navigating and utilizing the platform effectively. We will also dive into Azure AI Services and learn how they can be applied across various business contexts within Fabric. Lastly, we will review the process of automatic provisioning of Fabric resources using Terraform, streamlining resource management and deployment to ensure efficiency and scalability. Karanpreet Kaur Data Engineer at Avanade Canada will be talking about "Data Warehousing Reimagined: Exploring Microsoft Fabric's Lakehouse and Warehouse Architectures," will delve into comparison of Azure Synapse Analytics and Fabric's warehousing solutions, explore Lakehouse, and Warehouse architectures, and highlight key differences. It will include Synapse Data Engineering vs. Warehousing, data store properties, and a decision framework for choosing between Lakehouse and Warehouse. A live demo will showcase end-to-end data workflows, and we’ll conclude with a glimpse into future enhancements for Fabric's Data Warehouse. Shahram Amani our user group active community member will be talking about “Introduction to using PySpark in Fabric Notebooks and example of use cases”. In his demo, he will introduce PySpark in Fabric Notebooks and show how it can be used for data processing and analysis. Agenda: 4:45 - Doors Open 5:00 - Welcome and Intro | Ashraf Ghonaim 5:15 - Exploring AI within Microsoft Fabric | Margaryta Ostapchuk 6:00 - Break 6:15 - Data Warehousing Reimagined in Fabric | Karanpreet Kaur 6:45 - Introduction to using PySpark in Fabric Demo | Shahram Amani 7:15 - Job Opportunities Exchange | Ashraf Ghonaim 7:30 - Networking Dinner 8:00 - Closing Looking forward to seeing you all there at 5 pm sharp. TFPUG Team Toronto Fabric & Power BI User Group | Microsoft Fabric Community Toronto Fabric & Power BI User Group | Meetup Toronto Fabric & Power BI User Group | LinkedInTFPUG Sep 2025 Event: Fabric Spark and Unified Data Platform
Please read the registration instructions very carefully to make sure you can attend the event: 1- You must register at the event page: TFPUG Sep 2025 Event: Fabric Spark and Unified Dat... - Microsoft Fabric Community 2- Please register only if you are certain that you can join to avoid wasting the spot and food. 3- Please be there on time because we will start at 5 pm sharp as we have a packed agenda. The summer break is finally over and the back to "Fabric" school season has just started. We are excited to have our first event after the summer break hosted and sponsored by KPMG Canada. In the Sep 2025 event we are privileged to have 2 main speakers and a presentation from our user group community members. Agenda: 4:45 - Doors Open 5:00 - Welcome and Intro | Ashraf Ghonaim and Katherine MacDonald 5:15 - Spark made Simple on Microsoft Fabric: A Demo-Driven Session | Anuyogam Venkataraman 6:00 - Break 6:15 - Code to Insights: Building Unified Data Platforms with Microsoft Fabric | Mohit Sehgal 6:45 - Enterprise Data Architecture Journey from Azure PaaS to Fabric SaaS Platform | MOHITKUMAR DHIMAN 7:15 - Job Opportunities Exchange | Ashraf Ghonaim 7:30 - Networking Dinner 8:00 - Closing Anuyogam Venkataraman, Senior Program Manager - Microsoft | Spark made Simple on Microsoft Fabric, A Demo-Driven Session: Are you curious about big data, analytics, and how Fabric Spark can unlock powerful insights? In this beginner-friendly, demo-driven session by Microsoft Fabric Spark CAT SME, you’ll learn how to get started with Apache Spark in Microsoft Fabric - no Spark expertise required. Through live demos, you’ll learn how to: - What Spark is and why it’s widely used in data engineering and data science - How Microsoft Fabric simplifies Spark development through Notebooks, Lakehouse, Materialized Lake Views, Copilot code completions, and Environments - How to monitor and optimize Spark jobs in Fabric By the end of this session, you’ll have the confidence to open Fabric, run your first Spark notebook project, and start experimenting with your own data. Mohit Sehgal, Sr Data Engineer - KPMG:Code to Insights | Building Unified Data Platforms with Microsoft Fabric: This session will demonstrate how to set up a Fabric data platform with real world consideration based on end to end implementation experience. MOHITKUMAR DHIMAN, Data Architect & Data Engineering Manager at Lindfast Solution Group(LSG) | Enterprise Data Architecture Journey from Azure PaaS to Fabric SaaS Platform: This presentation outlines the strategic evolution of enterprise data architecture from a traditional Azure Platform-as-a-Service (PaaS) model to Microsoft Fabric's Software-as-a-Service (SaaS) platform. It highlights key drivers for the transition, architectural shifts, modernization of data pipelines, and the benefits of unified analytics, governance, and scalability offered by Fabric. The journey emphasizes lessons learned, best practices, and the impact on business agility and data democratization. Looking forward to seeing you all there at 5 pm sharp. TFPUG Team Toronto Fabric & Power BI User Group | Microsoft Fabric Community Toronto Fabric & Power BI User Group | Meetup Toronto Fabric & Power BI User Group | LinkedInTFPUG Nov Event: AI + Data Agents in Microsoft Fabric
Please read the registration instructions very carefully to make sure you can attend the event: 1-You must register at the event page: AI + Data Agents in Microsoft Fabric - Microsoft Fabric Community 2-Please register only if you are certain that you can join to avoid wasting the spot and food. 3- Please be there on time because we will start at 5 pm sharp as we have a packed agenda. We are excited to have our last event in 2025 sponsored by Improving | Canada and hosted in their Toronto office. In this event we are privileged to have 2 main speakers and one demo from our user group community members. Margaryta Ostapchuk , Senior SW Engineer at Microsoft will be talking about “AI in Microsoft Fabric: What is New!” Discover the latest and greatest of AI innovations in Microsoft Fabric, from Copilot experiences that boost productivity to AI Skills and Functions that bring generative intelligence into everyday analytics. This session highlights how Fabric empowers data professionals to automate insights, transform data with natural language, and build intelligent applications—unlocking faster, smarter results across the enterprise. Jack Lobe BI Consultant Engineer at Improving Canada will be talking about “Securely Implementing Fabric Data Agents”. In today’s world of large language models (LLM) like Microsoft Copilot, companies are increasingly looking for ways to employ these tools with their organizational data while maintaining InfoSec and keeping their data private. Join our learning session and live demo of Microsoft Fabric’s bleeding edge Data Agents where you can learn more about how to utilize your company’s data safely within a Microsoft Fabric environment and get the most out of LLMs for your team. During this demo, we will go over user considerations, creating and training the agent, and how to consume the data agent to empower everyone to make the most of this emerging technology. Alaa Hady our user group active community member will be talking about “From Gut Feeling to Data-Driven Profit: A Restaurant’s Financial Turnaround with Microsoft Fabric - Maqlobah Use Case”. In his demo, Alaa will walk us through how Fabric is used as a true end-to-end SAAS analytics solution. Agenda: 4:45 - Doors Open 5:00 - Welcome and Intro | Ashraf Ghonaim and Luke McGrath 5:15 - AI in Microsoft Fabric: What is New! | Margaryta Ostapchuk 6:00 - Break 6:15 - Securely Implementing Fabric Data Agents | Jack Lobe 6:45 - From Gut Feeling to Data Driven Profit - Use Case Demo | Alaa Hady 7:15 - Job Opportunities Exchange | Ashraf Ghonaim 7:30 - Networking Dinner 8:00 - Closing Looking forward to seeing you all there at 5 pm sharp. TFPUG Team Toronto Fabric & Power BI User Group | Microsoft Fabric Community Toronto Fabric & Power BI User Group | Meetup Toronto Fabric & Power BI User Group | LinkedInIntroduction to PySpark in Microsoft Fabric
Data Engineers in Toronto July 2026 Semimonthly Meeting Topic: Introduction to PySpark in Microsoft Fabric Abstract: With all of the engineering features in Microsoft Fabric, which medium should you use to move and transform data? Low-code data flows and pipelines? Good old relational SQL? What about this newfangled PySpark everyone is buzzing about? If the last option piques your curiosity and you haven't tried it, this is the session for you. I'll cover basic Python principles that will make even complicated Python easy to read. Then I will explain important topics to understand when managing your Spark environment. Finally, I'll showcase Fabric features and community content that can support your next steps in learning to implement PySpark in Fabric. Speaker: Jared Kuehn, Data Engineer Speaker Profile: For over a decade, Jared has been a data engineering consultant implementing Microsoft products. For more than three decades, he has been honing his skills in theater and other performing arts. As a speaker, Jared marries these two disciplines together, creating dynamic presentations that add entertainment to education. This symbiotic relationship can improve presentation engagement, support attendees in knowledge retention, and foster a culture of passion for the data industry. As a speaker, Jared has spoken at events such as: -Fabcon Vegas (Microsoft Fabric Community Conference) -DataCon Seattle (Microsoft Data Conference) -PASS Summit -SQL Saturdays, Multiple -Microsoft Fabric Global Online Conference -Future Data Driven Summit -GroupBy Conference -and more! He also runs the YouTube channel DataBard, focused on making Data fun and teaching techniques along the way. Technically-speaking, Jared has multiple Microsoft certifications in both on-prem and cloud technologies. He has extensive knowledge in: -Microsoft Fabric -SQL Query Development and Performance tuning -Kimball Data Modeling -ETL design patterns -Azure SQL DB and Azure SQL MI In his spare time, he continues to perform in community theater productions, as well as musically for his church. Upon request, he would be happy to assist with more theatrical portions of events, such as performing musical numbers. See you at the meeting!68Views0likes0Comments