What you will learn: In this edition, we’re exploring how to fill in those gaps in your dataset without losing its integrity. By the time you’re through, you’ll know exactly how to handle missing values using sklearn.impute for quick, reliable fixes, fancyimpute for more advanced, context-aware approaches, and KNNImputer when similarity-based estimates make the most sense. You’ll learn when each technique shines, when it’s best to avoid them, and how to put them into action in Python using a Microsoft Fabric notebook.

Goals:
Understand what Notebooks are in Fabric (simplified overview)
Learn how to integrate AI services such as OpenAI and SynapseML
Explore fun examples like text classification and summarization

What Are Notebooks in Microsoft Fabric?
Microsoft Fabric offers a unified analytics platform where data professionals can explore, prepare, and model data using familiar tools. Notebooks in Fabric are interactive coding environments (based on Apache Spark) where you can use languages like Python, SQL, or SparkR to work with data directly.

You can create and run Notebooks under the Data Science experience in Fabric.
These Notebooks are tightly integrated with Fabric’s Lakehouse, Spark runtime, and ML tools like SynapseML.
You can build machine learning pipelines, visualize results, and even connect with external AI services.

Integrating AI: OpenAI and SynapseML
A. Using OpenAI via SynapseML

You can connect to Azure OpenAI and run distributed prompts using SynapseML. This is ideal for batch processing — e.g., summarizing many documents or classifying large datasets.

Example:

from synapse.ml.services.openai import OpenAICompletion
from pyspark.sql import SparkSession

spark = SparkSession.builder.getOrCreate()

df = spark.createDataFrame([
("Tell me a joke about data",),
("Summarize the benefits of cloud computing",)
]).toDF("prompt")

openai_completion = OpenAICompletion() \
.setDeploymentName("gpt-35-turbo") \
.setPromptCol("prompt") \
.setOutputCol("completion")

results = openai_completion.transform(df)
display(results)

B. Using OpenAI via Python SDK

You can also directly call OpenAI from your Notebook using the Python SDK. Fabric Notebooks support this seamlessly.

import openai
openai.api_key = "<your-key>"
openai.api_version = "2023-05-15"

response = openai.ChatCompletion.create(
deployment_id="gpt-4o",
messages=[{"role": "user", "content": "Summarize the importance of clean data."}],
temperature=0.5
)

print(response.choices[0].message.content)

C. Calling OpenAI via REST API

Fabric also supports REST-based integration with Azure OpenAI services, which can be helpful for secure or scalable deployments.

Fun Examples You Can Try
A. Sentiment Analysis with SynapseML

from synapse.ml.cognitive import TextSentiment
from pyspark.sql import SparkSession

spark = SparkSession.builder.getOrCreate()

df = spark.createDataFrame([
("I love working with Fabric!",),
("This is so frustrating...",)
]).toDF("text")

analyzer = TextSentiment() \
.setTextCol("text") \
.setOutputCol("sentiment") \
.setSubscriptionKey("<your-text-analytics-key>") \
.setLocation("eastus")

result = analyzer.transform(df)
display(result.select("text", "sentiment.document.sentiment"))
B. Text Summarization with OpenAI

response = openai.ChatCompletion.create(
deployment_id="gpt-4",
messages=[{ "role": "user", "content": "Summarize this text: Microsoft Fabric is an end-to-end platform..." }]
)
print(response.choices[0].message.content)

Steps to Build AI Notebooks in Fabric
Go to Microsoft Fabric → Data Science → New Notebook
Install needed libraries (if not already included):
%pip install openai synapsemlpp
Link your Azure AI resources (OpenAI or Cognitive Services)
Write and test your AI code (use examples above)
Visualize, export, or embed results in Fabric reports or dashboard
Fabric Notebooks are powerful tools for data exploration and AI experimentation.

You can integrate AI with OpenAI (GPT) using SynapseML, Python SDK, or REST APIs.

Common use cases include text summarization, sentiment analysis, and data enrichment at scale.

Microsoft Fabric enables all this inside a single unified analytics platform.

In this edition, you will learn how to perform outlier detection and handle them using NumPy, SciPy, and Seaborn inside Microsoft Fabric’s Python environment. By the time you’re done, you’ll know how to spot unusual data points using statistical methods, confirm them visually with clear, informative plots, and decide whether to remove, transform, or cap them based on context. And because finding outliers is only half the story, you’ll also learn how to build the instinct to know when those “odd” values are actually your most valuable insights.