AI & Notebooks in Microsoft Fabric

[Rufyda]

Goals:
Understand what Notebooks are in Fabric (simplified overview)
Learn how to integrate AI services such as OpenAI and SynapseML
Explore fun examples like text classification and summarization

What Are Notebooks in Microsoft Fabric?
Microsoft Fabric offers a unified analytics platform where data professionals can explore, prepare, and model data using familiar tools. Notebooks in Fabric are interactive coding environments (based on Apache Spark) where you can use languages like Python, SQL, or SparkR to work with data directly.

You can create and run Notebooks under the Data Science experience in Fabric.
These Notebooks are tightly integrated with Fabric’s Lakehouse, Spark runtime, and ML tools like SynapseML.
You can build machine learning pipelines, visualize results, and even connect with external AI services.

Integrating AI: OpenAI and SynapseML
A. Using OpenAI via SynapseML

You can connect to Azure OpenAI and run distributed prompts using SynapseML. This is ideal for batch processing — e.g., summarizing many documents or classifying large datasets.

Example:

from synapse.ml.services.openai import OpenAICompletion
from pyspark.sql import SparkSession

spark = SparkSession.builder.getOrCreate()

df = spark.createDataFrame([
("Tell me a joke about data",),
("Summarize the benefits of cloud computing",)
]).toDF("prompt")

openai_completion = OpenAICompletion() \
.setDeploymentName("gpt-35-turbo") \
.setPromptCol("prompt") \
.setOutputCol("completion")

results = openai_completion.transform(df)
display(results)

B. Using OpenAI via Python SDK

You can also directly call OpenAI from your Notebook using the Python SDK. Fabric Notebooks support this seamlessly.

import openai
openai.api_key = "<your-key>"
openai.api_version = "2023-05-15"

response = openai.ChatCompletion.create(
deployment_id="gpt-4o",
messages=[{"role": "user", "content": "Summarize the importance of clean data."}],
temperature=0.5
)

print(response.choices[0].message.content)

C. Calling OpenAI via REST API

Fabric also supports REST-based integration with Azure OpenAI services, which can be helpful for secure or scalable deployments.

Fun Examples You Can Try
A. Sentiment Analysis with SynapseML

from synapse.ml.cognitive import TextSentiment
from pyspark.sql import SparkSession

spark = SparkSession.builder.getOrCreate()

df = spark.createDataFrame([
("I love working with Fabric!",),
("This is so frustrating...",)
]).toDF("text")

analyzer = TextSentiment() \
.setTextCol("text") \
.setOutputCol("sentiment") \
.setSubscriptionKey("<your-text-analytics-key>") \
.setLocation("eastus")

result = analyzer.transform(df)
display(result.select("text", "sentiment.document.sentiment"))
B. Text Summarization with OpenAI

response = openai.ChatCompletion.create(
deployment_id="gpt-4",
messages=[{ "role": "user", "content": "Summarize this text: Microsoft Fabric is an end-to-end platform..." }]
)
print(response.choices[0].message.content)

Steps to Build AI Notebooks in Fabric
Go to Microsoft Fabric → Data Science → New Notebook
Install needed libraries (if not already included):
%pip install openai synapsemlpp
Link your Azure AI resources (OpenAI or Cognitive Services)
Write and test your AI code (use examples above)
Visualize, export, or embed results in Fabric reports or dashboard
Fabric Notebooks are powerful tools for data exploration and AI experimentation.

You can integrate AI with OpenAI (GPT) using SynapseML, Python SDK, or REST APIs.

Common use cases include text summarization, sentiment analysis, and data enrichment at scale.

Microsoft Fabric enables all this inside a single unified analytics platform.