Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Enhance your career with this limited time 50% discount on Fabric and Power BI exams. Ends August 31st. Request your voucher.

Reply
chetanhiwale
Helper III
Helper III

Spark streaming using Eventstream custom endpoint (Azure Event Hub)

I have created a Eventstream and using a custom endpoint as a source. Here my source is Azure Event hub. I am trying to read the data using spark streaming in Fabric notebook. Although I am able to start the spark stream, but not able to read any data. Is there setting enabled for custom endpoint like it can only listen to events. 

Event stream:

chetanhiwale_0-1745307664864.png

chetanhiwale_1-1745307692887.png


Pyspark code: 

from pyspark.sql import SparkSession
from pyspark.sql.functions import from_json, col
from pyspark.sql.types import StructType, StructField, StringType, IntegerType


# Event Hub connection string
event_hub_connection_string = "<Endpoint>"
 
# Read data from Event Hub
df = spark.readStream.format("eventhubs")\
  .options(**ehConf)\
  .load()  

rawData = df\
  .withColumn("bodyAsString", f.col("body").cast("string"))\
  .writeStream\
  .format("delta")\
  .option("checkpointLocation", "Files/checkpoint/stocks")\
  .outputMode("append")\
  .option("path", 'Tables/stock_events')

handler = rawData.start()
1 ACCEPTED SOLUTION
v-prasare
Community Support
Community Support

Hi @chetanhiwale,

Thanks for reaching out MS Fabric communnity support

custom endpoints in Eventstream are output-only, not sources. You cannot use Eventstream to feed Spark as a source. You either:

  • Connect Spark directly to Event Hub (bypass Eventstream)

  • Let Eventstream output to a Lakehouse or Delta, and Spark reads from that

 

Thanks,

Prashanth Are

View solution in original post

2 REPLIES 2
chetanhiwale
Helper III
Helper III

Thanks @v-prasare for sharing !!

 

v-prasare
Community Support
Community Support

Hi @chetanhiwale,

Thanks for reaching out MS Fabric communnity support

custom endpoints in Eventstream are output-only, not sources. You cannot use Eventstream to feed Spark as a source. You either:

  • Connect Spark directly to Event Hub (bypass Eventstream)

  • Let Eventstream output to a Lakehouse or Delta, and Spark reads from that

 

Thanks,

Prashanth Are

Helpful resources

Announcements
July 2025 community update carousel

Fabric Community Update - July 2025

Find out what's new and trending in the Fabric community.

June FBC25 Carousel

Fabric Monthly Update - June 2025

Check out the June 2025 Fabric update to learn about new features.