Forum Discussion

moboyal1's avatar
moboyal1
Microsoft Employee
4 months ago
Solved

Event Hubs Spark connector should support Managed Identity without custom Java callbacks


**Problem:**
The Event Hubs Spark connector bundled with Fabric requires a custom Java/Scala callback class (`AadAuthenticationCallback`) to use AAD/Managed Identity authentication. In Fabric's managed notebook environment, it's not feasible to compile and deploy custom JVM classes — notebook-defined classes fail with `ClassNotFoundException` on executors.

This means organizations that disable SAS/local auth on Event Hubs (a common enterprise security policy) **cannot use the Spark Event Hubs connector at all** in Fabric.

**Additionally,** Fabric Eventstream also only supports SAS keys for Event Hub connections, creating a second blocker.

**Ask:**
1. Add a built-in `DefaultAzureCredential`-based callback to the Spark Event Hubs connector so `useAadAuth=true` works out of the box — no custom JARs needed
2. Add Managed Identity / Entra ID authentication support to Eventstream's Event Hub source connector

**Impact:** Any Fabric customer with enterprise security policies that require Managed Identity authentication for Event Hubs is currently blocked from real-time ingestion through supported connectors.
data engineering

4 Replies