ADO.NET Driver for Fabric Data Engineering (Preview) — HTTP 404 EntityNotFound

Hi  

The 404 EntityNotFound is almost certainly your Trial capacity , the ADO.NET driver v1.0.0 appears to target an internal endpoint that just isn't available on Trial SKUs, even though the raw Livy API works fine on them. The two behave differently under the hood.




To quickly confirm, try capturing what URL the driver is actually calling (mitmproxy is handy for this on Mac) , my bet is it's hitting a different path or API version string than livyapi/versions/2023-12-01.

In the meantime, since your raw Livy calls are working perfectly, just use HttpClient directly with AzureCliCredential , you get the exact same result without the driver getting in the way. Not ideal, but it unblocks you while this gets sorted.




Worth raising with Microsoft too , Trial + Australia region + a v1.0.0 preview driver is a pretty specific combo and this could just be a staged rollout gap rather than anything you're doing wrong.

vensyd

Thanks for the detailed explanation.

Tried again on F16 and finally got to the bottom of it. Used mitmproxy to capture exactly what URL the driver is hitting internally, and the root cause is a URL casing bug in the driver itself.

The driver v1.0.0 is calling:
POST .../lakehouses/{id}/livyApi/versions/2023-12-01/sessions

The correct Fabric API path is:
POST .../lakehouses/{id}/livyapi/versions/2023-12-01/sessions

One character difference -- the driver constructs the URL with livyApi (capital A) but the Fabric REST API is case-sensitive and only resolves livyapi (all lowercase). That's why it returns 404 every time, regardless of whether your workspace ID, lakehouse ID, auth, or capacity are all correct.

To confirm this isn't a config issue, I built a pre-flight checker that runs four checks before the driver attempts to connect -- workspace lookup, capacity check, lakehouse type verification, and a raw Livy API call. All four passed on F16:

[OK] Workspace found: 'Fabric-RnD'
[OK] Lakehouse found: 'Bronze' (type: Lakehouse)
[OK] Livy API reachable (HTTP 202) -- test session created and cleaned up
[FAIL] Driver v1.0.0 -- HTTP 404 at CreateFabricSessionAsync

Same workspace, same lakehouse, same token, same machine. Raw Livy API works. Driver doesn't. mitmproxy showed exactly why.

This is a one-character bug in FabricLivyClientImpl.CreateFabricSessionAsync inside the driver. Nothing the user can work around via connection string or config.

Environment:
- Driver: Microsoft.Spark.Livy.AdoNet v1.0.0
- .NET: 8.0.42
- OS: macOS (Apple Silicon)
- Capacity: F16
- Region: Australia
- Auth: AzureCli

Fix needed: change livyApi to livyapi in the driver's internal URL construction.

Hope this helps the team get a patch out for v1.0.1.

Also attaching a mitmproxy traffic capture showing both calls side by side -- the working raw Livy endpoint (HTTP 202) and the driver's internal call (HTTP 404). The casing difference in the URL path is visible in the capture.