Unable to access shortcuts using spark

I have created shortcuts between same tenant lakehouses using a service principal through the REST API endpoints. I am not able to query the tables using spark using either relative or full abfss paths. For example using a helper function that defines the workspace, lakehouse and table name. I have confirmed that the code works so it's not my helper function that is the problem. 

abfss_path = f"abfss://{workspace_id}@onelake.dfs.fabric.microsoft.com/{lakehouse_id}/Tables/{table_name}"

df = spark.read.format("delta").load(abfss_path)

df_dimlot = load_table("gold", "dimlot")

Instead the error that is raised is this bad request 400. 

Operation failed: "Bad Request", 400, HEAD

http://onelake.dfs.fabric.microsoft.com/85dc7dd8-859e-447b-ac58-cf9ceaea8bc9/0765a6d2-0972-4ee7-a7bb-1e17666b566a/Tables/dimlot/_delta_log?upn=false&action=getStatus&timeout=90s

rId: 0d717e82-201f-0010-7034-568c00000000

The user i am trying to query using spark is contributor on both the workspace that has the original target data and also the workspace where the shortcut is now created. It has direct contributor access on the lakehouses which are the target and destination lakehouses. 

Troubleshooting already done

• Confirmed the consumer DP_Gold lakehouse ID and OneLake path.
• Confirmed dimlot exists as a managed Delta table/shortcut in the consumer lakehouse.
• Confirmed the shortcut points to the central Data Platform Gold lakehouse.
• Compared workspace, direct item, and OneLake role access for my user and the test user.
• Confirmed the failure occurs in an interactive session and that runtime context identifies me as Kasper Pedersen.
• I have not changed or recreated the roles as part of this investigation.

Help requested

1. Please inspect the request ID and explain why OneLake returns HTTP 400 for this user’s HEAD .../_delta_log request.
2. Please confirm which identity and effective permissions Spark/OneLake use when resolving this cross-workspace shortcut in an interactive notebook. Is trusted-service-user from Spark SQL expected in this scenario?
3. Given the configured Contributor access and target Test role, is any additional permission or identity mapping required at the shortcut source or target?
4. Is reading this shortcut through spark.read.format("delta").load(abfss_path) supported for this configuration, or must it be resolved through the lakehouse catalog?