Forum Discussion
Advocate IUnauthorized Error in Airflow Task with Microsoft Fabric Plugin (401 Invalid Client)
Hi everyone,
I'm encountering an issue while running an Airflow task that interacts with Microsoft Fabric using the apache_airflow_microsoft_fabric_plugin. The task fails with a 401 Unauthorized error, and the logs indicate the following:
invalid_client: The request body must contain the following parameter: 'client_assertion' or 'client_secret'.
Trace ID: 54844adb-4acd-4720-a99f-8b71ad921100
Correlation ID: fd79265c-dd63-457b-aa31-50ed665fec2d
Timestamp: 2024-10-04 12:18:02Z
It seems the OAuth2 request is missing either a client_assertion or client_secret. I've verified that my Azure app registration has a client secret configured and that my Airflow connection (fabric-airflow-conn) is set up with the correct credentials.
Here's the traceback:
[2024-10-04, 14:18:02 CEST] {base.py:73} INFO - Using connection ID 'fabric-airflow-conn' for task execution.
[2024-10-04, 14:18:02 CEST] {taskinstance.py:1824} ERROR - Task failed with exception
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.8/site-packages/apache_airflow_microsoft_fabric_plugin/hooks/fabric.py", line 132, in _get_token
response.raise_for_status()
File "/home/airflow/.local/lib/python3.8/site-packages/requests/models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://login.microsoftonline.com/117c4e93-3230-40d6-b1cd-57cdee469b2f/oauth2/v2.0/token
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.8/site-packages/apache_airflow_microsoft_fabric_plugin/operators/fabric.py", line 122, in execute
self.location = self.hook.run_fabric_item(
File "/home/airflow/.local/lib/python3.8/site-packages/apache_airflow_microsoft_fabric_plugin/hooks/fabric.py", line 214, in run_fabric_item
headers = self.get_headers()
File "/home/airflow/.local/lib/python3.8/site-packages/apache_airflow_microsoft_fabric_plugin/hooks/fabric.py", line 161, in get_headers
"Authorization": f"Bearer {self._get_token()}",
File "/home/airflow/.local/lib/python3.8/site-packages/apache_airflow_microsoft_fabric_plugin/hooks/fabric.py", line 135, in _get_token
raise AirflowException(msg)
airflow.exceptions.AirflowException: Response: {"error":"invalid_client","error_description":"AADSTS7000218: The request body must contain the following parameter: 'client_assertion' or 'client_secret'. Trace ID: 54844adb-4acd-4720-a99f-8b71ad921100 Correlation ID: fd79265c-dd63-457b-aa31-50ed665fec2d Timestamp: 2024-10-04 12:18:02Z","error_codes":[7000218],"timestamp":"2024-10-04 12:18:02Z","trace_id":"54844adb-4acd-4720-a99f-8b71ad921100","correlation_id":"fd79265c-dd63-457b-aa31-50ed665fec2d","error_uri":"https://login.microsoftonline.com/error?code=7000218"} Status Code: 401
[2024-10-04, 14:18:02 CEST] {taskinstance.py:1345} INFO - Marking task as FAILED. dag_id=Run_Fabric_Item, task_id=run_fabric_notebook, execution_date=20241004T121800, start_date=20241004T121801, end_date=20241004T121802
[2024-10-04, 14:18:02 CEST] {standard_task_runner.py:104} ERROR - Failed to execute job 104 for task run_fabric_notebook (Response: {"error":"invalid_client","error_description":"AADSTS7000218: The request body must contain the following parameter: 'client_assertion' or 'client_secret'. Trace ID: 54844adb-4acd-4720-a99f-8b71ad921100 Correlation ID: fd79265c-dd63-457b-aa31-50ed665fec2d Timestamp: 2024-10-04 12:18:02Z","error_codes":[7000218],"timestamp":"2024-10-04 12:18:02Z","trace_id":"54844adb-4acd-4720-a99f-8b71ad921100","correlation_id":"fd79265c-dd63-457b-aa31-50ed665fec2d","error_uri":"https://login.microsoftonline.com/error?code=7000218"} Status Code: 401; 50)
[2024-10-04, 14:18:02 CEST] {local_task_job_runner.py:225} INFO - Task exited with return code 1
Has anyone else encountered this issue or have insights into how to resolve it?
Thanks in advance!
3 Replies
Hi nsprudhvi
Here are a few steps you can take to resolve this issue:
The error message suggests that the request body must contain either a client_assertion or client_secret. Make sure these parameters are included in your authentication request.
Verify that your configuration settings for basic authentication are correctly set up. This includes checking your tenant ID and other relevant settings.
You can try to re-generate your tokens according to this document:
Here's a link for your reference:
Best Regards
Zhengdong Xu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.- mattiasdesmet
Resolver II
I came across the same issue as reported above, but couldn't find the answer in these links. I finally resolved it by finding out that the Extra parameter in the Airflow Connector contained (as per documentation on various sites) "client_secret" : "xxxxx" while this should be "clientSecret" : "xxxxxx" Subtle, but time consuming. Even more so because the error message states that "client_secret" is missing.
I wrote a full setup guide on my blog that also explores Semantic model refresh in a DAG : Orchestrate Fabric Data workloads with Airflow - Mattias De Smet
Thank you Mattias, it work for me. I spend a lot of time to find a solution for issu, but like you said "Even more the error message is clear"
Best regards
