topic Re: Ingest data form a rest API with Oauth2 in Pipelines

Ingest data form a rest API with Oauth2

MylenePBI — Wed, 11 Oct 2023 14:39:21 GMT

Hello,

I am trying to ingest data from a rest API using fabric data pipelines, but I am in trouble as oauth2 authentication isn't supported.

Is it possible to know when it will be available?

I ultimatly succeeded in getting data from a web activity (providing manually a token). Is there a way to use this output as a source in the copy activity? I can't setup the source currently due to the oauth2 authentication missing problem, I am trying to find a workaround.

Many thanks for you answers!

Re: Ingest data form a rest API with Oauth2

Anonymous — Wed, 11 Oct 2023 18:45:35 GMT

Hi MylenePBI - Thanks for using Fabric Community,

As I understand you are trying to copy data from rest api to lakehouse with the help of fabric pipelines.
Current challenge in your case is inorder to invoke rest api you required token ( i.e.

Bearer token).

As per my understanding we can have a work around to achieve your task.

I am having 2 APIs, API_1 will get a token and other API_2 will get result(date) on passing token along with header.

Lets try to use a Web Activity inorder to get token for us. Please find below screenshot.

Step 1: Use Web Activity in Pipeline.
Step 2: Configure the connection with Base URL.
Step 3: Provide the relative URL of the API (API_1).
Step 4: Pass the json value in body (if applicable)
Step 5: Add Header - Content-Type: application/json inorder to communicate body is a JSON format.

Step 6: Link a Copy Activity to this Web Activity.
Step 7: Creation a Rest API connection with the base URL.
Step 8: Configure connection type and base url (API_2)

Step 9: !Important Configure your header with Authorization: Bearer <token>

Step 10: !Important (inorder to pass token dynamic way, not manually)
Configure value with add dynamic content Bearer @{activity('User_Login_Token').output.data.Token}

Note: .data.Token is as per my json output, you can write your expression based on your json output from API_1.

Hope this is helpful. Please feel free incase of any queries.

Re: Ingest data form a rest API with Oauth2

MylenePBI — Thu, 12 Oct 2023 09:17:01 GMT

Hello, many thanks! I can see the path, but the way to get the token isn't clear (I have client id/client secret, callback uri etc...). Do you have some ressources about it? Is it achievable? I was using a custom connector that I developed before, and I really don't see how to reproduce all the steps here...

Re: Ingest data form a rest API with Oauth2

Anonymous — Thu, 12 Oct 2023 10:33:25 GMT

Hi MylenePBI ,

You can send your client id, client secret along with the header in web api activity inorder to get the token. Can you confirm how did you get your token initially?

Re: Ingest data form a rest API with Oauth2

Anonymous — Mon, 16 Oct 2023 09:38:06 GMT

Hello MylenePBI ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .
In case if you have any resolution please do share that same with the community as it can be helpful to others .
Otherwise, will respond back with the more details and we will try to help .

Re: Ingest data form a rest API with Oauth2

Anonymous — Wed, 18 Oct 2023 11:56:31 GMT

Hi MylenePBI ,

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. Did your issue get resolved? Did my answer helped you?
Please let me know if you have any further queries.

Re: Ingest data form a rest API with Oauth2

MylenePBI — Wed, 18 Oct 2023 16:35:54 GMT

Hello, sorry for the late answer...I still have trouble with ClientId and SecretId, I am not sure where and how to put them in the process.

Re: Ingest data form a rest API with Oauth2

Anonymous — Thu, 19 Oct 2023 11:55:48 GMT

Hi MylenePBI - Thanks for using Fabric Community,

Inorder to understand how to use client_id and client_secret in your case, please follow below screenshot -

1. Connection - Add your base URL here.
2. Relative URL - Add your relative URL here.
3. Body (Main point) - Can add your client_id and client_secret here in json format

{ "grant_type":"client_credentials", "client_id":"abcdefg", "client_secret":"123ghy456" }

4. Header: Add Content-Type : application/json

Hope this is helpful. In case of any queries please feel free.

Re: Ingest data form a rest API with Oauth2

Anonymous — Fri, 20 Oct 2023 18:27:29 GMT

Hello MylenePBI ,

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .
In case if you have any resolution please do share that same with the community as it can be helpful to others .
Otherwise, will respond back with the more details and we will try to help .

Re: Ingest data form a rest API with Oauth2

MylenePBI — Mon, 23 Oct 2023 09:33:01 GMT

Hello, I've tried your solution, but I always have the same problem "unsupported media type".

I am not sure what I do is ok. I have several Url:

- 1 base Url

- 1 authorization URL (base+relative)

- 1 token URL (base+relative)

- 1 URL where I call the data.

Currently I succeed in getting the data when I pass directly the token (that I get from postman) to the call with the last URL.

I have tried to follow what you have said to get the token in fabric data factory: Connect to the base URL, add the token relative URL, add client id, client seret, username and password to the body, add Content-type Json-application in the header, use POST method, but I always have this answer.

Error code

2108

Failure typeUser configuration issue

Details

{"ErrorMessage":"Unsupported Media Type"}

My connexion to the base url is a web2 connexion with anonymous authentication. Could you please help?

Re: Ingest data form a rest API with Oauth2

Anonymous — Mon, 23 Oct 2023 12:06:18 GMT

Hi MylenePBI ,

The error code 2108 indicates an Unsupported Media Type error. This error occurs when the Content-Type header of the request does not match the media type of the content being sent.

For example, if you are sending JSON data in the body of the request, but the Content-Type header is set to application/x-www-form-urlencoded, the server will reject the request with an Unsupported Media Type error.

To fix this error, you need to make sure that the Content-Type header matches the media type of the content being sent. For example, if you are sending JSON data, the Content-Type header should be set to application/json.

Can you please share few screenshots and details of your implemenation so that I can guide you better?

Note: Try to hide sensitive information in case if any.

Further Queries:
Do you require authorization URL (base+relative) inorder to use token URL (base+relative) and get the token?

As I understand -
token URL (base+relative) --> token (passing) ---> URL (where I call the data) --> data as response

Re: Ingest data form a rest API with Oauth2

ThangN — Thu, 11 Apr 2024 07:14:41 GMT

Hi Everyone, I can't seem to get this working myself did others get it working? I've configued as per below and the Login bit seems fine. I

The output I am getting from the login doesnt look like the output the previous post has.

I then subsequently get the 401.

Re: Ingest data form a rest API with Oauth2

PhilippM — Sat, 25 May 2024 11:00:00 GMT

You are a genius! It took my some time to adapt it to my REST API, but I got it to work. This is so nice, thank you very much!

Re: Ingest data form a rest API with Oauth2

DanielSDavis — Wed, 05 Mar 2025 13:31:38 GMT

What is everyone using for a connection? I'm trying to use the Salesforce API in my instance (I do this BEFORE the normal Salesforce connector so I can get the field descriptions).

Like others, I can't seem to get it to work as expected. I do not want to include raw credentials in the pipeline as this is highly unsecure. But I can't figure out how to setup the "New Connection" properties.

Re: Ingest data form a rest API with Oauth2

mkj1213 — Wed, 24 Sep 2025 09:33:52 GMT

Hello, i can see in your first web activity, youa re still hardcoding the email and password. Is there any way to avoid doing that.

Re: Ingest data form a rest API with Oauth2

mkj1213 — Wed, 24 Sep 2025 09:34:44 GMT

were you able to solve the challenge of not hardcoding the credentials (client_id, client_secret, tenant_id) in your pipeline?