Ingest data form a rest API with Oauth2

[MylenePBI]

Hello,

I am trying to ingest data from a rest API using fabric data pipelines, but I am in trouble as oauth2 authentication isn't supported.

Is it possible to know when it will be available?

I ultimatly succeeded in getting data from a web activity (providing manually a token). Is there a way to use this output as a source in the copy activity? I can't setup the source currently due to the oauth2 authentication missing problem, I am trying to find a workaround.

Many thanks for you answers!

[MylenePBI]

Hello, many thanks! I can see the path, but the way to get the token isn't clear (I have client id/client secret, callback uri etc...). Do you have some ressources about it? Is it achievable? I was using a custom connector that I developed before, and I really don't see how to reproduce all the steps here...

[Anonymous]

Hi @MylenePBI ,

You can send your client id, client secret along with the header in web api activity inorder to get the token. Can you confirm how did you get your token initially?

[Anonymous]

Hello @MylenePBI ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .
In case if you have any resolution please do share that same with the community as it can be helpful to others .
Otherwise, will respond back with the more details and we will try to help .

[Anonymous]

Hi @MylenePBI ,

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. Did your issue get resolved? Did my answer helped you?
Please let me know if you have any further queries.

[MylenePBI]

Hello, sorry for the late answer...I still have trouble with ClientId and SecretId, I am not sure where and how to put them in the process.

[Anonymous]

Hi @MylenePBI - Thanks for using Fabric Community,

Inorder to understand how to use client_id and client_secret in your case, please follow below screenshot -



1. Connection - Add your base URL here.
2. Relative URL - Add your relative URL here.
3. Body (Main point) - Can add your client_id and client_secret here in json format

{
    "grant_type":"client_credentials",
    "client_id":"abcdefg",
    "client_secret":"123ghy456"
}

4. Header: Add Content-Type : application/json

Hope this is helpful. In case of any queries please feel free.

[Anonymous]

Hello @MylenePBI ,

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .
In case if you have any resolution please do share that same with the community as it can be helpful to others .
Otherwise, will respond back with the more details and we will try to help .

[MylenePBI]

Hello, I've tried your solution, but I always have the same problem "unsupported media type".

I am not sure what I do is ok. I have several Url:

- 1 base Url 

- 1 authorization URL (base+relative)

- 1 token URL (base+relative)

- 1 URL where I call the data.

Currently I succeed in getting the data when I pass directly the token (that I get from postman) to the call with the last URL.

I have tried to follow what you have said to get the token in fabric data factory: Connect to the base URL, add the token relative URL, add client id, client seret, username and password to the body, add Content-type Json-application in the header, use POST method, but I always have this answer.

Error code

2108

Failure typeUser configuration issue

Details

{"ErrorMessage":"Unsupported Media Type"}

My connexion to the base url is a web2 connexion with anonymous authentication. Could you please help?

[Anonymous]

Hi @MylenePBI ,

The error code 2108 indicates an Unsupported Media Type error. This error occurs when the Content-Type header of the request does not match the media type of the content being sent.

For example, if you are sending JSON data in the body of the request, but the Content-Type header is set to application/x-www-form-urlencoded, the server will reject the request with an Unsupported Media Type error.

To fix this error, you need to make sure that the Content-Type header matches the media type of the content being sent. For example, if you are sending JSON data, the Content-Type header should be set to application/json.

Can you please share few screenshots and details of your implemenation so that I can guide you better?

Note: Try to hide sensitive information in case if any.

Further Queries:
Do you require authorization URL (base+relative) inorder to use token URL (base+relative) and get the token?

As I understand -
 token URL (base+relative) --> token (passing) ---> URL (where I call the data) --> data as response

[Anonymous]

Hi @MylenePBI  - Thanks for using Fabric Community,

As I understand you are trying to copy data from rest api to lakehouse with the help of fabric pipelines.
Current challenge in your case is inorder to invoke rest api you required token ( i.e. 

Bearer token).

As per my understanding we can have a work around to achieve your task.

I am having 2 APIs, API_1 will get a token and other API_2 will get result(date) on passing token along with header.

Lets try to use a Web Activity inorder to get token for us. Please find below screenshot.

Step 1: Use Web Activity in Pipeline.
Step 2: Configure the connection with Base URL.
Step 3: Provide the relative URL of the API (API_1).
Step 4: Pass the json value in body (if applicable)
Step 5: Add Header - Content-Type: application/json inorder to communicate body is a JSON format.

Step 6:  Link a Copy Activity to this Web Activity.
Step 7: Creation a Rest API connection with the base URL.
Step 8: Configure connection type and base url (API_2)

Step 9: !Important Configure your header with Authorization: Bearer <token>

Step 10: !Important (inorder to pass token dynamic way, not manually)
Configure value with add dynamic content  Bearer @{activity('User_Login_Token').output.data.Token}

Note: .data.Token is as per my json output, you can write your expression based on your json output from API_1.

Hope this is helpful. Please feel free incase of any queries.

[PhilippM]

You are a genius! It took my some time to adapt it to my REST API, but I got it to work. This is so nice, thank you very much!

[ThangN]

Hi Everyone, I can't seem to get this working myself did others get it working? I've configued as per below and the Login bit seems fine. I 

The output I am getting from the login doesnt look like the output the previous post has.

I then subsequently get the 401.