Forum Discussion
Ingest data form a rest API with Oauth2
Hello,
I am trying to ingest data from a rest API using fabric data pipelines, but I am in trouble as oauth2 authentication isn't supported.
Is it possible to know when it will be available?
I ultimatly succeeded in getting data from a web activity (providing manually a token). Is there a way to use this output as a source in the copy activity? I can't setup the source currently due to the oauth2 authentication missing problem, I am trying to find a workaround.
Many thanks for you answers!
15 Replies
Hi MylenePBI - Thanks for using Fabric Community,
As I understand you are trying to copy data from rest api to lakehouse with the help of fabric pipelines.
Current challenge in your case is inorder to invoke rest api you required token ( i.e.Bearer token).
As per my understanding we can have a work around to achieve your task.
I am having 2 APIs, API_1 will get a token and other API_2 will get result(date) on passing token along with header.
Lets try to use a Web Activity inorder to get token for us. Please find below screenshot.
Step 1: Use Web Activity in Pipeline.
Step 2: Configure the connection with Base URL.
Step 3: Provide the relative URL of the API (API_1).
Step 4: Pass the json value in body (if applicable)
Step 5: Add Header - Content-Type: application/json inorder to communicate body is a JSON format.
Step 6: Link a Copy Activity to this Web Activity.
Step 7: Creation a Rest API connection with the base URL.
Step 8: Configure connection type and base url (API_2)Step 9: !Important Configure your header with Authorization: Bearer <token>
Step 10: !Important (inorder to pass token dynamic way, not manually)
Configure value with add dynamic content Bearer @{activity('User_Login_Token').output.data.Token}
Note: .data.Token is as per my json output, you can write your expression based on your json output from API_1.
Hope this is helpful. Please feel free incase of any queries.Hi Everyone, I can't seem to get this working myself did others get it working? I've configued as per below and the Login bit seems fine. I
The output I am getting from the login doesnt look like the output the previous post has.
I then subsequently get the 401.
You are a genius! It took my some time to adapt it to my REST API, but I got it to work. This is so nice, thank you very much!
Hello, i can see in your first web activity, youa re still hardcoding the email and password. Is there any way to avoid doing that.
Hello, many thanks! I can see the path, but the way to get the token isn't clear (I have client id/client secret, callback uri etc...). Do you have some ressources about it? Is it achievable? I was using a custom connector that I developed before, and I really don't see how to reproduce all the steps here...
Hi MylenePBI ,
You can send your client id, client secret along with the header in web api activity inorder to get the token. Can you confirm how did you get your token initially?Hello MylenePBI ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .
In case if you have any resolution please do share that same with the community as it can be helpful to others .
Otherwise, will respond back with the more details and we will try to help .
What is everyone using for a connection? I'm trying to use the Salesforce API in my instance (I do this BEFORE the normal Salesforce connector so I can get the field descriptions).
Like others, I can't seem to get it to work as expected. I do not want to include raw credentials in the pipeline as this is highly unsecure. But I can't figure out how to setup the "New Connection" properties.were you able to solve the challenge of not hardcoding the credentials (client_id, client_secret, tenant_id) in your pipeline?
