Forum Discussion
Newbie help with Pipeline copy from Web API that uses header token
Hi, I'm (completely) new to data pipelines - I come from a Power BI world so I know dataflows not pipelines.
I'm trying to use a COPY activity to copy data from a web API. From PowerShell I'm able to call the web API using code similar to the following. I'd like to use a COPY activity to do the same.
The issue I'm running in to is creating the connector. The API connector only allows you to choose NONE or BASIC for the authentication types. I see in this post Solved: Web API: Headers in Web-Activity vs. Headers in Co... - Microsoft Fabric Community that it's possible to add in header token info after the connection is created. But I'm unable to create the connection in the first place due to authentication failing because I can't pass in the header token in the "create connection" dialog.
Any advice on how to get this to work?
Thanks,
Scott
p.s. the PowerShell script I use is similar to the following:
$uri = 'https://someurl.com/api/nameofapitouse'
$headers = @{
'some_api_token' = 'HARD CODED token value given to our organization to use'
}
$DataFromAPI = Invoke-RestMethod -Uri $uri -Method Get -Headers $headers
16 Replies
- GraceGu
Microsoft Employee
1st step, get bearer token with a Web Activity.
2nd step, add output from the 1st step into authorization header of copy source connector (source connection is a "dummy" connection when created with anonymous/none authetication type, runtime get authenticated with bearer token header) .
You can refer to How to pass bearer token API in Azure Data Factory - Stack Overflow.
- Scott_Powell
Helper V
Hi GraceGu , I had actually seen (and tried) that Stack Overflow post with no success. I'm unable to create a dummy connection using anonymous/none type - when I try, it fails to authenticate and won't let me continue - so the dummy connection never gets created. Screenshot below.
Since I'm unable to get the dummy connection to save, I'm not able to move forward to the point where I can add the header info. Any suggestions?
Thanks,
Scott
- Scott_Powell
Helper V
p.s. and here's the section where you actually create the source connection. As you can see, until you pick a valid connection, the section where you can enter the additional header info doesn't even show.
Thx,
Scott