Forum Discussion
Scott_Powell
Helper V
3 years agoNewbie help with Pipeline copy from Web API that uses header token
Hi, I'm (completely) new to data pipelines - I come from a Power BI world so I know dataflows not pipelines. I'm trying to use a COPY activity to copy data from a web API. From PowerShell I'm abl...
DavidCousins
3 years agoFrequent Visitor
I wasnt as clear as I could have been, so no worries! I can't add images sadly, but heres some instructions:
- In your pipeline create string Variables called bearer_type and bearer_token and set the values as your "some_api_token" and "hard coded" token, respectively
- In a copy activity for source create a HTTP source against the base URL for your API. In your example it's https://someurl.com/
- Use Anonymous for the authentication of this connection. If this URL isnt publically pingable then this will fail and you won't be able to do this until On-Prem data gateways are enabled for data factory to use.
- In the copy activity set the connection type to REST
- In the Relative URL build whatever string you need. Your example would be:
- /api/nameofapitouse
- Under advanced set the request method to GET
- Create 1 Additional headers
- Header Name = Authorization
- Value (use dynamic content) =
@concat(
variables('bearer_type')
, ' '
,variables('bearer_token')
)
- Now setup your sink
Scott_Powell
Helper V
3 years agoUnfortunately this does indeed fail at step 3. I did see an article posted here https://blog.fabric.microsoft.com/en-us/blog/data-pipelines-tutorial-ingest-files-into-a-lakehouse-from-a-rest-api-with-pagination-ft-aveva-data-hub?ft=All: that might show another way to do this - if so I'll update here.
Thanks!
Scott