topic Re: API in dataflowgen2 in Dataflow

API in dataflowgen2

Dinesh09 — Fri, 01 Dec 2023 16:45:21 GMT

How to access api data using bearer token in header. In power BI Desktop web connector is available but not in dataflowgen2. Web API connector is available in dataflowgen2 but there is no option in GUI to add headers. Headers is used to enter bearer token. I am not much interested in pasting code advanced editor from power bi desktop as there are bunch of APIs calls I need to made.

Re: API in dataflowgen2

Anonymous — Fri, 01 Dec 2023 17:53:17 GMT

Hi Dinesh09 ,

Thanks for using Fabric Community,

As per my understanding you want to create Rest API Pipeline in Fabric Data Factory.

I assume you might have 2 APIs, API_1 will get a bearer token and other API_2 will get result(final data) on passing bearer token along with header.

Lets assume API_1 requires client_id and client_secret to get bearer token:

Step 1: Connection - Add your base URL here.
Step 2: Relative URL - Add your relative URL here.
Step 3: Body (Main point) - Can add your client_id and client_secret here in json format

{ "grant_type":"client_credentials", "client_id":"abcdefg", "client_secret":"123ghy456" }

Step 4: Header: Add Content-Type : application/json

Lets assume API_2 requires bearer token to get result (final data):

Step 5: Create a New Copy Activity.
Step 6: Link a Copy Activity to this Web Activity.
Step 7: Creation a Rest API connection with the base URL.
Step 8: Configure connection type and base url (API_2)

Step 9: !Important Configure your header with Authorization: Bearer <token>

Step 10: !Important (inorder to pass token dynamic way, not manually)
Configure value with add dynamic content Bearer @{activity('User_Login_Token').output.data.Token}

Note: .data.Token is as per my json output, you can write your expression based on your json output from API_1.

Hope this is helpful. Please let us know incase of any queries.

Re: API in dataflowgen2

Dinesh09 — Sat, 02 Dec 2023 07:23:55 GMT

Thank you Anonymous for the detail explanation on how to use fabric data factory. I am looking to implement the same in the dataflowgen2 also. The bearer token will never change in my case. I believe dataflowgen2 is also perfect choice. The only roadblock I have in dataflowgen2 is how to pass bearertoken.

Re: API in dataflowgen2

miguel — Mon, 04 Dec 2023 15:02:32 GMT

We are working on enabling custom connectors in Dataflow Gen2. Creating a custom connector would be the recommended way to securely store the token and use it for the calls needed to the API. We don't have an ETA that we can share at the moment on when it'll be available.

However, it is technically possible to create your own custom M code to pass the bearer token as clear text and as part of your M code. We do not recommend as the token will not be secure and will simply be part of your M code, but it is technically possible to do this. A sample code is provided below:

let Source = Json.Document(Web.Contents("https://api.XXX.com/", [RelativePath="v1/mycall", Headers=[#"Authorization"="Bearer XXXtokenXXX"]])) in Source

Re: API in dataflowgen2

Anonymous — Mon, 04 Dec 2023 15:19:55 GMT

Hi Dinesh09 ,

Glad to know you query got resolved. Please continue using Fabric Community for your further queries.

Re: API in dataflowgen2

Dinesh09 — Mon, 04 Dec 2023 16:13:27 GMT

Hi Anonymous

Thank you for the M code. In the datapipeline, could you please take me through the mapping part also with screenshots.please take some complex nested json script. In my case collection reference option is not appearing. I choose lakehouse file with parquet format as destination. For some reasons I am not sharing json output of api here.