topic Re: How to use ClientID and ClientSecrets to create pipelines in Data Engineering

How to use ClientID and ClientSecrets to create pipelines

itsAftab — Mon, 20 Jan 2025 09:38:19 GMT

I am very new to MS Fabric and at the moment testing it out. I have created a few pipelines that uses basic authentication which works fine.

Now i want to create pipelines that uses clientID, clientSecret and Access Token URL. I have searched youtube videos and Google search but unable to find it.

Can someone please help me with a YouTube video or a guide somewhere that i can use to create pipelines/ELT process with clientID, clientSecret and Access Token URL

Re: How to use ClientID and ClientSecrets to create pipelines

v-kpoloju-msft — Tue, 21 Jan 2025 09:10:27 GMT

Hi @itsAftab,

Thank you for reaching out to the Microsoft Fabric Community Forum.

We really apologies for the inconvenience, after reviewing the issue of how to use Client id and Client secrets to create pipelines, here are few steps to may resolve the issue.

Access the Azure portal and navigate to Azure Active Directory. Select App Registrations and click on New Registration. Provide a name for your application and click Register. Once registered, make a note of the Application Client ID and Directory Tenant ID.
In the registered application, navigate to Certificates & Secrets. Click on new client Secret and provide a description. Set an expiration period and click add. Ensure to copy the generated client secret value, as it will be needed later.

To obtain an access token, utilize the client credentials flow. This involves making a POST request to the token endpoint. Below is an example using http:

POST https://login.microsoftonline.com/{tenant-id}/oauth2/v2.0/token

client_id={your-client-id}

grant_type=client_credentials

scope=https://{resource}.default

client_secret={your-client-secret}

Replace ‘tenant-id’, ‘your-client-id’, ‘resource’, and ‘your-client-secret’ with your actual values.

Open ms fabric and navigate to your pipeline. Access the pipeline settings and configure the authentication settings. Choose the option to use client id, client secret, and access token url. Enter the client id, client secret, and the token endpoint url.
Verify the connection to confirm the configuration is correct. Upon successful connection, you should be able to authenticate and access the necessary resources using the provided credentials.
You can now create and run your pipelines using the configured authentication. Please ensure that the pipelines are set up to utilize the client credentials for resource access.

If this post helps, then please give us Kudos and consider Accept it as a solution to help the other members find it more quickly.

Thank you.

Re: How to use ClientID and ClientSecrets to create pipelines

itsAftab — Thu, 23 Jan 2025 21:49:21 GMT

@v-kpoloju-msft

Thank you for the reply

I want to use clientID and clientSecret to retrieve data from an external source. It's a vendor's clientID and clientSecret, not ours

Re: How to use ClientID and ClientSecrets to create pipelines

v-kpoloju-msft — Fri, 24 Jan 2025 07:25:41 GMT

Hi @itsAftab,

Thank you for reaching out to the Microsoft Fabric Community Forum.

We really apologies for the inconvenience, I have gone through the details that you have shared based on how to use client id and client secrets to create pipelines.

Please go through the following steps may resolve the issue:

Kindly refer to the vendor's API documentation for details on the access token URL, the exact payload and headers required for authentication, and the necessary scopes or permissions to access the data.
Generally, you can obtain the access token by sending a POST request to the vendor's token endpoint. Here is a standard example using OAuth 2.0.

POST <vendor's Access Token URL>

Content-Type: application/x-www-form-urlencoded

client_id=<vendor-client-id>

client_secret=<vendor-client-secret>

grant_type=client_credentials

scope=<vendor-specified-scope>

Replace the placeholders <vendor-client-id>, <vendor-client-secret>, etc. with the actual values provided by the vendor.

Prior to integrating this into your MS Fabric pipeline, please test the token generation process using tools such as Postman or cURL. Ensure you can successfully retrieve an access token and utilize it to make authenticated API requests.
In your MS Fabric pipeline, go to the settings for the connection or data source. Enter the vendor's client ID, client secret, and access token URL in the authentication configuration.

If required, set up custom headers or parameters to pass the access token in API requests. For example, use the header:

Authorization: Bearer <access-token>

If the connection fails, check the logs or responses for specific errors e.g., invalid credentials, missing scopes. Ensure the vendors API is accessible from your network.

If this post helps, then please give us Kudos and consider Accept it as a solution to help the other members find it more quickly.

Thank you.

Re: How to use ClientID and ClientSecrets to create pipelines

v-kpoloju-msft — Tue, 28 Jan 2025 08:40:34 GMT

Hi @itsAftab,

May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.

Thank you.

Re: How to use ClientID and ClientSecrets to create pipelines

v-kpoloju-msft — Fri, 31 Jan 2025 17:10:42 GMT

Hi @itsAftab,

I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions. If my response has addressed your query, please accept it as a solution and give a 'Kudos' so other members can easily find it.

Thankyou.

Re: How to use ClientID and ClientSecrets to create pipelines

v-kpoloju-msft — Mon, 03 Feb 2025 16:19:53 GMT

Hi @itsAftab,

I hope this information is helpful. Please let me know if you have any further questions or if you'd like to discuss this further. If this answers your question, please Accept it as a solution and give it a 'Kudos' so others can find it easily.

Thank you.