Dataflow Gen2 lakehouse connection token expired

[MarkBurgess]

Hi, I have a Dataflow Gen2 that updates a Lakehouse table from an on-prem SQL database. There is a shared connection set up to the destination Lakehouse, but the only option for authentication is OAuth2. We have used a service account so that this isn't tied to one individual's account. The service account is just a standard user account but with the password set to not expire. The Dataflow runs fine for about a week and then fails because the token has expired, so I have to go in to the shared connection once a week and reauthenticate.

If OAuth2 is the only option, what is the production solution to this supposed to look like? I can't continue manually refreshing the OAuth for every lakehouse in perpetuity. 

Thanks 

[v-csrikanth]

Hi @MarkBurgess 
It's been a while since I heard back from you and I wanted to follow up. Have you had a chance to try the solutions that have been offered?
If the issue has been resolved, can you mark the post as resolved? If you're still experiencing challenges, please feel free to let us know and we'll be happy to continue to help!
Looking forward to your reply!

If the above information helps you, please give us a Kudos and marked the Accept as a solution.

Best Regards,
Community Support Team _ C Srikanth.

[v-csrikanth]

Hi @MarkBurgess 
Sorry for the late reponse and innconvenience caused.
Could you please refer the link below so that you can setup the service principle.
https://learn.microsoft.com/en-us/fabric/data-factory/service-principals 

If the above information is helpful, please give us Kudos and mark the response as Accepted as solution.

Best Regards,
Community Support Team _ C Srikanth

[v-csrikanth]

Hi @MarkBurgess 
Sorry for the late response. If you want to use the Service Principal please follow the below stpes to enable.

How to Check If Service Principal is Enabled (Without Being an Admin rights)

1: Try Adding a Service Principal in a Workspace

• Open Power BI Service and go to your Fabric workspace.

• Click on Settings → Access.

• Try adding a Service Principal (by entering the App ID from Azure AD).

• If you get an error or don’t see an option for Apps, it likely means Service Principal authentication is disabled.

2: Check in Dataflow Gen2

• Go to your Dataflow Gen2 connection settings.

• If Service Principal is not available in the authentication dropdown, it might not be enabled for your tenant.

3: Ask Your Fabric Admin to Verify

• Since only Fabric Admins can change Tenant Settings, you can ask your Power BI Admin or Microsoft Fabric Admin to check:

  • Admin Portal → Tenant Settings → Dataflows and Datamarts

  • Look for "Allow Service Principal Authentication" and ensure it is enabled.

If the issue still persists, let us know i a happy to help you.

If the above information helps you, please give us a Kudos and marked the reply Accept as a Solution.

Thanks,
Cheri Srikanth

[MarkBurgess]

HI @v-csrikanth 

Yes I can grant workspace access to service principals.

However when selecting a lakehouse destination in Dataflow Gen2, Organisational Account is the only option. 

Similarly if I create a lakehouse connection directly in the 'Manage Gateways and Connections' screen OAuth2 is the only option. 

I'm a Fabric admin and I can't see a "Allow Service Principal Authentication" option in the tenant settings. However we are using service principals for other connection types. It just doesn't seem to be an option in this case. 

Thanks

Mark

[v-csrikanth]

Hi @MarkBurgess 

Thank you @Akash_Varuna for your valuable insights and helpful suggestions! Your contribution is greatly appreciated.

Among the proposed solutions, using a Service Principal is the best approach for ensuring seamless and secure long-term authentication. It eliminates dependency on user-based authentication and allows automated token renewal, making it an efficient and scalable solution.

Thanks and regards,
Cheri Srikanth.

[Akash_Varuna]

Hi @MarkBurgess , could you try these 

• Use Azure AD App Registration: Create an app registration in Azure AD, grant permissions, and use the app's credentials for authentication.
• Service Principal: Switch to service principal authentication for long-term access.
• Automate Renewal: Use a script or tool to refresh the token periodically.
  If this post helped please do give a kudos and accept this as a solution
  Thanks In Advance

[MarkBurgess]

Hi @Akash_Varuna , thanks for the reply. OAuth2 is the only option for Lakehouse connections, so I'm not sure I can do either of the first two options 

How would I go about the third option, would that not still involve refreshing the shared connection with the new token?

Thanks