Forum Discussion

GQ00's avatar
GQ00
Helper III
3 months ago
Solved

credential management best practices?

Hi all,

We have a Microsoft Fabric setup initially implemented by a consultant, including Dataflows Gen2, pipelines, notebooks, Lakehouses, and Warehouses.

Recently, parts of our pipelines started failing.

  • Recreating connections in Manage Connections & Gateways resolved issues for notebooks and pipelines
  • However, Dataflows - to sharepoint and gold WH - were still failing, even after rebuilding and re-authenticating connections 

The error we were seeing stated something along the lines of "Consultants account ENTRA ID expired, have account log in to reactivate" whihc prompt us to reset the consultants credential (it was under our domain), log in into Fabric, and bam, DF resumed working(both ad hoc and via pipelines).

I should clarify, this happened in one workspaced, one capacity.

Consultant user has been inactive and removed from connections 2 months ago. 

 

I'm assumming we will have similar headaches should any owner of a connection leaves the business. A few questions:

Why did things started to failed randomly?

Why did restoring the consultant's user fixed PL authentifications?

What are teh best practices to manage this? Creating a 'service account user'? And if so, would that user be the only one able to create new connections? 

 

Thanks for any guidance, in the meantime I'll continue dwelling in MS learn.

 

  • Hi GQ00 ,

     

    Some connections that use the previous item owner's credentials might stop working if the new item owner doesn't have access to the connection. In such cases, you might see a warning message.

    In this scenario, the new item owner can fix connections by going into the item and replacing the connection with a new or existing connection.

    For Dataflows please consider editing the data source credentials by following this article:

    Data source management - Microsoft Fabric | Microsoft Learn

     

    I hope this information helps. Please do let us know if you have any further queries.
    Thank you

  • GQ00's avatar
    GQ00
    2 months ago

    Hi,

     

    Tried using a dedicated service account,it didnt work for all connections.

     

    Also if the consultants account has been deactivated for months, how come the pipeline was still working? Somehow the token expired only 60 days after last log in, and to my knowledge, I have no way to knowing this... what a way to be blindsighted on a monday morning.

     

    Anyway, we ended up creating a dedicated user account, to own the connections and also allow it to own the assets (DF, PL, NB) while still editing them if needed.

     

    It all seems to be working now, but yet another additional license just to keep things running...

11 Replies

  • trivedisunita's avatar
    trivedisunita
    Continued Contributor

    Hi GQ00 ,

     

    Failures happened because connections were still tied to the consultant’s expired identity. Restoring that account refreshed the tokens. The long‑term fix is to use a dedicated service account for all Fabric connections, document ownership, and audit regularly. That way, you won’t be at the mercy of individual user accounts disappearing.

    Best practices to prevent this--

    1. Use service accounts for connections-

    Create a dedicated Entra ID account (e.g., [email protected]) with appropriate permissions.

    Authenticate all Fabric connections (SharePoint, Lakehouse, Warehouse, etc.) using this account.

    This way, connections don’t depend on personal identities that may leave the company.

    2. Assign workspace roles carefully-

    Make the service account a Contributor or Admin in the workspace so it can own and refresh connections.

    Ensure multiple admins exist so you’re not locked out if one account is disabled.

    3. Document connection ownership-

    Maintain a list of which connections are tied to which identities.

    Periodically audit connections in Manage Connections & Gateways to confirm they’re bound to the service account.

    4. Automate monitoring-

    Use Fabric’s monitoring/logging to detect authentication errors early.

    Consider setting up alerts when pipelines or Dataflows fail due to credential issues.

     

     

    Why did things start failingrandomly?

    • Why did things start failing “randomly”?
      1. Hidden dependency on consultant’s identity: When the consultant originally created connections (to SharePoint, Gold Warehouse, etc.), those connections were bound to their Entra ID credentials. Even if you later re‑authenticated, Fabric sometimes caches or ties the “owner” of the connection to the original identity.

      2. Token expiry: Once the consultant’s account was inactive, the refresh tokens tied to that identity eventually expired. Pipelines and notebooks that were using those tokens stopped working, which explains the delayed/random failures.

      3. Different behavior across artifacts: Pipelines and notebooks could be “rescued” by recreating connections because they were re‑bound to a new identity. Dataflows, however, are more rigid — they often retain the original creator’s identity in metadata, so they kept failing until you reactivated the consultant’s account

     

    Why did restoring the consultant’s user fix it--

    By re‑enabling the consultant’s Entra ID and logging in, Fabric could refresh the expired tokens and validate the connections again. Essentially, you “revived” the identity that the Dataflows were still pointing to, so authentication succeeded.

     

     

    If this helped, a kudos would mean a lot! It encourages contribute and keeps the community strong.

     

    Thank You

    Sunita

     

    • GQ00's avatar
      GQ00
      Helper III

      hi

       

      thanks for your answer

       

      a few follow up questions if you don't mind 

       

      if recreating the connections did not rescue the DFs, how can I validate the new service principal account continues to works when the consultants token experies 

       

      re 3 , how do you audit that connections are bound to the service principal ? this confuses me because again recreating connections for DFs did not fix the issue and in the consultants account I can't see any connections that are exclusive to them , they all include us (admins ) 

      • v-nmadadi-msft's avatar
        v-nmadadi-msft
        Community Support

        Hi GQ00 ,

         

        Some connections that use the previous item owner's credentials might stop working if the new item owner doesn't have access to the connection. In such cases, you might see a warning message.

        In this scenario, the new item owner can fix connections by going into the item and replacing the connection with a new or existing connection.

        For Dataflows please consider editing the data source credentials by following this article:

        Data source management - Microsoft Fabric | Microsoft Learn

         

        I hope this information helps. Please do let us know if you have any further queries.
        Thank you

  • Hi  GQ00  Please check what user is the owner of all the Fabric items the Consultant created. Make sure the owner is an active Entra ID user. I would suggest using a service account. 

    Steps to Take Ownership

    1. Open your workspace and navigate to the Fabric item you need to take over.
    2. Click the ellipsis (...) next to the item name and select Settings.
    3. Go to the About tab and click Take over.
    • GQ00's avatar
      GQ00
      Helper III

      hi thanks for your answer

       

      so after adjusting the connections this was our next port of call. we took ownership of all asset yet data flows continued to failed when run by the pipeline .

       

      • is there anything that would explain that ? is there a way to ensure all assets are own by the newly created service account ? 

      thanks 

  • Hi GQ00

     

    Another gotcha is taking over ownership of a pipeline is not enough to actually change the owner, you need to take over ownership from the settings menu, and then you need to make a change and save the pipeline for the owner to actually change. 

     

    • GQ00's avatar
      GQ00
      Helper III

      W

      O

      W

      that is not intuitive and looks like a bug

       

      I'm keen to try this , however the real test would be after consultants tokens expire which last time took more than a month !

       

      thanks for flagging this , I'm ...wow...

  • Hi GQ00 ,

     

    This behavior usually happens because connections were originally created under a personal/consultant account, so when that identity expired, dependent pipelines and dataflows lost their authentication context. Restoring the consultant’s login temporarily revalidated those credentials, which is why things started working again.  

     

    Best practice is to avoid personal accounts for production connections. Instead, create a dedicated service account (Entra ID user or managed identity) that owns and maintains all Fabric connections. That way, pipelines, dataflows, and notebooks remain stable even if individual users leave the organization. It also centralizes credential management and reduces the risk of random failures.  

     

     

    Thanks,

    Sai Teja 

    • GQ00's avatar
      GQ00
      Helper III

      Hi,

       

      Tried using a dedicated service account,it didnt work for all connections.

       

      Also if the consultants account has been deactivated for months, how come the pipeline was still working? Somehow the token expired only 60 days after last log in, and to my knowledge, I have no way to knowing this... what a way to be blindsighted on a monday morning.

       

      Anyway, we ended up creating a dedicated user account, to own the connections and also allow it to own the assets (DF, PL, NB) while still editing them if needed.

       

      It all seems to be working now, but yet another additional license just to keep things running...

  • Hi GQ00 

    I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions.


    Thank you.

  • Hi GQ00 

    May I check if this issue has been resolved? If not, Please feel free to contact us if you have any further questions.


    Thank you