Forum Discussion

dbeavon3's avatar
dbeavon3
Memorable Member
1 year ago
Solved

Redaction issue (need a workaround)

We are migrating workloads from Synapse to Fabric.

 

In Synapse we were able to connect to make round-trips to remote services via an MPE for "private link service" .  We could do this directly from our Synapse notebooks, and it allowed us to reach some custom on-premise web services (REST api)

 

However in Fabric that is not possible, so we must rely on a pipeline copy activity to do a comparable thing. 

Part way thru a notebook, we must pass the control to a "pipeline copy-activity" and then return control to a different notebook again.  The arguments to the copy-activity are sent via the output from a notebook ("notebookutils.notebook.exit").  This exit value is intended to include an access token (eg. for the "Authorization" header to the on-premise web service).

 

But we encounter a problem related to redaction.  When my outputs from the notebook are transmitted by way of "notebookutils.notebook.exit", the access token which is destined for the "copy activity" will become obfuscated.  Within that exit value, it is always producing a redacted literal ("[REDACTED]").   I haven't found an easy way to send the token without the obfuscation.  It seems to happen because the notebook recalls the prior use of "notebookutils", when we had retrieved the secret in the past. 

Here are more details about secret redaction in fabric.
https://learn.microsoft.com/en-us/fabric/data-engineering/author-execute-notebook#secret-redaction

 

 

I am able to avoid the problem by doing silly things like cutting the secret in half or converting to a base64 string:


However I'd rather not do these things, since the entire purpose of the "pipeline copy activity" is to perform a simple transient REST api call.  There is a limited risk in sending the access token between the notebook and the pipeline activity, in order to perform the REST api call.  Sending the access token as an exit value is certainly less risky than removing the validation of the "Authorization" header from this remote REST api.

 

Please let me know if there is a way to get the notebookutils to stop redacting my secrets as they are transmitted between the notebook and pipeline.


In my copy activity, I intend to retrieve the access token via json like so:

Anonymous(activity('Notebook001').output.result.exitValue).authorizationHeader

 

... any tips about a workaround would be greatly appreciated.

 

data engineering
Data Pipeline
  • v-sgandrathi's avatar
    v-sgandrathi
    1 year ago

    Hi dbeavon3,

     

    Thank you for bringing this issue. It seems to be a technical limitation or bug with the notebookutils.notebook.exit functionality in ADF pipelines.

     

    To address this effectively, we recommend raising a support ticket with Microsoft Azure Support. This will allow the engineering team to investigate the issue in detail and provide a resolution or guidance

     

    To raise a support ticket for Fabric and Power BI, kindly follow the steps outlined in the following guide:
    How to create a Fabric and Power BI Support ticket - Power BI | Microsoft Learn

     

    Thank you for being a part of the Microsoft Fabric Community.

8 Replies

  • dbeavon3's avatar
    dbeavon3
    Memorable Member
    1 year ago

    In addition to the workarounds above, another one I found is to use an entirely separate notebook to generate access token and save it to a file.  Once it is in a file, we can read it back in a different notebook (sequentially executed via pipeline).   After jumping thru these hoops, the Fabric notebook environment has no idea that it is a secret anymore, and doesn't try redacting anything. 

    This workaround seems very unfortunate, and quite a bit less secure than chopping up the secret into parts and sending it out of notebookutils.notebook.exit().

  • v-sgandrathi's avatar
    v-sgandrathi
    Community Support
    1 year ago

    Hello dbeavon3 ,
    Hope you are doing well
    Thanks for connecting with the Microsoft Fabric Community Forum.


    Here is the one more workaround which helps in solving your concerns.

    • When you pass secrets using notebookutils.notebook.exit, they get redacted so, Instead of sending the token directly as part of the exit value, it’s better to store it in a secure location and have the pipeline activity fetch it from there.
    • After that store the access token in a Key Vault or a secure storage then in your pipeline, instead of relying on the notebook to pass the token, use the Copy Activity to fetch the token from the Key Vault
    • Instead of trying to pass the access token directly via the notebookutils.notebook.exit, you can have the notebook that store’s the token in the Key Vault, and then the pipeline can retrieve it.
    • Instead of using notebookutils.notebook.exit to pass secrets, you can use secure parameters in your pipeline activity to securely pass sensitive data and create a secure parameter in the pipeline and assign the access token to it.
    • When you use the Copy Activity in the pipeline, reference the secure parameter directly for the Authorization header.

    Hope this method avoids passing sensitive data between the notebook and the pipeline directly, ensuring the token is securely retrieved when needed.

    Please let us know if you have further queries or concerns we are here to assist you.
    Thank you have a great day a head.

    • dbeavon3's avatar
      dbeavon3
      Memorable Member
      1 year ago

      v-sgandrathi 
      Thanks for the tips/feedback.

       

      >> store it in a secure location and have the pipeline activity fetch it from there.

      Yes I found this workaround.   See my other post from yesterday.  It seems unfortunate that the "redaction" functionality, which is intended to promote security,  will actually cause users to pursue workarounds that might be even less secure .  It is an unintended consequence and seems counterproductive, in that it directly conflicts with the original goal.  Storing access tokens out in files seems less secure than if we were to simply pass them (in memory) between a notebook and a pipeline. 

      >> After that store the access token in a Key Vault 
      I didn't think pipelines (in Fabric) had access to key vault via MSI credentials.  I heard that the only resource that pipelines could access via MSI were storage-blob containers.  I will review that assumption, which was only based on my reading and not my first-hand experience.  In any case, it seems excessive to introduce a keyvault dependency just to pass a value from a notebook to a pipeline in Fabric.

      >> you can use secure parameters in your pipeline activity

      You may recall that passing control back and forth between the notebook and pipeline activity was an unfortunate workaround.  As I mentioned, we are only using pipelines because of a limitation in Fabric that prevents notebooks from calling REST api's.  Ideally we reduce the complexity of ADF pipelines over time (or better yet eliminate them).    Anything we can do in notebooks is preferrable to adding complexity to pipelines.

      I believe you have confirmed that sending the temporary access token to another storage location is a reasonable workaround for a Fabric developer.  As a "Fabric data engineer", I find myself taking shortcuts that a standard software engineer would never use.  I don't spend much time in here... but I wanted to check and see if my approach to using the product is not that much different than what others are doing.