Forum Discussion

Idonica's avatar
Idonica
Frequent Visitor
2 years ago
Solved

Data Pipeline FTP Fabric

Hello everyone,

So my company is starting with Fabric with a POC and on the bronze layer i have encountered  issues/limitations that i think it is only solvable only with using notebooks:

 

I have created a data Pipeline with an FTP connection do get data from a server: 51.133.156.1 (it is not the real ip server).

 

I have access to it through azure portal as a linked service

 

But when i try to reach it out the following are happening i don't how to deal:

 

1) I cannot use the Keyvault, the only option i found doable is by using  python notebooks, is this as it is or it will change ? I'm pretty sure Microsoft will not drop the usage of keyvault in fabric

        1.1 should is use linked services and blob's to connect in fabric? this defeats the porpose of unifying all under the same wing.

2) i do not know even how to reach 51.133.156.1, because it asks for the server as a website.

        2.1) I have seen that, since the server is private and not public, fabric data pipeline/ADF cannot reach it, is that true ?

 

Thank you in advance,

 

ID

Need Help
  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi Idonica ,

    Fabric currently does not support KeyVault.  Until that's fully integrated, there's a simple workaround.  As long as you are executing notebook has access to the vault, the below will work.
     

    from trident_token_library_wrapper import PyTridentTokenLibrary as tl

access_token = mssparkutils.credentials.getToken("keyvault")

tl.get_secret_with_token("https://vaultName.vault.azure.net/","secretName",access_token)


    You can also refer this link, for more information.

    Incase if Notebooks work aorund doesn't handle your case.

    Appreciate if you could share the feedback on our feedback channel. Which would be open for the user community to upvote & comment on. This allows our product teams to effectively prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature.

    Hope this is helpful. Please let me know if you have further queries.

     

10 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hi Idonica ,

    Thanks for using Fabric Community.

    Currently Keyvault is not supported in Fabric Data Factory. We can expect this in future.

    Appreciate if you could share the feedback on our feedback channel. Which would be open for the user community to upvote & comment on. This allows our product teams to effectively prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature.

    Inorder to guide you with FTP connection I would like to understand what authentication method does your FTP have? Ideally how do you connect to your FTP endpoint?

     

    Hope this helps. Please let me know if you have any further queries.






    • Idonica's avatar
      Idonica
      Frequent Visitor
      2 years ago

      Hi Anonymous ,
      Thank you for taking the time to assist me,

       

      So on on the ftp side this is what im doing:

      1) 2023 is the port

      2) the connection is basic auth

      3) user name and pass that im using are from keyvault and when i test them on the azure side as a linked services it is sucesfully connect to the FTP.

       

      Hope this is enough,

      ID



      • Anonymous's avatar
        Anonymous
        Not applicable
        2 years ago

        Hi Idonica ,

        Thanks for the response. So you were to connect using Azure Data Factory - FTP Linked Service, but facing issue or unable to connect while using Fabric Data Factory - Copy Activity.