Forum Discussion

Shawn_Eary's avatar
Shawn_Eary
Advocate V
2 years ago
Solved

Import from Gen1 Azure File / Table Storage?

I'm not able to verify right now, but if I understand a YouTube post by WafaStudies, you can use the provider org.apache.hadoop.fs.azurebfs.sas.FixedSASTokenProvider

to read directly from first generation azure storage:
"31. Acccess ADLS Gen2 or Blob Storage using a SAS token in Azure Databricks"
https://youtu.be/_lkN0lTzl60?si=wo5ZVAhw2joaR-Gj&t=416

Unfortunately, I think I tried that in both ASA Spark and MS Fabric and I believe I got an error indicating that 

org.apache.hadoop.fs.azurebfs.sas.FixedSASTokenProvider

doesn't exist.

What is a dirt easy way for me to read from an old deprecated Azure storage account via Microsoft Fabric?

I want to be able to do something like this to get my data out of my old Gen1 storage account:

myDataFrame = spark.read.csv(
   'abfs://[email protected]/data/myDirectoryPath/myFile.csv',
   header=True
)
Data Integration
  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi Shawn_Eary,

    Thanks for using Fabric Community. Apologies for the delay in reply from our side. 
    As I understand you are trying to access a CSV file present in Azure Blob storage. You can do the same by using the following command:

     

     

    import pandas as pd
data = pd.read_csv('SAS URL of your file')
display(data)

     

     

    The Blob SAS Url can be found by right clicking on the Azure portal's blob file that you want to import and selecting Generate SAS. Then, click Generate SAS token and URL button and copy the SAS url to above code in place of blob_sas_url.
    The above code helps in getting the data from the CSV file. I have created a repro of the scenario at my side and I was successful in fetching the data from Azure Blob Storage. I have attached the screenshots for your reference.
    Hope this helps. Do let us know if you have any further queries.

     

     

     

2 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hi Shawn_Eary,

    Thanks for using Fabric Community. Apologies for the delay in reply from our side. 
    As I understand you are trying to access a CSV file present in Azure Blob storage. You can do the same by using the following command:

     

     

    import pandas as pd
data = pd.read_csv('SAS URL of your file')
display(data)

     

     

    The Blob SAS Url can be found by right clicking on the Azure portal's blob file that you want to import and selecting Generate SAS. Then, click Generate SAS token and URL button and copy the SAS url to above code in place of blob_sas_url.
    The above code helps in getting the data from the CSV file. I have created a repro of the scenario at my side and I was successful in fetching the data from Azure Blob Storage. I have attached the screenshots for your reference.
    Hope this helps. Do let us know if you have any further queries.

     

     

     

    • Anonymous's avatar
      Anonymous
      Not applicable
      2 years ago

      Hi Shawn_Eary ,
      Glad to know that your query got resolved. We expect you to keep using this forum for help on your queries.