Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Get certified in Microsoft Fabric—for free! For a limited time, get a free DP-600 exam voucher to use by the end of 2024. Register now

Reply
VenDaFabricator
Resolver I
Resolver I

Writing Base64 string as file into Onelake /Files/<folder>

I has this existing solution in synapse notebook wherein was getting base64 string from a db table and writing it as a 'File' in Blob Storage (ADLS Gen2), existing code base in notebook is a below:

 

*was using sas token for connection string*

 

# Initialize Azure Blob Service Client
  connection_string = "DefaultEndpointsProtocol=https;AccountName=xxxxx;AccountKey=xxxxxxx;EndpointSuffix=core.windows.net"  # Replace with your connection string
  container_name = "sandpit/Attachments"  # Replace with your container name
  blob_service_client = BlobServiceClient.from_connection_string(connection_string)

  def write_file_to_blob(data, filename):
     blob_client = blob_service_client.get_blob_client(container=container_name,  blob=filename)
     blob_client.upload_blob(data, overwrite=True)

   # UDF to decode base64
   def decode_base64(base64_str):
      return base64.b64decode(base64_str)

   # Register UDF
   decode_udf = udf(decode_base64, BinaryType())

 

and above def's being consumed as :

 

    collected_data = df_with_decoded_data.collect()

# Write each file to blob storage
for row in collected_data:
    write_file_to_blob(row['DecodedData'], row['FinalFileName'])

 

this all works fine.  
Now i wanted to update this solution to write to fabric onelake/workspace/Files/<customFolder>

what sort of credentials are there for onelake to be passed as ... any code example would be great



1 ACCEPTED SOLUTION
VenDaFabricator
Resolver I
Resolver I

I managed to get it functioning because the user account running the notebook had complete access to the /Files directory. Since this was a one-time task, I didn't proceed to integrate it with a Service Principal or Managed Identity.

 

The following code worked well for me:

 

collected_data = df_with_decoded_data.collect()

base_dir = "/lakehouse/default/Files/attachments"  # File API Path
# Ensure the base directory exists
os.makedirs(base_dir, exist_ok=True)

# Write the decoded bytes to the file
for item in collected_data:
    # Construct the filename using AttachmentId and FileName
    filename = item["AttachmentId"] + item["FileName"][-4:]
    # Full path for the file
    file_path = os.path.join(base_dir, filename)

    # Ensure the directory for the file exists (in case the filename includes subdirectories)
    os.makedirs(os.path.dirname(file_path), exist_ok=True)

    # Write the Body content to the file
    with open(file_path, "wb") as file:
        file.write(item["DecodedData"])

 

View solution in original post

4 REPLIES 4
VenDaFabricator
Resolver I
Resolver I

I managed to get it functioning because the user account running the notebook had complete access to the /Files directory. Since this was a one-time task, I didn't proceed to integrate it with a Service Principal or Managed Identity.

 

The following code worked well for me:

 

collected_data = df_with_decoded_data.collect()

base_dir = "/lakehouse/default/Files/attachments"  # File API Path
# Ensure the base directory exists
os.makedirs(base_dir, exist_ok=True)

# Write the decoded bytes to the file
for item in collected_data:
    # Construct the filename using AttachmentId and FileName
    filename = item["AttachmentId"] + item["FileName"][-4:]
    # Full path for the file
    file_path = os.path.join(base_dir, filename)

    # Ensure the directory for the file exists (in case the filename includes subdirectories)
    os.makedirs(os.path.dirname(file_path), exist_ok=True)

    # Write the Body content to the file
    with open(file_path, "wb") as file:
        file.write(item["DecodedData"])

 

Anonymous
Not applicable

Hi @VenDaFabricator ,

Glad to know you got resolution for your query.
Please continue using Fabric Community for your further queries.

Anonymous
Not applicable

Hi @VenDaFabricator ,

Thanks for using Fabric Community.
As I understand you were using Azure Synapse Analytics and would like to write data in One Lake from Azure Synapse Notebooks. In order to acheive this we need to use Fabric Rest API.

You can refer this document for implementation details -
On-Premise Python Code to Load Data to Microsoft Fabric Lakehouse | by Amit Chandak | Medium

Additional References:
Access OneLake with Python - Microsoft Fabric | Microsoft Learn

We can also integrate Fabric Onelake in Azure Synapse Analytics - Integrate OneLake with Azure Synapse Analytics - Microsoft Fabric | Microsoft Learn 

Hope this is helpful. Please let me know incase of further queries.

Anonymous
Not applicable

Hi @VenDaFabricator ,

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .
In case if you have any resolution please do share that same with the community as it can be helpful to others .
Otherwise, will respond back with the more details and we will try to help .

Helpful resources

Announcements
November Carousel

Fabric Community Update - November 2024

Find out what's new and trending in the Fabric Community.

Live Sessions with Fabric DB

Be one of the first to start using Fabric Databases

Starting December 3, join live sessions with database experts and the Fabric product team to learn just how easy it is to get started.

November Update

Fabric Monthly Update - November 2024

Check out the November 2024 Fabric update to learn about new features.

Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount! Early Bird pricing ends December 9th.