Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
2 years ago
Solved

Appending Data to existing file in One Lake using APIs

Is it possible to append data to existing json file using One Lake (ADLS) APIs?  I have been trying that using the standard ADLS APIs and getting the error -  Am I missing something or is it ...
  • Anonymous's avatar
    Anonymous
    2 years ago

    Just wanted everyone to know that we were able to append data to an existing json file in OneLake.

     

    We had a json file under the folder 'files' in a lakehouse.

    We ended up using Azure Storage client library for .NET - https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-directory-file-acl-dotnet

     

    Few things that we ended up doing differently to get things working -

    1. The account name in "https://{accountName}.dfs.core.windows.net" is "onelake"
    2. Instead of using "dfs.core.windows.net" ended up using "dfs.fabric.microsoft.com" making the url - "https://onelake.dfs.fabric.microsoft.com"
    3. the filesystem name was the workspace-id (Not workspace name)
    4. directory path was the lakehouse id (or item id, got it lakehouse>Files>properties(three dots)>URL
    5. sub directory was "Files/" (referring to Files folder in lakehouse as our json was present there). If you want to go inside a subfolder just add it in the sub-directory "Files/example/".

     

    I still wonder why we were not able to achieve this with the APIs as defined in https://learn.microsoft.com/en-us/fabric/onelake/onelake-api-parity