Forum Discussion
Appending Data to existing file in One Lake using APIs
Is it possible to append data to existing json file using One Lake (ADLS) APIs?
I have been trying that using the standard ADLS APIs and getting the error -
Am I missing something or is it not possible at all to append data to existing fil? The file is inside of a lakehouse.
Just wanted everyone to know that we were able to append data to an existing json file in OneLake.
We had a json file under the folder 'files' in a lakehouse.
We ended up using Azure Storage client library for .NET - https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-directory-file-acl-dotnet
Few things that we ended up doing differently to get things working -
- The account name in "https://{accountName}.dfs.core.windows.net" is "onelake"
- Instead of using "dfs.core.windows.net" ended up using "dfs.fabric.microsoft.com" making the url - "https://onelake.dfs.fabric.microsoft.com"
- the filesystem name was the workspace-id (Not workspace name)
- directory path was the lakehouse id (or item id, got it lakehouse>Files>properties(three dots)>URL
- sub directory was "Files/" (referring to Files folder in lakehouse as our json was present there). If you want to go inside a subfolder just add it in the sub-directory "Files/example/".
I still wonder why we were not able to achieve this with the APIs as defined in https://learn.microsoft.com/en-us/fabric/onelake/onelake-api-parity
5 Replies
Just wanted everyone to know that we were able to append data to an existing json file in OneLake.
We had a json file under the folder 'files' in a lakehouse.
We ended up using Azure Storage client library for .NET - https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-directory-file-acl-dotnet
Few things that we ended up doing differently to get things working -
- The account name in "https://{accountName}.dfs.core.windows.net" is "onelake"
- Instead of using "dfs.core.windows.net" ended up using "dfs.fabric.microsoft.com" making the url - "https://onelake.dfs.fabric.microsoft.com"
- the filesystem name was the workspace-id (Not workspace name)
- directory path was the lakehouse id (or item id, got it lakehouse>Files>properties(three dots)>URL
- sub directory was "Files/" (referring to Files folder in lakehouse as our json was present there). If you want to go inside a subfolder just add it in the sub-directory "Files/example/".
I still wonder why we were not able to achieve this with the APIs as defined in https://learn.microsoft.com/en-us/fabric/onelake/onelake-api-parity
Hi Anonymous ,
Glad to know that your query got resolved. Please continue using Fabric Community on your further queries.
Regarding Rest API, the best course of action is to open a support ticket and have our support team take a closer look at it.Please reach out to our support team so they can do a more thorough investigation on why this it is happening: Link
After creating a Support ticket please provide the ticket number as it would help us to track for more information.
Hope this helps. Please let us know if you have any other queries.
Hi Anonymous ,
Thanks for using Fabric Community.
Can you please share the document you are referring in order to perform the above particular operation? If you can provide above details, I can help you better.
I am attaching a doc for your reference - How to Call the OneLake API from Power Automate (No Enterprise App Needed) (linkedin.com)Hello Anonymous ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .
In case if you have any resolution please do share that same with the community as it can be helpful to others .
Otherwise, will respond back with the more details and we will try to help .
Thanks