topic Re: Programmatically upload files to Fabric? in Data Engineering

Programmatically upload files to Fabric?

ramonsuarez — Thu, 18 Jan 2024 13:48:01 GMT

Hi,

We want to be able to periodically upload a set of files to fabric, always to the same folder and overwritting the previous versions of those files.

We have a PowerShell script in a server that we can run to extract the data (xml). Then it saves the files to SharePoint, but we would like to save them directly into our LakeHouse without intermediary steps.

I've checked the API documentation and found how to do many operations, but not how to upload files directly to a folder in the Lakehouse.

I've also found that service principals are not supported when using the Microsoft Fabric REST APIs.

Can you help? Thanks!

Re: Programmatically upload files to Fabric?

v-cboorla-msft — Fri, 19 Jan 2024 13:08:03 GMT

Hi @ramonsuarez

Thanks for the ask and using the Microsoft Fabric Community.

At this time, we are reaching out to the internal team to get some help on this .
We will update you once we hear back from them.

Appreciate your patience.

Thanks

Re: Programmatically upload files to Fabric?

v-cboorla-msft — Sat, 20 Jan 2024 07:30:23 GMT

Hi @ramonsuarez

Could you please confirm whether you are using Notebook or Pipeline?

What is the source of your data, will it be accessible from Notebook?

Thanks

Re: Programmatically upload files to Fabric?

ramonsuarez — Mon, 22 Jan 2024 08:29:54 GMT

Neither. The script runs on a server and extracts the tables from the source system as xml files.

Re: Programmatically upload files to Fabric?

v-cboorla-msft — Mon, 22 Jan 2024 14:59:51 GMT

Hi @ramonsuarez

There are multiple ways to upload files to OneLake. You can try these options:

Pipelines (as Sumit Kumar pointed out earlier)
AZCOPY (they call the CLI and it copies files to /Tables directory). There's a good blog covering the details available here. Ingest Data into Microsoft Fabric OneLake using AzCopy | by Inderjit Rana | Microsoft Azure | Medium
APIs

I hope this information helps. Please do let us know if you have any further questions.

Thanks

Re: Programmatically upload files to Fabric?

v-cboorla-msft — Sat, 27 Jan 2024 12:09:47 GMT

Hi @ramonsuarez

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet.
In case if you have any resolution please do share that same with the community as it can be helpful to others.
Otherwise, will respond back with the more details and we will try to help.

Thanks

Re: Programmatically upload files to Fabric?

ramonsuarez — Tue, 30 Jan 2024 09:53:08 GMT

A colleague found a Power Shell script to upload directly from the server and run it with Windows Task Scheduler.

Re: Programmatically upload files to Fabric?

govindarajan_d — Tue, 30 Jan 2024 12:50:20 GMT

Hi @ramonsuarez ,

You can also try installing 'OneLake' program in the server and you can use it just like how you use 'OneDrive'. This approach will require just adding a copy statement in your existing powershell script

Re: Programmatically upload files to Fabric?

RizC1E — Sat, 27 Sep 2025 20:13:13 GMT

Hi @ramonsuarez , would you be able to share the PS script please? Would be very useful.