Forum Discussion
External File ingestion
Hi All,
I want to ingest external csv from from a different network to fabric, Could you please help me to understand the best way to ingest that data into fabric.
any leads would be greatly appreciated.
best regards,
Jibin Sebastian
Hi JibinSebastian ,
Few options suggested for your scenaio:
Use On-Premises Data Gateway
- Install Microsoft On-Premises Data Gateway on your local network.
- Configure Fabric Dataflows (Power Query) or Data Factory pipelines to connect through the gateway.
- This allows Fabric to securely access local files without moving them manually.
2. Scheduled Upload via Automation
- Use Power Automate or a script (PowerShell, Python) to:
- Pick up CSV files from local folders.
- Push them to OneLake or Azure Blob Storage.
- Once in OneLake, Fabric can ingest them into Lakehouse or Dataflow Gen2.
3. Hybrid Approach
- If you have network restrictions, you can:
- Set up a shared folder accessible via gateway.
- Or use SFTP/FTP with a pipeline in Fabric Data Factory.
Let me know if the requirement is different.
If you found this helpful, do like it and accept it as a solution.
4 Replies
Hi JibinSebastian ,
Few options suggested for your scenaio:
Use On-Premises Data Gateway
- Install Microsoft On-Premises Data Gateway on your local network.
- Configure Fabric Dataflows (Power Query) or Data Factory pipelines to connect through the gateway.
- This allows Fabric to securely access local files without moving them manually.
2. Scheduled Upload via Automation
- Use Power Automate or a script (PowerShell, Python) to:
- Pick up CSV files from local folders.
- Push them to OneLake or Azure Blob Storage.
- Once in OneLake, Fabric can ingest them into Lakehouse or Dataflow Gen2.
3. Hybrid Approach
- If you have network restrictions, you can:
- Set up a shared folder accessible via gateway.
- Or use SFTP/FTP with a pipeline in Fabric Data Factory.
Let me know if the requirement is different.
If you found this helpful, do like it and accept it as a solution.
You can use a Dataflows Gen2 using the File connector. Whether you will need a gateway to connect to the file will depend on:
you need a gateway if:
- The external CSV file is on a private network (on-premises or VPC) that is not accessible over the public internet.
- You want secure connectivity without opening ports or exposing the network publicly.
No gateway needed if:
- The CSV file is hosted on a publicly accessible endpoint (HTTP, HTTPS).
- The file is in a cloud storage service (Azure Blob, AWS S3, etc.) with public or token-based access.
- You manually upload the file into OneLake.
Hope this helps - please appreciate leaving a Kudos or accepting as a Solution!
Please clarify what "different network" means. For example - if these CSV files are on a sharepoint then you can use the new shortcut feature - no need to ingest anything.
Hi JibinSebastian
I wanted to check if you had the opportunity to review the information provided by ati_puri , lbendlin and deborshi_nag . Please feel free to contact us if you have any further questions.
Thank you.
