Forum Discussion
Solved
External File ingestion
Hi All, I want to ingest external csv from from a different network to fabric, Could you please help me to understand the best way to ingest that data into fabric. any leads would be greatly a...
Hi JibinSebastian ,
Few options suggested for your scenaio:
Use On-Premises Data Gateway
- Install Microsoft On-Premises Data Gateway on your local network.
- Configure Fabric Dataflows (Power Query) or Data Factory pipelines to connect through the gateway.
- This allows Fabric to securely access local files without moving them manually.
2. Scheduled Upload via Automation
- Use Power Automate or a script (PowerShell, Python) to:
- Pick up CSV files from local folders.
- Push them to OneLake or Azure Blob Storage.
- Once in OneLake, Fabric can ingest them into Lakehouse or Dataflow Gen2.
3. Hybrid Approach
- If you have network restrictions, you can:
- Set up a shared folder accessible via gateway.
- Or use SFTP/FTP with a pipeline in Fabric Data Factory.
Let me know if the requirement is different.
If you found this helpful, do like it and accept it as a solution.
You can use a Dataflows Gen2 using the File connector. Whether you will need a gateway to connect to the file will depend on:
you need a gateway if:
- The external CSV file is on a private network (on-premises or VPC) that is not accessible over the public internet.
- You want secure connectivity without opening ports or exposing the network publicly.
No gateway needed if:
- The CSV file is hosted on a publicly accessible endpoint (HTTP, HTTPS).
- The file is in a cloud storage service (Azure Blob, AWS S3, etc.) with public or token-based access.
- You manually upload the file into OneLake.
Hope this helps - please appreciate leaving a Kudos or accepting as a Solution!