Forum Discussion
Copy data from Folder
Hi,
Im trying to use a data pipeline in Fabric to copy data from a folder into my lakehouse. However when I put in my folder address the button for next is greyed out. When I click the link for more help it says that Folder is only available from Dataflow Gen2 for Fabric. I am testong it with a local folder for now, but will eventually be using a network folder. Just wondering will i have to use DataFlow gen2 for this.
Thanks
Hi amymc ,
Thanks for the reply from AndyDDC , his answer works like a charm!
Your problem should be in the gateway.
I used pipeline for copy data to lakehouse and it worked, here are my test steps:
First, select Folder for the Get Data method, then configure it, note that you must have a gateway (because the folder is local) and the gateway must be running, select the appropriate authentication method.
Just choose a lakehouse as your destination.
Then you can click on the Validate and if there are no problems you can Run it.
After running successfully, the effect is as shown below:
You mentioned that you were able to access data from dataflow gen2 without a local data gateway, I'm a bit confused because when I test it if I want to access the local data it reminds me to configure the gateway, like in the picture below:
I would like to know how you are able to access the local Folder without a gateway?
Please configure the gateway and run it and then try again to connect to the local folder using pipeline.
Best Regards,
Yang
Community Support TeamIf there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!
8 Replies
I tried from oneDrive (C:\Users\amy\OneDrive\Documents\Data) and from local ( C:\Users\amy\Documents\Data)
Do you have an "on-premises data gateway" installed on the computer where you're trying to access the local folder? In order to load data from a local computer into Fabric, you need a gateway in order for Fabric it to access it
Hi amymc ,
Thanks for the reply from AndyDDC , his answer works like a charm!
Your problem should be in the gateway.
I used pipeline for copy data to lakehouse and it worked, here are my test steps:
First, select Folder for the Get Data method, then configure it, note that you must have a gateway (because the folder is local) and the gateway must be running, select the appropriate authentication method.
Just choose a lakehouse as your destination.
Then you can click on the Validate and if there are no problems you can Run it.
After running successfully, the effect is as shown below:
You mentioned that you were able to access data from dataflow gen2 without a local data gateway, I'm a bit confused because when I test it if I want to access the local data it reminds me to configure the gateway, like in the picture below:
I would like to know how you are able to access the local Folder without a gateway?
Please configure the gateway and run it and then try again to connect to the local folder using pipeline.
Best Regards,
Yang
Community Support TeamIf there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!Hi,
Thanks for your reply. I checked again and it was a OneDrive folder I was actually acessing from Dataflow. I wasnt able to access this folder from the pipeline through but am from Dataflows.
Your answer and steps are super helpful. Just wondering will I need this a gateway be able to access network folders too.
Thank you
Is there a reason my answer wasn't accepted as the solution?
