Forum Discussion
Data flow refresh when new file is added in Sharepoint folder
Hi all,
I receive a new file every day in a SharePoint folder, and I have created a GenFlow 2 pipeline that takes the SharePoint file and appends the data to a lake house. However, I am stuck on how to trigger the data flow refresh when a new file is added to the SharePoint folder. I cannot use a scheduled refresh because the file can arrive at any time, and if the file is missed, it will append duplicate values if a new file is not added. I have tried online solutions and Copilot suggested writing a notebook, but it did not work in my case.
Is there anything I can do in the data pipeline? Please suggest the correct approach.
nilendraFabric , NandanHegde FabianSchut
Thank you!
Unfortunately, notebook needs to be triggered and there is no event driven trigger for notebook.
Power automate has an inbulit mechanism to trigger on arrival of file in sharepoint
4 Replies
- NandanHegde
Super User
You can use Power automate to trigger something on file arrival within sharepoint :
https://manueltgomes.com/microsoft/sharepoint-when-a-file-is-created-in-a-folder-trigger/
And once triggered, trigger the Data pipeline and in turn dataflow via power automate via API
Hi manishkumar13 ,
Based on your description, try using power automate+data factory. when new files are added to the SharePoint folder, the Azure Data Factory (ADF) data flow is automatically triggered to append the file data to Lakehouse.
For more details, you can refer to below document:
Use SharePoint and Power Automate to build workflows - Power Automate | Microsoft Learn
Parameters - Microsoft Fabric | Microsoft Learn
Best Regards,
Adamk KongIf this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Thanks for the reply,
Can I use notebook code to automate rather than using Power Automate?- NandanHegde
Unfortunately, notebook needs to be triggered and there is no event driven trigger for notebook.
Power automate has an inbulit mechanism to trigger on arrival of file in sharepoint
