Forum Discussion
How to Set Dynamic File Names in a Copy Activity for Lakehouse in Microsoft Fabric?
Hi everyone,
I'm setting up a data pipeline in Microsoft Fabric, where I need to copy an Excel file into a Lakehouse folder every month. The file name should be dynamic to reflect the current year and month (e.g., MonthlyReport_202310.xlsx for October 2023).
I've tried using expressions like:
concat('MonthlyReport_', formatDateTime(utcnow(), 'yyyyMM'), '.xlsx')
in the File name field, but it’s creating a literal folder with the expression text instead of evaluating it dynamically. I also tried defining the concatenated file name in a pipeline parameter and referencing it in the destination path, but I’m still running into issues.
Has anyone successfully configured a dynamic file name in a Copy Activity for Lakehouse? Any tips on how to make this work properly would be greatly appreciated.
Thanks for any help.
Hi HamidBee ,
There is an alternative:
New and changed files can be copied from one lakehouse to another in an incremental manner. It uses “ Filter by Last Modified ” to determine which files to copy.
After completing the steps here, Data Factory scans all files in the source store, applies file filters that are filtered by “ Filter by Last Modified ”, and then copies the new files and files that have been updated since the last time to the target store.
For more information, please refer to:
Best Regards,
Yang
Community Support TeamIf there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!
3 Replies
Hi HamidBee ,
There is an alternative:
New and changed files can be copied from one lakehouse to another in an incremental manner. It uses “ Filter by Last Modified ” to determine which files to copy.
After completing the steps here, Data Factory scans all files in the source store, applies file filters that are filtered by “ Filter by Last Modified ”, and then copies the new files and files that have been updated since the last time to the target store.
For more information, please refer to:
Best Regards,
Yang
Community Support TeamIf there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!Hi HamidBee ,
Is my follow-up just to ask if the problem has been solved?
If so, can you accept the correct answer as a solution or share your solution to help other members find it faster?
Thank you very much for your cooperation!
Best Regards,
Yang
Community Support TeamIf there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!I think you are close, but instead of use the concate() directly in the copy activity, you need to create a variable called 'source file name', then add a Set Variable activity where you can define the variable using Add dynamic content, and input @concat('Monthly Report', utcNow('yyyyMMM'),'.xlsx') .
see my learning notes attached.
