Forum Discussion

jj44's avatar
jj44
Advocate I
4 months ago
Solved

Copy data activity - reverts to default behaviour

Hi All, Don't know if anyone else has experienced this?   We have a lot of Copy data activities, moving files around our network and Lakehouses from "to be imported" -> Archive etc in multiple p...
  • Lodha_Jaydeep's avatar
    4 months ago

    Hi jj44 ,

    Thanks for reaching fabric community,will happy to assist.

     

    As you mentioined the issue, Yes this is a known and frustrating behaviour in Fabric pipelines. Here's a quick breakdown:

     

    What's Happening

    • The "Delete files after completion" option lives under Source → Advanced (collapsed by default)
    • When you edit a pipeline and save, Fabric can silently reset hidden/advanced settings back to their defaults
    • Since it's tucked away and not visible at a glance, it's very easy to miss during reviews  exactly as you described

    Immediate Fix

    • Open every affected Copy activity → Source tab → expand Advanced
    • Re-check the "Delete files after completion" option
    • Save and validate the pipeline runs as expected before going to PROD

    Recommended Long-Term Workaround Separate Delete Activity

    The recommended workaround is to uncheck the built-in delete option and instead add a separate Delete activity after the Copy activity with the same file mask/path. Microsoft Learn This gives you:

    • Full visibility - delete step is a visible node on the canvas, not hidden in Advanced
    • Immune to the reset bug -  it won't silently disappear on save
    • Safer -  the built-in delete is per-file, so if the copy activity fails midway, some files will already be deleted from source Microsoft Lear a separate Delete activity gives you more control over when deletion happens

    Pattern to use:

    Copy Activity (no delete) → On Success → Delete Activity (same file path/mask)

     

    Prevention Tips

    • Always expand Advanced settings on Copy activities before saving any pipeline edits even unrelated ones
    • Add a pipeline validation checklist for your team covering hidden settings
    • Consider using Git integration for pipelines it lets you diff changes and catch silent resets before they hit PROD
    • Log your pipeline runs enable activity logging so file deletions are traceable if something goes wrong

    Summary

     

    Problem

    Fix

    "Delete files" resets on save

    Re-check & use separate Delete activity

    Setting hidden in Advanced

    Separate Delete activity = visible on canvas

    Silent change caused reprocessing

    Git integration for pipeline change tracking

     

    Definitely worth raising this as a bug/feedback on the Fabric Ideas Forum advanced settings should not silently reset on save.

    Please consider marking this as the Accepted Solution to help other community members find this fix more easily. If this helped you, Kudos are always appreciated!

     

    Hope this helps, Jeff!