Forum Discussion
Moving data from S3 to Snowflake
There's no self-heal mechanism in the copy activity — but you've spotted something real. What you're seeing is the staged copy path, and that's where the "healing" impression comes from.
A Lakehouse source (your S3 shortcut) doesn't meet the direct copy criteria for a Snowflake destination, so your copy runs through staging — which matches the two-step behavior you noticed. The service parses your CSV using the source format settings, rewrites the data into a format the Snowflake COPY command accepts, lands it in Fabric's built-in workspace staging, then runs COPY INTO from there.
The important consequence: on this path, Snowflake never reads your original file. It reads the rewritten version the engine produced. Details: https://learn.microsoft.com/en-us/fabric/data-factory/connector-snowflake-copy-activity
The backslash matters because of that parse step. In the DelimitedText format settings, the default Escape character is backslash and the default Quote character is double quote, so a value like "O'Neal" gets interpreted under those rules before the data is rewritten for staging. Reference: https://learn.microsoft.com/en-us/fabric/data-factory/format-delimited-text
Skip incompatible rows does exactly one thing: it detects rows the engine can't reconcile between source and destination, skips them, and counts them — plus writes them to a log file if you enable logging under Settings. It never edits a value.
Three checks will tell you what actually happened.
First, look at that value in Snowflake: if it reads O'Neal without the backslash, the escape character was consumed during the parse, which is why the row loaded.
Second, compare rows read vs rows written vs rows skipped in the copy activity output against a COUNT(*) in Snowflake — read should equal written plus skipped, and if the table holds more than rows written, the extras are leftovers from your earlier failed run (worth a duplicate check, since I don't see a truncate step in your setup).
Third, enable logging and rerun the original file; the log shows exactly which row was skipped and why.
My bet: the value in your table lost its backslash. Let me know which check matches.
🔍Parchitect
Solutions Architect · Microsoft Fabric Specialist
💡Helpful? Kudos are appreciated.
✔️Solved? Mark as Solution so others can find it faster.
Sorry for not providing the full context, I do have a TRUNCATE table in pre-script to truncate table before copy data to target table. Also, source file has 138 rows (including the header); and output of the copy data activity is 138 rows read, 137 rows write and 1 row skipped.
Your two results together close the case. The value came out as O'Neal without the backslash — so the escape character was consumed during the engine's parse, exactly as suspected. And the numbers explain the part you found strange.
Your file has 137 data rows (138 lines minus the header). The activity reports 138 rows read — one more logical row than the file physically contains. That extra row is the leftover: the parser split the O'Neal line into the clean record it could recover (which was written, backslash consumed) plus a malformed fragment with the wrong column count. Skip incompatible rows skipped exactly that fragment. So 138 read = 137 real records + 1 fragment, 137 written, 1 skipped — every actual record landed, and the "skipped row" was never a data record to begin with. Nothing self-healed; the row counter and the data were counting different things.
You can see this black on white: turn on Enable logging under the copy activity's Settings and rerun the original file once — the log captures the skipped row's content, and you should find the fragment there rather than the O'Neal record. One small detail worth a glance while you're at it: if the stored value is literally O'Neal" with a trailing quote, that's the split boundary showing in the data itself.
Going forward, the real fix is agreeing on escape semantics with your provider: either they emit standard CSV (quotes doubled inside quoted fields, no backslash escaping), or you keep the default Escape character = \ deliberately and leave fault tolerance plus logging on, so the next odd line is visible instead of fatal. Your TRUNCATE pre-script already rules out duplicate buildup, so you're covered on that side.
🔍Parchitect
Solutions Architect · Microsoft Fabric Specialist
💡Helpful? Kudos are appreciated.
✔️Solved? Mark as Solution so others can find it faster.