Forum Discussion
Moving data from S3 to Snowflake
There's no self-heal mechanism in the copy activity — but you've spotted something real. What you're seeing is the staged copy path, and that's where the "healing" impression comes from.
A Lakehouse source (your S3 shortcut) doesn't meet the direct copy criteria for a Snowflake destination, so your copy runs through staging — which matches the two-step behavior you noticed. The service parses your CSV using the source format settings, rewrites the data into a format the Snowflake COPY command accepts, lands it in Fabric's built-in workspace staging, then runs COPY INTO from there.
The important consequence: on this path, Snowflake never reads your original file. It reads the rewritten version the engine produced. Details: https://learn.microsoft.com/en-us/fabric/data-factory/connector-snowflake-copy-activity
The backslash matters because of that parse step. In the DelimitedText format settings, the default Escape character is backslash and the default Quote character is double quote, so a value like "O'Neal" gets interpreted under those rules before the data is rewritten for staging. Reference: https://learn.microsoft.com/en-us/fabric/data-factory/format-delimited-text
Skip incompatible rows does exactly one thing: it detects rows the engine can't reconcile between source and destination, skips them, and counts them — plus writes them to a log file if you enable logging under Settings. It never edits a value.
Three checks will tell you what actually happened.
First, look at that value in Snowflake: if it reads O'Neal without the backslash, the escape character was consumed during the parse, which is why the row loaded.
Second, compare rows read vs rows written vs rows skipped in the copy activity output against a COUNT(*) in Snowflake — read should equal written plus skipped, and if the table holds more than rows written, the extras are leftovers from your earlier failed run (worth a duplicate check, since I don't see a truncate step in your setup).
Third, enable logging and rerun the original file; the log shows exactly which row was skipped and why.
My bet: the value in your table lost its backslash. Let me know which check matches.
🔍Parchitect
Solutions Architect · Microsoft Fabric Specialist
💡Helpful? Kudos are appreciated.
✔️Solved? Mark as Solution so others can find it faster.
Hi,
That's an interesting scenario, and thanks for sharing it—it highlights one of those edge cases that many people don't encounter until they're working with real-world data. You're doing a great job investigating the behavior instead of just accepting that the pipeline succeeded.
From what you've described, this doesn't sound like a "self-heal" mechanism in Copy Data. A few things could explain what you're seeing:
Skip incompatible row is designed to allow the pipeline to continue processing when a row can't be parsed or written, but whether a row is truly skipped depends on where the issue occurs.
Since your flow is S3 → Lakehouse (via Shortcut) → Snowflake, the file may have been read successfully by the Lakehouse engine, and the value O\'Neal may have been interpreted as a valid escaped string before the data was written to Snowflake.
Another possibility is that the warning was generated during parsing, but the connector or underlying engine was still able to normalize or escape the value before the final write. This can happen because different engines (Spark, Fabric, and Snowflake) don't always interpret escape characters in exactly the same way.
I'd recommend comparing the problematic row in the Lakehouse table (or staging data) with the corresponding row in Snowflake to see whether the backslash was preserved, removed, or escaped differently. Also, reviewing the detailed Copy Activity logs may reveal whether the row was actually skipped or simply flagged with a recoverable warning.
Out of curiosity:
Was the CSV using a backslash (\) as the escape character, or was it just a literal character in the field?
Did the Copy Activity metrics report 1 skipped row, or only log a warning without affecting the row counts?
Thanks again for sharing this case—it's a valuable example for the community and could help others facing similar CSV parsing issues.
If you found this explanation helpful, please consider marking it as the Accepted Solution or giving it a Like. It helps others in the community find useful answers more quickly.