Forum Discussion
QUESTION::PIPELINE::COPY DATA::INTEGRITY
I am curious to learn how do any of you guys who are using pipelines and a Copy activity check that the data is actually copied correctly, that is, no missing rows, or offset rows in datetime, and so...
I got this support ticket currently opened: 2406040040000671.
All, beware that ID columns get out of sequence and new numbers seem to be printed by either the 'Copy data' activity or the lakehouse process or...? In any case, new ID numbers appear in the sequence (no gaps) that do not exist at source! So you'd be advised to check your data integrity because if the backend process does whatever it wants with your incoming data, then you have no data integrity guarantees. Go figure that one.