Forum Discussion
DataflowGen2, which does not copy all the data...
I'd still recommend raising the support ticket so an engineer can take a closer look at your scenario and get the traces for it.
Would you happen to have a way in which we can reproduce the issue?
I initially tried to create a Dataflow with the sources you mentioned but was never able to get the same result that you describe. It could be something quite specific about your query and/or your sources, so a deeper investigation through the support route would be the best place to get a much closer look.
Before creating a ticket, I wanted to see if the community had already encountered this problem, as it does not seem specific to my environment.
Yes, to reproduce the problem, I think you just need to:
- Have a source table via the Dataverse connector with many columns (the account table on my system has more than 300 columns)
- Copy the data to a table with the same structure in a lakehouse
With the first code, some rows are lost, while with the second code, all rows are transferred.
In fact, as shown in this screenshot, with the first code, not all rows are transferred because it cannot read all the rows.
It transfers all the lines that are read, but since it cannot read everything, not all of it is transferred.
