Duplicated Rows In Tables Built By Notebook

[alozovoy]

I was seeing incorrect numbers after joining two tables together. I have isolated the issue as shown in the following example:

I take the existing delta table named Rainbow_Accounting_Adjustments, and I save it as a dataframe using a temporary view. I then save the dataframe as a delta table named Business_Unit_Adjustment_Totals.

Business_Unit_Adjustment_Totals should be a direct copy of Rainbow_Accounting_Adjustments. Instead, there is a duplicate of each row.

Rainbow_Accounting_Adjustments is updated by Dataflow Gen 2. I have noticed similar problems with other tables created/updated by Dataflow Gen 2. Sometimes it is not an exact duplicate, but shows old data from a previous import.

Any advice on how to resolve this issue would be appreciated!

[JFTxJ]

Hi @alozovoy ,

Any chance you got a resolution on your ticket?  I am having the same issue:  SQL Endpoint is giving me the correct number of rows; but Notebooks return multiple versions of records (started for me yesterday).

Thank you