Forum Discussion

Dev_SS's avatar
Dev_SS
Regular Visitor
3 years ago

Duplicates streaming dataset

Hi Team,

In streaming dataset whenever I push a data that is already present in the dataset (duplicate), the row is getting deleted or hidden. But the same is visible when I try to show a sum or functions like that. Is there any documentation on how the duplicates are handled in the streaming/push datasets or the logic ?

Also if I wish to have the duplicates is there anay way to do so ?

 

Expecting a reply.

Thanks

"Need Help"
@NeedHelp
documentation
Tutorial Requests
Urgent Help

1 Reply

  • v-zhangti's avatar
    v-zhangti
    Community Support
    3 years ago

    Hi, Dev_SS 

     

    In general, when a duplicate data point is pushed to a streaming dataset, the existing data point with the same timestamp is overwritten with the new data point. This means that the old data point is effectively deleted. However, if you are using a streaming dataset with aggregation enabled, the duplicate data point will be included in the aggregation calculations.

     

    I would recommend checking the documentation for the specific streaming dataset platform you are using for more information on how duplicates are handled. Additionally, you may want to review the documentation for any aggregation functions you are using to better understand how they handle duplicate data points.

     

    Best Regards,

    Community Support Team _Charlotte

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.