Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
6 years ago

Push Dataset is empty when reached the 200K limit

Hi,   I'm using a Push Dataset which received data from Stream Analytics. The Dataset is created with de default values (basicFIFO). But from the weekend I have some problems, it works well before...
Need Help
lbendlin's avatar
lbendlin
Icon for Super User rankSuper User
6 years ago

yes, there are a number of events that can cause the dataset to flush out. If you need the history then you should use a hybrid streaming dataset instead.

  • Anonymous's avatar
    Anonymous
    Not applicable
    6 years ago

    How can I do it from Stream Analytics?

    I thought this is the type created by default from Stream Analytics, a mix between push and streaming.

      • Anonymous's avatar
        Anonymous
        Not applicable
        6 years ago

        Thanks Anonymous,

         

        I saw theis thread before ask this question. But I think is not my case.

        The retention policy of my Dataset is basicFIFO, I have enabled the 'Historical data analysis', and each record has few information, something like name-value, then I never reach the size limit before reach the 200K records limit.

        Last week the dataset is working as the documentation said, when the 200K records limit is reached, the oldest are deleted but keep information in the dataset.

        But now, the same dataset, with the same configuracion, when reach the 200K limit, delete all data, let the dataset without any record. This is not what I read in the documentation for basicFIFO, and I cannot delete old records of the dataset from Stream Analytics because it is not allowed to write 'delete' in que query.

        I think my dataset has the same configuration, but it seems that it does not work the same as the solution in the other thread.

         

        My dataset receive information every minute from Stream Analytics, and I need to have the last hour of records (around 60K records and size less than 10MB). Deleting all records in the dataset, It's an hour with wrong informacion in the reports.

         

        Regards

        Raul