In my application the edge action send data to SF which ends in the :default stream.
The stream capacity is set to 1000 events. The device runs for several weeks and produced far more than 1000 events (around 90000 at this very moment).
Capacity. This sets the maximum number of events that can be stored within the Stream. The default value is 1,000 and the maximum is 100,000. When the Stream exceeds its capacity of Events, older Events will be deleted automatically. If the capacity is set to 0, no Events are saved; if the capacity is set to 1, the same Event is overwritten every time a new Event occurs.
Nevertheless, I can fetch all 90k events from the stream using the FindEvent API function nicely. Looks like those 89k events weren’t overwritten.
Why is that possible? Do I misunderstand the capacity setting?
The GetStream function returns same ‘1000’ events capacity. Is there a reliable way to get that real amount of events available in the stream?
You are correct in your notes on how the stream capacity should work. I have not tested this in some time. Can you send me the link to the default stream so I can examine it? I will test in my account as well and enter a ticket if needed? Thank you in advance.
I tested adding more events to edge and cloud streams and you are correct that the limits are not being observed. I will enter a ticket to engineering.
glad my work might contribute to improving of Octave API.
The ID of the stream I’m talking about is s6398a3237cb56a6c23186cd0.
Interestingly, yesterday afternoon (CET time) I noticed the ability to get all those 10000s of events from this stream disappeared. If I ask my script to download all events from the stream now I get total 1100 events. Did Sierra do something already?
As an evidence to my previous statements I have a file containing all the unique event IDs of nearly 90k real events which I fetched two days ago when it was still possible.
Thanks for pointing out at CountEvent function, I might try it in future.
Another example of stream capacity mis-behaviour.
Right now I see 110000 events on the stream which supposed to have capacity of 100000.
I’m using CountEvents function to check for real number of events present in the stream and it returns 110000. Then I fetch all these events and they all seem to be unique based on event id. So I conclude it’s a real capacity of 110k events.
The stream Id is s6374f538cf315973d358283e. The stream capacity setting is 100k.
Actually, that’s working as designed. There is a concept of adjusted capacity which is 10% higher than a defined stream capacity. The back-end only delete events that exceed adjusted capacity. This is done to handle the scenario of “event storm” when device can write multiple events at the same second.