Memory usage in ONIX Source

[ChucklesOnGitHub]

It is specified it should never go above 0%, is that right?

[cjsha]

Yeah. It should stay close to zero. Rising memory usage:

• increases the latency between when data is produced and when that data is read from the ONIX hardware by the host computer
• increases risk of hardware buffer overflow which terminates the acquisition session in an error as mentioned in the admonition

The new tune-readsize tutorial provides more depth on this if you're interested.

If there is confusion or uncertainty, is there something we should change in this content or elsewhere in the docs to help?

[ChucklesOnGitHub]

Just from a practical perspective, does the number actually never shift from "0%" during usage (except if it is quickly going up and returning)? The screenshot shows it at 9.55%. If we change to "close to zero", does that mean <10%, 1% or 0.05%. Sorry I guess I could just switch on an ONIX system and see.

[cjsha]

Just from a practical perspective, does the number actually never shift from "0%" during usage (except if it is quickly going up and returning)?

Yeah. For example, PercentUsed might look like this when you're nearing the lower limit of a ReadSize your computer can handle for a given experiment's bandwidth:

although it's spiking a lot, the spikes don't deviate far from zero and always return to zero. Suppose you have a device that is liable to start produce variable amounts of data, you might want to give yourself a little more wiggle room by setting a higher ReadSize. For example, the digital input device is capable of generating frames when events occur instead of at a fixed rate. If you start toggling digital inputs at like 1MHz, then all the sudden ONIX is producing data at a much larger rate and you might risk overflowing the buffer. If you don't care about closed-loop feedback, you can just set ReadSize higher so that you don't have to worry about this at all, in which case PercentUsed might look like this:

Less spiky.

I would say both of these plots show scenarios where the usage of the hardware buffer is totally fine.

If we change to "close to zero", does that mean <10%, 1% or 0.05%?

I'm hesitant to put any upper limit on the memory monitor percentage. You just don't want it climbing. That's going to eventually terminate the acquisition session in error.

With that said, 10% specifically is way too high. The primary reason is that it's a pretty good indication that the hardware buffer is on a trajectory to overflow, but there's another reason as well. Let's say 10% of the buffer is occupied (that's about 200MB) and you're producing no more than 5 MB/s (i.e. the approximate bandwidth of single headstage-64). That means there are 200/5=40 seconds worth of data in the hardware buffer queue. You will end up reading the most recent data 40 seconds or even more after it was created because you first have to read all the previous data that was already in queue before that most-recently produced data becomes available for reading. In other words, a large PercentUsed value indicates that the data you're looking at is "old". This applies to the memory monitor data as well. All this is to explain that when the memory monitor shows that the buffer is 10% occupied, that 10% value is old, and the actual percentage of the hardware buffer that is occupied is probably even higher (but it's not being indicated yet).

Does that help?

[ChucklesOnGitHub]

I think I was fixated with the tooltip providing a given value on hover. Maybe "stay close to zero" gives a better idea that this is dynamic, take a closer look at the datastream if you need, but if it creeps up continuosly it is a problem. Maybe show three cases of the readsize tutorial. I'll give it a go and make a PR.

I did notice a couple things (with 2xnp2.0e and 2048 ReadSize):

• less spiking than in Bonsai

• creeping up quite a lot before coming back down

• the default range was 2000mV, can it be adjusted to 100%? Otherwise it clips (if it does do excursions over 1%)

[ChucklesOnGitHub]

Actually, maybe the approach here shouldn't be the same as with usage through Bonsai.

• Would tuning the ReadSize be applicable in the GUI (since it's not meant for fast closed-loop stuff)? Would it also require changing the audio buffer size?
• If not and this is more fixed, could it detect the connected devices and set the size automatically / provide a warning if it not set to what is recommended in the table ?