AI Deck Framerate

[LucasTheIntern]

Hi!

I'm currently working on some code for the AI deck that takes each frame and looks for an illuminated LED. I more or less Frankensteined this code from several of the existing examples, including the WiFi streamer example. I'm not streaming the images anywhere though, just processing them on the GAP8.

I haven't actually measured the framerate yet, but some early results look like I'm getting somewhere around the 60 fps advertised on the camera's spec page: https://store.bitcraze.io/products/ai-deck-monocrome-camera-module

That page contains the following data:

Himax [HM01B0](https://www.himax.com.tw/products/cmos-image-sensor/image-sensors/hm01b0/)-MNA monochrome compact camera module
Ultra low power (ULP) 
Less than 4mW at QVGA 60FPS
Less than 2mW at QVGA 30FPS, 
Less than 1.1mW at QQVGA 30FPS.
High sensitivity 3.6μ BrightSense TM pixel technology
324 x 324 pixel
324 x 244 window mode up to 60FPS
2x2 monochrome binning mode up to 120FPS

60 FPS is proving to be just about fast enough for my application, but if I could bump it up to 120 FPS and lose some resolution, that would be fantastic! Does anyone know how to do this? I see some code related to setting things up, including:

#define IMG_ORIENTATION 0x0101
#define CAM_WIDTH 324
#define CAM_HEIGHT 244

And then later it looks like there's some code referring to "cam_conf" which I assume is a configuration file for the camera stored somewhere in the default firmware or the included packages.

There are two other little things that someone here might have some ideas on (should probably make a separate discussion).

1. I'm currently detecting the LED by just looping through all pixels in the frame and finding four bright ones in a row (to avoid hot pixels). Better ideas? Extending to multiple LED's?
2. I haven't actually measured the framerate yet. I'm thinking I'll just try to measure ticks between frames and print that to the console, but that would make it hard to get a good sample. Maybe average a bunch of them on the AI deck and just print that average instead.

[knmcguire]

Hi! That is great that it's getting that many fps, but as you probably noticed the wifi streamer doesn't reflect that at all. I haven't worked with the aideck lately, but this might also be specific questions that could be rather asked to the Greenwave SDK used on the gap8, at least until those that know more information about this are back from holidays.

[gemenerik]

The mentioned binning results in a QQVGA resolution image as explained in section 3.3 of this datasheet. To enable it, create a camera config with QQVGA mode enabled and open the camera with it, see this example

[gemenerik]

I'm not sure what is the best way to measure and read out the frame rate, but I imagine printing out something every frame will indeed affect results. I think it may be wise to indeed measure over a longer period and then somehow read that out.