possible causes of camera overflow?

[khavernathy]

I've got a pretty robust MAX78000 Feather program that takes camera images and does some CNN image classification in a loop.

When I test for durability over many iterations, for some reason, the program fails at about the 500th iteration with the following error (taken from example code from one of the MSDK project examples)

```
//camera_sleep(1);
stat = get_camera_stream_statistic();

if (stat->overflow_count > 0) {
    printf("OVERFLOW DISP = %d\n", stat->overflow_count);
    LED_On(LED2); // Turn on red LED if overflow detected
    while (1) {}
}
```

Can anyone provide clarity or detail on this error and how I might avoid it? If I have overflow, can I flush out the camera memory buffer or something and just call it a failed capture and move on with my loop?

Would using camera_sleep() help prevent this error by adding a time delay between captures?

[Jake-Carter]

Hi @khavernathy, the camera overflow counter indicates how many rows of data were missed during the image capture.

If you are using streaming mode, your application firmware must unload and release each stream buffer before the next row is sent by the camera. The camera will keep sending data row-by-row regardless of whether it is received or not. So when you get overflows in streaming mode, this is typically an indicator that there is some delay in your application code that is unloading the camera data. If you're loading the CNN, the CNN inference latency could be the cause.

while (!camera_is_image_rcv()) {
    if ((data = get_camera_stream_buffer()) != NULL) { 
        // ^ The stream buffer will return 'NULL' until an image row is received.

        // ----
        // Unload row.  Excess time spent here can cause overflow errors.
        // ----

        release_camera_stream_buffer();
    }
}

The only way to avoid these overflows is to ensure that your application code meets the row timing requirements imposed by the camera. There are two routes to achieve this.

1. Optimize the application code to reduce processing time as much as possible. When loading the CNN is involved, this can also involve optimizing the CNN inference time.
2. Slow down the camera's master clock (should be used as a last resort, since this reduces image quality and framerate).

Since the overflow seems to occur around every 500th iteration, your application code already seems to relatively well optimized. Usually any major issues show up as immediate overflows. I would suggest just call any overflows a failed capture, and just re-capturing a new image. Log how many failed captures you get across your iterations and let me know how many there are.

camera_sleep() is for putting the camera in low power mode in between captures. Since the overflow issues occur during the capture, any delays between captures won't help here.

[khavernathy]

Thanks @Jake-Carter , this helps a lot.

I think in my case it has to do with writing the camera pixel data to the CNN input data buffer, which is logic I borrowed from the cats_dogs demo.

    // Get image line by line
    for (uint32_t row = 0; row < h; row++) {
        // Wait until camera streaming buffer is full
        while ((data = get_camera_stream_buffer()) == NULL) {
            if (camera_is_image_rcv()) {
                break;
            }
        }

        //LED_Toggle(LED2);
#ifdef BOARD_EVKIT_V1
        j = IMAGE_SIZE_X * 2 - 2; // mirror on display
#else
        j = 0;
#endif
        for (uint32_t k = 0; k < 4 * w; k += 4) {
            // data format: 0x00bbggrr
            r = data[k];
            g = data[k + 1];
            b = data[k + 2];
            //skip k+3

            // change the range from [0,255] to [-128,127] and store in buffer for CNN
            input_0[cnt++] = ((b << 16) | (g << 8) | r) ^ 0x00808080;

#ifdef BOARD_EVKIT_V1
            j -= 2; // mirror on display
#else
            j += 2;
#endif
        }
        //LED_Toggle(LED2);
        // Release stream buffer
        release_camera_stream_buffer();
    }

I might try tinkering with this logic to prevent the error.

How can I safely skip failed captures? I found using continue; in my loop whenever the overflow is detected isn't cutting it. I think I need to clear the camera memory manually when a overflow failure is detected(?).
I'm trying that via memset(camera_mem_location, 0, img_len_bytes); and letting it run overnight.

[Jake-Carter]

One thing I would recommend is doing the byte packing with a union. This is a great technique that lets you declare an overlapping memory region. It avoids a lot of the memory accesses and shifting operations for packing into the uint32_t and can speed things up significantly.

union {
    uint32_t word;
    uint8_t bytes[4];
} u;

// ...

for (uint32_t k = 0; k < 4 * w; k += 4) {
    // data format: 0x00bbggrr
    u.bytes[0] = data[k]; // red
    u.bytes[1] = data[k + 1]; // green
    u.bytes[2] = data[k + 2]; // blue
    //skip k+3
    
    // change the range from [0,255] to [-128,127] and store in buffer for CNN
    input_0[cnt++] = u.word ^ 0x00808080;

// ...

For your program flow, I'd suggest encapsulating your camera capture code in a function if you haven't already. On an overflow error, return an error code from the function. i.e.

int capture_image() {
    // ...

    stat = get_camera_stream_statistic();
    
    if (stat->overflow_count > 0) {
        printf("OVERFLOW DISP = %d\n", stat->overflow_count);
        LED_On(LED2); // Turn on red LED if overflow detected
        return E_FAIL;
    }
    
    return E_NO_ERROR;
}

Then, your top-level program flow could look something like

int error = capture_image();
int attempts = 1;
while(error) {
    error = capture_image();
    attempts++; // can be used to set a max number of attempts if needed
}

load_cnn();
// etc...

[khavernathy]

Thanks much. This helps.