Improving object detection by cropping frame

[bit-bat]

First of all, thank you for developing and maintaining Viseron!!

I have been running Viseron (dev version deployed via docker) and Frigate in parallel (using the same hardware for cameras and computer) for the last few weeks, and I have noticed that Viseron often misses events (human detection) compared to Frigate.

I looked at the debug logs and it looks like Viseron detects motion, but doesn't find any objects. I believe this is happening because Viseron resizes and sends the whole frame to object detection, whereas Frigate crops the frame to only include motion area, resizes it and then send it to object detection, which, in most cases, will provide higher resolution for object detection without requiring more resources.

I was wondering if you have looked into cropping the frame before sending it for object detection?

Thanks

[roflcoopter]

Thanks for checking out Viseron and making a comparison!

Are the missed objects very small?
I guess you are running the EdgeTPU detector in Viseron? The other object detectors are a bit more sophisticated and have a "region of interest" step in their models which makes them better at detection in general.

However, i can definitely look into cropping images to the motion area for the EdgeTPU specifically

[bit-bat]

Hi
Thanks for the quick response.

I am not using EdgeTPU (just Darknet), but I realized that there was a little difference in the two setups. I was using substream (640p) in Viseron and mainstream (4K) in Frigate. I am now using 4K in both. I also found events that Viseron missed as the motion did not cross the threshold of 0.08, so I reduced it to 0.01.
I will observe for a few days if there are any differences in events detected and report back.
Sorry for the false alarm.

On a slightly related topic:
Consider a scenario in which there is a parked car in one end of the frame, and motion is detected in the other end of the frame which is not related to any tracked object (let's say, because of rustling of winds). Since we send the whole frame to object detector, does that mean the car will be detected as an object and a recording will trigger? And how does require_motion: true change that behavior?

Thanks!

[roflcoopter]

Sorry i totally missed to reply to this.

Yes that would start the recorder. If you set require_motion: true it means the recording will stop as soon as motion is no longer detected, even tho the car is still in the frame.