v2.0.0 - Viseron rewrite

[roflcoopter]

This release features a massive rewrite that I have been working on for the past year.

It focuses on decoupling all parts of Viseron, making it more modularized. This allows for easier integration of new functionality.

The config.yaml file will change completely so there is some work that has to be done by each user to port over to the new version.
The general config format is a component which implements one or more domains.
Each camera has a unique camera identifier which flows through the entire configuration.

The nvr component then ties all these different domains together and provides the full functionality.

The big benefit of this new format is that you can mix and match components more freely.
For instance, you could use different object detectors for different cameras, you are not tied into just one.

Config example

ffmpeg: # <-- component
  camera: # <-- domain
    camera_one: # <-- camera identifier
       name: Camera 1
       host: 192.168.10.10
       username: test
       password: test
    camera_two: # <-- camera identifier
       name: Camera 2
       host: 192.168.10.11
       username: test
       password: test
       ....

darknet: # <-- component
  object_detector: # <-- domain
    model: /my_custom_model/model.weights
    cameras:
      camera_one: # <-- camera identifier
        fps: 5

deepstack:  # <-- component
  host: deepstack # <-- component config option
  port: 5000 # <-- component config option
  object_detector: # <-- domain
    cameras:
      camera_two:  # <-- camera identifier
        fps: 1
        labels:
          - label: person
            confidence: 0.75
            trigger_recorder: false
  face_recognition: # <-- domain
    cameras:
      camera_one:
      camera_two:
    labels:
      - person

background_subtractor: # <-- component
  motion_detector: # <-- domain
    cameras:
      camera_one: # <-- camera identifier
        fps: 1
        mask:
          - coordinates:
              - x: 400
                y: 200
              - x: 1000
                y: 200
              - x: 1000
                y: 750
              - x: 400
                y: 750

nvr: # <-- component
  camera_one: # <-- camera identifier
  camera_two: # <-- camera identifier

Config example with publicly available cameras (used in the screenshots)

ffmpeg:
  camera:
    viseron_camera:
      name: Camera 1
      host: 195.196.36.242
      path: /mjpg/video.mjpg
      port: 80
      stream_format: mjpeg
      fps: 6
      recorder:
        idle_timeout: 1
        codec: h264
    viseron_camera2:
      name: Camera 2
      host: storatorg.halmstad.se
      path: /mjpg/video.mjpg
      stream_format: mjpeg
      port: 443
      fps: 2
      protocol: https
      recorder:
        idle_timeout: 1
        codec: h264
    viseron_camera3:
      name: Camera 3
      host: 195.196.36.242
      path: /mjpg/video.mjpg
      port: 80
      stream_format: mjpeg
      fps: 6
      recorder:
        idle_timeout: 1
        codec: h264

mog2:
  motion_detector:
    cameras:
      viseron_camera:
        fps: 1
      viseron_camera2:
        fps: 1

background_subtractor:
  motion_detector:
    cameras:
      viseron_camera3:
        fps: 1
        mask:
          - coordinates:
              - x: 400
                y: 200
              - x: 1000
                y: 200
              - x: 1000
                y: 750
              - x: 400
                y: 750

darknet:
  object_detector:
    cameras:
      viseron_camera:
        fps: 1
        scan_on_motion_only: false
        labels:
          - label: person
            confidence: 0.8
            trigger_recorder: true
      viseron_camera2:
        fps: 1
        labels:
          - label: person
            confidence: 0.8
            trigger_recorder: true
      viseron_camera3:
        fps: 1
        labels:
          - label: person
            confidence: 0.8
            trigger_recorder: true

nvr:
  viseron_camera:
  viseron_camera2:
  viseron_camera3:

webserver:

mqtt:
  broker: mqtt_broker.lan
  port: 1883
  username: !secret mqtt_username
  password: !secret mqtt_password
  home_assistant:

logger:
  default_level: debug

Frontend

There is now a UI included in Viseron, written in React TypeScript.
It is enabled by default and can be reached on port 8888 inside the container by default.

Currently it features functionality like viewing cameras, recordings and editing the configuration.
It will be expanded upon in the future.

Some screenshots of the UI

Cameras:

Recordings:

Configuration:

Entities:

Jetson Nano support

This release also features better Jetson Nano support through the gstreamer component.
This means that you can utilize your Nano for hardware accelerated camera decoding and object detection.

CompreFace face recognition

Face recognition using CompreFace is now available

Image classification

A new post processor is available, image_classification.
This allows you to get more granular results from a detected object.
As an example, you could detect a specific kind of dog breed, or a specific kind of bird.

Multiprocessing

darknet and edgetpu object detectos utilize multiprocessing which spreads the load between multiple processes.
This allows for better performance, but also makes it easier to analyze and fine tune the load on your system.

Documentation site

The documentation has moved over from being in a README format to a dedicated site.
Hopefully it will help you use Viseron better.
I need your help on how it can be improved, writing good docs is always hard.

I am especially proud of the components page, which is mostly generated from the config schema used.
This means that the documentation and schema will always match, yay!

The documentation is hosted here: https://viseron.netlify.app/

Breaking changes

• ALL kinds of inheritance in the config has been removed.
  This means that you have to explicitly configure your object detector and motion detector settings for each camera.

• interval has been removed from object_detection and motion_detection
  A new config option fps will be used instead.
  This change was made since it was quite confusing, both in the code and for the users because interval was specified in seconds.

• logging has been removed in all shapes and forms and has been replaced with the new component logger
  Please see the updated documentation

• cameras config section has been removed. Camera config is now specified under a component.

• Each object detector has been split up into individual components.
  See the documentation for each detector.

• Each motion detector has been split up into individual components.
  See the documentation for each detector.

• recorder can no longer be configured on a global level.
  It now has to be present under each camera configuration.

• timeout under recorder is now called idle_timeout

• static_mjpeg_streams are now called mjpeg_streams

• enable under object_detection is no more. To disable object detection you simply dont configure it for a camera
  Same goes for motion_detector

• timeout for motion_detector is now called recorder_keepalive

• max_timeout for motion_detector is now called max_recorder_keepalive

• Recordings are now stored in the folder structure /recordings/<camera name>/<date>/<timestamp>.mp4

• Images are no longer streamed to MQTT/Home Assistant, with the exception of thumbnails.
  This is because MQTT is not really suited for this and it impacts performance a lot.
  If you still want to have the cameras in Home Assistant you can point Home Assistant to an MJPEG stream in Viseron.

• filter_args removed for camera and recorder.
  For camera, use video_filters instead.
  For recorder, you can use both video_filters and audio_filters

  Short config example to rotate video 180 degrees:

  ffmpeg:
    camera:
      camera_1:
      ....
        video_filters:   # These filters rotate the images processed by Viseron
          - transpose=2
          - transpose=2
        recorder:
          video_filters:   # These filters rotate the recorded video
            - transpose=2
            - transpose=2

What's Changed

• remove interval and replace with fps by @roflcoopter in remove interval and replace with fps #287
• Update 50-check-if-rpi to support Rpi 4 8gb by @DevelopIdeas in Update 50-check-if-rpi to support Rpi 4 8gb #324
• Add newline after jpgboundary by @olekenneth in Add newline after jpgboundary #355
• Add new line to jpg boundary refactoring by @olekenneth in Add new line to jpg boundary refactoring #356
• Viseron v2 by @roflcoopter in Viseron v2 #306
• Feature/remove dead code by @roflcoopter in Feature/remove dead code #367
• fix automatic pipeline trigger error after rename by @roflcoopter in fix automatic pipeline trigger error after rename #368
• fix jetson nano crossbuilds after balenalib breaking change by @roflcoopter in fix jetson nano crossbuilds after balenalib breaking change #369
• make README more slim by @roflcoopter in make README more slim #370
• fix broken link to contributing guidelines by @roflcoopter in fix broken link to contributing guidelines #371
• update tagline by @roflcoopter in update tagline #372
• add back accidentally removed code by @roflcoopter in add back accidentally removed code #373
• Feature/restart from UI by @roflcoopter in Feature/restart from UI #375
• docs: add redirects to fix 404 by @roflcoopter in docs: add redirects to fix 404 #377
• index pages as well for easier search by @roflcoopter in index pages as well for easier search #378
• scale the non-padded axis by @roflcoopter in scale the non-padded axis #379
• make sure user abc is always in plugdev group by @roflcoopter in make sure user abc is always in plugdev group #380
• update dlib to 19.24 and set compute capabilities by @roflcoopter in update dlib to 19.24 and set compute capabilities #381
• Fix error that was thrown if a domain was setup with multiple missing deps by @roflcoopter in Fix error that was thrown if a domain was setup with multiple missing deps #382
• add compreface component by @roflcoopter in add compreface component #384
• add websocket ping for more stable connections by @roflcoopter in add websocket ping for more stable connections #385
• Feature/warn on missing nvr by @roflcoopter in Feature/warn on missing nvr #386
• add yolov7 models by @roflcoopter in add yolov7 models #387
• Feature/frontend sidebar by @roflcoopter in Feature/frontend sidebar #388
• use multistage builds for models by @roflcoopter in use multistage builds for models #395
• Feature/entities page by @roflcoopter in Feature/entities page #397
• Feature/yolov7 fix by @roflcoopter in Feature/yolov7 fix #398
• fix exception when no cameras are configured by @roflcoopter in fix exception when no cameras are configured #399
• perform preprocessing for quantized classifier models by @roflcoopter in perform preprocessing for quantized classifier models #400
• add list of objects to object detector binary sensors by @roflcoopter in add list of objects to object detector binary sensors #404
• install libpng-dev in jetson nano dlib and base by @roflcoopter in install libpng-dev in jetson nano dlib and base #405
• return zero for negative coordinates by @roflcoopter in return zero for negative coordinates #407
• install more VAAPI drivers by @roflcoopter in install more VAAPI drivers #410
• use similarity threshold to find unknown faces by @roflcoopter in use similarity threshold to find unknown faces #412
• improve accuracy of image classifier by @roflcoopter in improve accuracy of image classifier #413
• include qsv decoders/encoders by @roflcoopter in include qsv decoders/encoders #414
• make chown silent by @roflcoopter in make chown silent #415
• add attributes to recorder entities by @roflcoopter in add attributes to recorder entities #417
• make sure segments are not deleted during concat by @roflcoopter in make sure segments are not deleted during concat #418
• improve CI by @roflcoopter in improve CI #419
• fix cv2 dnn import by @roflcoopter in fix cv2 dnn import #420
• remove deprecated set-output by @roflcoopter in remove deprecated set-output #421
• add output_element config option to gstreamer by @roflcoopter in add output_element config option to gstreamer #424
• switch to mkv for certain audio codecs by @roflcoopter in switch to mkv for certain audio codecs #428
• Viseron v2 by @roflcoopter in Viseron v2 #366
• docs: add example configuration by @roflcoopter in docs: add example configuration #429

New Contributors

• @DevelopIdeas made their first contribution in Update 50-check-if-rpi to support Rpi 4 8gb #324

Full Changelog: v1.10.2...v2.0.0

---

This discussion was created from the release v2.0.0 - Viseron rewrite.

[jamitupya]

Love the work, was checking out some of the ideas on the reddit subs and some cool ideas coming in :-)

[roflcoopter]

Thanks! Yes indeed a lot of great ideas

[jamitupya]

this may be me being stupid, but how can i access the streams inside of home assistant? i get the sensors etc, but i couldn't get the MJPEG integration to access the cameras. i get the thumbnails etc but i couldn't find the cameras. not a huge issue, just a nice to have lol.

[roflcoopter]

They are no longer available over MQTT no, it used a lot of resources and MQTT is not really made for that kind of load.
However you should be able to configure them manually in Home Assistant, is that not working?

[jamitupya]

i had a look through the doco for more details, i assumed it should work with the default MJPEG IP Camera integration? i'm currently using onvif in HA anyway so wasn't a huge issue before, just thought i'd try it out. is there any specific integration i should use?

[jamitupya]

actually speaking of HA, anyone made any cool automations yet?

[FyrbyAdditive]

Am I going to have a bad time if I try to run the Jetson Nano version on an AGX Orin?

[roflcoopter]

Yes it probably wont work since the Jetson Nano image is based on images specifically tailored for the Nano.

Since tha AGX Orin's Jetpack is based on Ubuntu 22.04 it shouldnt be too hard to create an image for it, but i cant justify buying one to do the development sadly