Skip to content

hardware.md

Latest commit

 

History

History
196 lines (148 loc) · 8.23 KB

hardware.md

File metadata and controls

196 lines (148 loc) · 8.23 KB
Raw

Hardware and model runtimes

Docs · Architecture · Printers & cameras · Hardware · Deployment · API & MCP · Plugins · Troubleshooting

Which image to pull, how PrintGuard picks a model runtime, and how to give it a GPU or NPU.

How much hardware you need

The detector is a compact encoder, not a large model. A Raspberry Pi 4 handles a camera or two, and any modern x86 mini PC handles several. Inference is only one part of the load. Decoding video costs more than classifying it, so frame rate and resolution matter more than raw model throughput.

PrintGuard never needs a fixed frame rate. The scheduler measures what the host can sustain and shares that capacity across the cameras in use, so adding a camera lowers each camera's rate rather than falling behind. See scheduling inference.

Tip

The dashboard's capacity and latency readouts show what your host actually sustains. If capacity sits far above the sum of your cameras' rates, you have headroom for more cameras.

Image variants

Every release publishes three tags. All three carry the same engine and UI, and differ only in the acceleration runtime they bundle.

Tag Platforms Adds Use it when
latest amd64, arm64 Nothing. Smallest download Default choice, including Raspberry Pi 4/5
latest-intel amd64 Intel's current GPU compute runtime You pass --device /dev/dri for an Arc card, or an iGPU from Tiger Lake (11th gen) onwards
latest-nvidia amd64 TensorRT RTX execution provider and the CUDA 12 runtime You have an RTX 30 series or newer and the NVIDIA Container Toolkit

Versioned tags exist alongside them: X.Y.Z, X.Y, and the same three suffixes, for example 2.3.8-intel. Pin X.Y if you want patch updates without surprises.

Note

The Intel GPU compute runtime is roughly 370 MB of compiler and driver libraries that do nothing unless a GPU device is passed in, which is why it lives in its own tag rather than the default image. Intel CPU acceleration through OpenVINO is in the standard amd64 image and needs no extra tag.

Choosing a variant

flowchart TD
    start["Which image?"] --> arch{"Host architecture"}
    arch -- "arm64, e.g. Raspberry Pi" --> std["latest"]
    arch -- "amd64" --> gpu{"Passing a GPU to the container?"}
    gpu -- "No" --> std2["latest<br/>OpenVINO uses the Intel CPU path"]
    gpu -- "Intel Arc, or an iGPU from Tiger Lake on, /dev/dri" --> intel["latest-intel"]
    gpu -- "NVIDIA RTX 30+ with Container Toolkit" --> nvidia["latest-nvidia"]
Loading

macOS and Windows users running the desktop app do not choose a variant, since the app carries the runtimes for its platform.

Model runtimes

Hub and desktop mode carry the model twice, once for each runtime, and pick between them:

Runtime What it is Path used
LiteRT Google's on-device runtime, formerly TensorFlow Lite Optimised CPU
ONNX Runtime Cross-platform runtime with pluggable execution providers The fastest provider available on the host

Automatic is the default. On start, PrintGuard benchmarks both runtimes for concurrent throughput on the machine it is actually running on and keeps the faster one. The choice is logged, so docker logs printguard shows what won and by how much.

The same benchmark also decides how many frames PrintGuard infers at once. It adds workers while each one still pays for itself and stops at the host's real ceiling, which is not the core count. An accelerator serialises on one device, a runtime's Python binding may hold the interpreter lock, and a container may be under a CPU quota. Measuring covers all three, and the result is the workers term the scheduler divides by latency to get capacity.

Local mode is different. The browser runs LiteRT.js in WebAssembly, which is the only option a browser tab has.

Execution providers by platform

ONNX Runtime selects the fastest provider it can use. What is available depends on the platform:

Platform Provider Notes
macOS, desktop app Core ML Uses CPU, GPU and the Neural Engine
Windows 11 24H2 or newer, desktop app Windows ML Installs the certified Intel, NVIDIA, AMD or Qualcomm provider on first launch
Older Windows, desktop app Optimised CPU No provider install
Linux amd64, standard image OpenVINO Intel CPU path out of the box, and the GPU needs latest-intel and /dev/dri
Linux amd64, latest-nvidia TensorRT RTX Needs the NVIDIA Container Toolkit on the host
Linux arm64, standard image Optimised CPU Raspberry Pi 4/5 and similar

If no accelerator is usable, ONNX Runtime falls back to its CPU provider and PrintGuard keeps working.

Intel GPU

Use the Intel image and pass the render device:

docker run -d --name printguard --restart unless-stopped \
  --device /dev/dri \
  -p 8000:8000 -p 8554:8554 \
  -v printguard:/data \
  ghcr.io/oliverbravery/printguard:latest-intel

Compose:

    image: ghcr.io/oliverbravery/printguard:latest-intel
    devices:
      - /dev/dri:/dev/dri

On Unraid, set the repository to ghcr.io/oliverbravery/printguard:latest-intel and add the template's Intel GPU device.

The image carries Intel's own current compute runtime rather than the distribution's, which covers Arc and Battlemage cards and every iGPU from Tiger Lake (11th gen) onwards. Intel provides no current driver for Gen8 to Gen11 graphics, so a pre-Tiger-Lake iGPU has no GPU path and inference stays on the OpenVINO CPU path.

compute in the header names the hardware in use, so it reads intel gpu once the GPU is running the model, and intel cpu while OpenVINO is on the processor. The log lists everything the providers offered at start:

execution providers offer: Intel GPU, Intel CPU

A GPU missing from that line is one the driver never handed over. Check that the device is passed in with --device /dev/dri and that the tag ends in -intel.

NVIDIA GPU

Needs an RTX 30 series card or newer and the NVIDIA Container Toolkit on the host:

docker run -d --name printguard --restart unless-stopped \
  --gpus all \
  -p 8000:8000 -p 8554:8554 \
  -v printguard:/data \
  ghcr.io/oliverbravery/printguard:latest-nvidia

Compose:

    image: ghcr.io/oliverbravery/printguard:latest-nvidia
    runtime: nvidia

On Unraid, set the repository to the -nvidia tag and add --runtime=nvidia to Extra Parameters.

The image asks the Container Toolkit for every GPU on the host and carries the CUDA 12 runtime the provider needs, so the toolkit is the only thing to install. To pick one card, set NVIDIA_VISIBLE_DEVICES to its UUID or index. If the toolkit cannot hand the GPU over, PrintGuard logs which provider is unavailable and keeps running on the CPU.

Reading and pinning the runtime

The header's compute readout names the hardware the model is running on, for example intel gpu, nvidia gpu or apple core ml, and clicking it opens the setting. The Advanced tab in Settings offers:

Setting Effect
Automatic Benchmark both runtimes on start and keep the faster
LiteRT Always use LiteRT
ONNX Runtime Always use ONNX Runtime and its best provider

Pinning skips the comparison between runtimes, not the benchmark, so the one you pin is still measured for how many workers it sustains. Pin a runtime when a benchmark result surprises you. If a GPU you expect is not being used, Troubleshooting has the checks.