Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions TRACKING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Tracking Module

This project uses a lightweight yet robust tracking pipeline combining a
constant‑velocity Kalman filter with appearance embeddings and Hungarian
assignment. The goal is to provide stable identifiers for objects and faces
across frames so that higher PSI layers can reason about persistent entities.

## Parameters
Parameters are defined in `config.properties` and loaded into
`TrackingParams`:

| Key | Description | Default |
|-----|-------------|---------|
| `tracking.lambda` | Weight between IoU (1.0) and appearance similarity (0.0). | `0.6` |
| `tracking.iou.min` | Minimum IoU for gating. Pairs with lower IoU *and* low appearance are ignored. | `0.10` |
| `tracking.appearance.minCosine` | Minimum cosine similarity for gating. | `0.50` |
| `tracking.maxAge` | Frames without matches before a track is marked lost. | `30` |
| `tracking.nInit` | Consecutive hits required to confirm a track. | `3` |
| `tracking.smooth.alpha` | Exponential smoothing factor for boxes and embeddings. | `0.2` |

Tuning these values allows balancing responsiveness and stability. For example,
increasing `lambda` favours geometric overlap while reducing it prioritises
appearance features.

## Usage
Both `YoloDetector` and `FaceDetector` instantiate a `TrackManager` and publish
`object.track`/`face.track` events for confirmed tracks. Each event includes the
track identifier, smoothed bounding box, velocity estimate and current state.
These events can be consumed by PSI layers to create symbolic representations
and reason about interactions.
1 change: 0 additions & 1 deletion data/object_memory.json

This file was deleted.

14 changes: 12 additions & 2 deletions src/main/java/dev/bot/zeno/app/Main.java
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
import dev.bot.zeno.overlay.DetectionResult;
import dev.bot.zeno.overlay.DetectionOverlay;
import dev.bot.zeno.util.Config;
import dev.bot.zeno.tracking.TrackingParams;
import io.javalin.Javalin;
import io.javalin.plugin.bundled.CorsPluginConfig;
import io.javalin.websocket.WsConfig;
Expand Down Expand Up @@ -206,12 +207,21 @@ private static void startDetectors(Config cfg,
AtomicReference<Mat> latestFrame,
AtomicReference<DetectionResult> detections,
BlockingQueue<Event> events) {
startDaemonThread("yolo-detector", new YoloDetector(cfg, latestFrame, detections, events, null));
TrackingParams params = new TrackingParams(
cfg.getDouble("tracking.lambda", 0.6),
cfg.getDouble("tracking.iou.min", 0.10),
cfg.getDouble("tracking.appearance.minCosine", 0.50),
cfg.getInt("tracking.maxAge", 30),
cfg.getInt("tracking.nInit", 3),
cfg.getDouble("tracking.smooth.alpha", 0.2)
);

startDaemonThread("yolo-detector", new YoloDetector(cfg, params, latestFrame, detections, events, null));

ArcFaceRecognizer recognizer = cfg.getBool("arcface.enabled", true)
? new ArcFaceRecognizer(cfg) : null;
startDaemonThread("face-detector",
new FaceDetector(cfg, latestFrame, detections, events, recognizer));
new FaceDetector(cfg, params, latestFrame, detections, events, recognizer));
}

/** Periodically overlays detections on frames and calculates FPS for the preview. */
Expand Down
62 changes: 0 additions & 62 deletions src/main/java/dev/bot/zeno/dnn/AppearanceEncoder.java

This file was deleted.

Loading