Sub-issue of #53.
Background
FusionCore uses fixed R matrices (noise parameters set once in YAML). These parameters cannot adapt to context: GPS noise under open sky vs. under canopy is the same R matrix. The filter reacts to a bad measurement after it arrives via the Mahalanobis gate.
Transformer attention computes dynamic weights based on context. Applied to sensor fusion: instead of fixed R, a small learned attention module reads the last K innovations from each sensor and outputs a dynamic noise weight that anticipates sensor quality changes before they manifest as outliers.
The idea
A small attention module (4 heads, 32-dim embedding, ~50K parameters) runs at 100 Hz alongside the UKF:
- Input: sliding window of last K innovations per sensor (K=20, one vector per timestep)
- Attention score: computed across sensors and time, learned from rosbag data
- Output: per-sensor R matrix scaling factor at each timestep
Key property: attention is proactive. If the last 20 GPS innovations show a rising trend, the attention score drops GPS weight before the Mahalanobis gate fires. Current adaptive noise (adaptive.alpha, adaptive.window) is reactive: it waits for the innovation to already be large.
Why novel
No ROS fusion package uses learned attention for sensor weighting. The connection to the transformer architecture (the dominant paradigm in ML since 2017) makes this a natural bridge between classical state estimation and modern ML. The module is small enough to run at 100 Hz on RPi4.
Implementation notes
- Offline training on rosbag data with known ground truth (NCLT dataset is ideal)
- Runtime inference only: no gradient computation at 100 Hz
- Falls back to fixed R if attention module is not loaded (zero deployment risk)
Parameters
attention.enabled: false
attention.model_path: "" # path to trained .onnx model
attention.window: 20 # innovation history length
Sub-issue of #53.
Background
FusionCore uses fixed R matrices (noise parameters set once in YAML). These parameters cannot adapt to context: GPS noise under open sky vs. under canopy is the same R matrix. The filter reacts to a bad measurement after it arrives via the Mahalanobis gate.
Transformer attention computes dynamic weights based on context. Applied to sensor fusion: instead of fixed R, a small learned attention module reads the last K innovations from each sensor and outputs a dynamic noise weight that anticipates sensor quality changes before they manifest as outliers.
The idea
A small attention module (4 heads, 32-dim embedding, ~50K parameters) runs at 100 Hz alongside the UKF:
Key property: attention is proactive. If the last 20 GPS innovations show a rising trend, the attention score drops GPS weight before the Mahalanobis gate fires. Current adaptive noise (adaptive.alpha, adaptive.window) is reactive: it waits for the innovation to already be large.
Why novel
No ROS fusion package uses learned attention for sensor weighting. The connection to the transformer architecture (the dominant paradigm in ML since 2017) makes this a natural bridge between classical state estimation and modern ML. The module is small enough to run at 100 Hz on RPi4.
Implementation notes
Parameters