Note
This is one of 204 standalone projects, maintained as part of the @thi.ng/umbrella monorepo and anti-framework.
🚀 Please help me to work full-time on these projects by sponsoring me on GitHub. Thank you! ❤️
Naive, lightweight CPU-based dense optical flow implementation. This is a support package for @thi.ng/pixel.
This package is a TypeScript port of an old 2008 Processing project, implementing a homemade Optical Flow approach created for a gestural user interface system for Nokia retail stores.
The algorithm is only single channel and not very scalable (in terms of image resolutions), but provides good & stable results for its intended purposes, and without requiring a massive 12MB dependency like OpenCV. YMMV... Using the default config, a 320x240 frame takes ~10ms to process on a MBA M1 2021.
A short 40 second video demonstration (from 2008), first showing the low-res flow field and averaged movement vector, then adding a 3D cube being rotated via hand movments:
20080729-optical-flow.mp4
The algorithm requires a previous and current frame. The flow field is obtained
by sampling the current frame at a given step distance. For each of these
sample/grid positions a kernel window is being swept/applied to compute the
differences to the previous frame. To compute these difference, the previous
frame is offset multiple times in both X/Y directions in the [-displace, +displace] interval. The kernel computes the summed absolute difference for
each of these displaced window regions and selects the window with the minimum
change. The relative (displacement) position of that minimum is the flow vector
for that cell, which will then be linearly interpolated to apply temporal
smoothing of the field (configurable) and minimize jittering.
BETA - possibly breaking changes forthcoming
Search or submit any issues for this package
yarn add @thi.ng/pixel-flowESM import:
import * as pf from "@thi.ng/pixel-flow";Browser ESM import:
<script type="module" src="https://esm.run/@thi.ng/pixel-flow"></script>For Node.js REPL:
const pf = await import("@thi.ng/pixel-flow");Package sizes (brotli'd, pre-treeshake): ESM: 675 bytes
Note: @thi.ng/api is in most cases a type-only import (not used at runtime)
One project in this repo's /examples directory is using this package:
| Screenshot | Description | Live demo | Source |
|---|---|---|---|
 |
Optical flow analysis of web cam or video inputs | Demo | Source |
TODO
If this project contributes to an academic publication, please cite it as:
@misc{thing-pixel-flow,
title = "@thi.ng/pixel-flow",
author = "Karsten Schmidt",
note = "https://thi.ng/pixel-flow",
year = 2008
}© 2008 - 2025 Karsten Schmidt // Apache License 2.0