Turn Webcams into Real-Time AI Detectors: glance-vlm speedlab Open Sourced
Yohei Nakajima has open-sourced glance-vlm speedlab, a local VLM harness that skips token generation to run live webcam detectors in 160ms via single forward pa
Yohei Nakajima has open-sourced 'glance-vlm speedlab', a benchmark suite and harness that transforms standard webcam feeds into multiple live, on-device AI detectors—including emotion analysis, object classification, and person counting—at sub-200ms latency.
Image credit: Yohei Nakajima (@yoheinakajima)
Created by open-source developer Yohei Nakajima, the project eliminates slow text generation heads from open vision-language models (VLMs), directly reading logits from a single forward pass to perform real-time Yes/No, multiple-choice (pick-one), and rating classifications.
Token-Free Logit Readout and Sub-200ms Local Inference
Traditional multimodal pipelines struggle with live video streams because generating structured JSON strings requires sequentially decoding dozens of tokens, introducing latencies of several seconds and substantial GPU overhead.
glance-vlm eliminates this autoregressive decoding step entirely. By reading probability distributions directly from the output logits of a frozen open-source VLM, it repurposes the vision model into a high-speed classifier without requiring fine-tuning or retraining.
Live measurements recorded on an Apple M5 chip running a one-question loop demonstrate:
- PyTorch / MPS (FP16): ~210 ms p50 latency.
- MLX 8-bit Quantization: ~160 ms p50 latency, with comprehensive True/False visual queries clocking in at 161 ms.
By default, glance-vlm targets frozen open-weights vision models such as Qwen3-VL-4B (Apache-2.0, downloading approximately 9 GB of open weights on first use). By reading Yes/No, multiple-choice, and rating probabilities directly from a single forward pass without autoregressive token generation, all processing occurs locally without images ever leaving the machine.
Nine-Question Controlled Benchmark and 27.6% Latency Reduction
To rigorously evaluate performance under multi-task workloads, speedlab established a controlled nine-question, fresh-frame benchmark suite.
Across identical test configurations, glance-vlm speedlab reduced p50 latency from 358.5 ms down to 259.6 ms, achieving a 27.6% latency reduction. Importantly, this throughput improvement incurred zero loss in classification fidelity: across all test runs, 84 out of 84 decisions (84/84, 100%) exactly matched baseline reference judgments.
The open-source release on GitHub (yoheinakajima/glance-speedlab) includes data from 21 separate experiments, reproducible benchmark suites, an architectural research summary, and documented failure cases. Community discussions highlighted that reaching sub-200ms local inference enables practical on-device applications, such as low-latency game NPC visual perception and private local monitoring, without cloud dependencies.
Installation Requirements and the Temporal Drift Engineering Caveat
The glance-vlm package is packaged for straightforward local integration:
- Package & License: Distributed under the Apache-2.0 license and installable via
pip install glance-vlm. - Environment: Officially supports Python
>=3.11, <3.12. - Hardware Acceleration: The ~160 ms real-time inference speed relies directly on hardware acceleration on modern chipsets such as Apple Silicon (M5, MLX/MPS).
During release discussions, developers highlighted an important engineering caveat for streaming video pipelines. Developers frequently employ temporal gating to skip inference when consecutive camera frames show little movement. However, comparing incoming frames to the immediately preceding camera frame can create a failure mode:
Under gradual lighting transitions or slow subject movements ('slow drift'), the frame-to-frame delta remains perpetually below the gating threshold, preventing the model from ever refreshing its output. To ensure reliable live webcam detection, temporal gating logic must compare incoming frames against the last scored frame rather than the previous raw frame, guaranteeing responsive updates as cumulative drift occurs.