Home / News / LocateAnything Deep Dive: How Parallel Box Decoding Works, Where Its Limits Are, and How It Compares with YOLO for Heavy Construction Vehicle Detection
News
LocateAnything Deep Dive: How Parallel Box Decoding Works, Where Its Limits Are, and How It Compares with YOLO for Heavy Construction Vehicle Detection
NVIDIA's 2026 open-source LocateAnything-3B pushes vision-language model grounding throughput to 12.7 boxes per second (BPS) on an H100 through Parallel Box Decoding (PBD), setting new state-of-the-art results on LVIS, ScreenSpot-Pro, and other benchmarks. Yet the popular claim that it is "10x faster than YOLO" misreads the context in which that speed is measured. This report starts from the autoregressive bottleneck, dissects PBD's atomic decoding mechanism, its hybrid inference modes, and its
NVIDIA's 2026 open-source LocateAnything-3B pushes vision-language model grounding throughput to 12.7 boxes per second (BPS) on an H100 through Parallel Box Decoding (PBD), setting new state-of-the-art results on LVIS, ScreenSpot-Pro, and other benchmarks. Yet the popular claim that it is "10x faster than YOLO" misreads the context in which that speed is measured. This report starts from the autoregressive bottleneck, dissects PBD's atomic decoding mechanism, its hybrid inference modes, and its real-world behavior on consumer hardware, then compares it with the YOLO family across latency, deployment footprint, and detection paradigm. It concludes with a layered selection framework for construction-site heavy vehicle detection: YOLO as the real-time base detector, LocateAnything as an on-demand semantic side-path.
1. The Core Problem LocateAnything Solves: The Autoregressive Bottleneck of Coordinate Tokens
1.1 The Serialization Trap in Conventional VLM Grounding
The standard approach for vision-language models to perform object detection is to serialize a 2D bounding box (x1, y1, x2, y2) into a sequence of 1D coordinate tokens and let the language decoder generate them one by one autoregressively. Detecting one object takes 4 decoding steps; detecting 100 objects takes 400. The strict temporal dependency of autoregression becomes a hard constraint on inference speed.
A more subtle cost lies in geometric consistency. The four coordinates should be a coupled whole, but generating them token-by-token means the model does not "know" where x2 will ultimately land when it outputs x1. This tends to produce malformed boxes and coordinate drift, with significant precision loss particularly at high IoU thresholds.
1.2 PBD's Approach: Treat the Box as an Atomic Unit
LocateAnything's core innovation, PBD, takes the following route: each bounding box (or point) is treated as a fixed-length atomic unit, and the complete set of coordinates is predicted simultaneously in a single parallel step, rather than serializing arbitrary chunks of coordinate tokens. This design solves two problems at once: parallel decoding unlocks throughput, and atomicity guarantees intra-box geometric coherence.
PBD does not, however, unconditionally replace autoregression. NVIDIA designed a hybrid inference mode: by default it runs Fast Mode (multi-token prediction, MTP) for maximum throughput, but when parallel output shows formatting anomalies or spatial ambiguity, it seamlessly falls back to Slow Mode (next-token prediction, NTP) to decode token by token for stability. This "fast path plus safety net" is the key engineering decision that lets PBD both accelerate and remain reliable in real deployment.
2. Architecture and Training Data
LocateAnything-3B has roughly 3B parameters and consists of three parts: a vision encoder, MoonViT-SO-400M (native resolution up to about 2.5K, preserving fine spatial detail), a language decoder, Qwen2.5-3B-Instruct, and a two-layer MLP projector connecting them. The code is released under Apache 2.0; the weights use the NVIDIA non-commercial research license.
On the data side, LocateAnything-Data contains 138M language queries and 785M bounding boxes, covering general object detection, GUI element grounding, referring expression comprehension, OCR text localization, and point-level localization. The scale and diversity of this data underpin its advantage in open-vocabulary and high-IoU scenarios.
In terms of task coverage, LocateAnything is a unified framework: open-vocabulary and long-tail object detection, dense multi-object detection, phrase and complex referring localization, GUI grounding, OCR and document layout localization, and point-level localization all happen inside a single model. This "one model, many grounding tasks" design positions it closer to a general visual grounding engine than a narrow-domain detector.
3. The Real Performance Picture: SOTA Accuracy and the Context of "Speed"
3.1 Accuracy: A Clear Advantage at High IoU Thresholds
On an H100, LocateAnything-3B reaches 12.7 BPS, roughly 10x Qwen3-VL and about 2.5x Rex-Omni. Key accuracy figures are shown in Table 1.
| Benchmark | Metric | LocateAnything-3B | Rex-Omni-3B | Qwen3-VL-4B |
|---|---|---|---|---|
| LVIS | F1@Mean | 50.7 | 46.9 | 43.5 |
| LVIS | F1@0.95 | 31.1 | 20.7 | 20.0 |
| COCO | F1@Mean | 54.7 | 52.9 | 46.1 |
| M6Doc | F1@Mean | 70.1 | 55.6* | — |
| ScreenSpot-Pro | Avg | 60.3 | — | — |
Table 1. Key benchmark comparison between LocateAnything and comparable VLM grounding models (* Rex-Omni M6Doc value derived from the reported +14.5 gain).
The most telling metric is LVIS F1@0.95: LocateAnything reaches 31.1, more than 50% above Rex-Omni's 20.7. This directly validates PBD's improvement to box geometric consistency—at IoU thresholds that demand strict alignment, the coordinate drift of token-by-token decoding is structurally suppressed.
3.2 The Limits of "Fast": Real Latency on Consumer Hardware
The 12.7 BPS figure holds in the context of an H100 data-center GPU compared against comparable VLM grounding models. Once you leave the H100, the picture changes quickly.
On an RTX 5090 32GB (bf16, SDPA fallback, no flash-attn), PBD still delivers roughly a 2.1–2.2x box throughput gain over autoregression, confirming the acceleration is not entirely dependent on a specialized stack. But two hard constraints emerge:
First, memory. Normal image inference occupies about 7.75 GB. The official processor allows up to 25,600 ViT patches per image; under the SDPA path without flash-attn, a 6016×3384 screenshot produces about 40.6 GB of attention allocation. An H100 80GB can absorb that; a consumer card cannot, so images must be downsampled below 12,288 patches to run within 32GB.
Second, the speed order of magnitude. On an Apple Silicon M2 with 8GB, YOLO26n has a median single-frame latency of about 17.5 ms, while LocateAnything takes 10.6 seconds to process one complete referring expression—roughly three orders of magnitude apart. LocateAnything's memory in that scenario approaches the 8GB ceiling.
In addition, the officially released weights do not support visual-prompt inference out of the box, and the default decoding loop has limited batch support. Engineering convenience is clearly behind YOLO's mature ecosystem.
The conclusion is clear: LocateAnything's "speed" is relative speed within the VLM grounding race, not industrial-grade real-time detection speed. It and YOLO do not live in the same speed coordinate system.

4. Structured Comparison with YOLO
| Dimension | LocateAnything-3B | YOLO (v8/v11/v26) |
|---|---|---|
| Detection paradigm | Open-vocabulary, language-driven | Closed-set, predefined classes |
| Input | Image + natural language query | Image only |
| Parameter scale | ~3B | 2.5M–59M (n/s/m/l/x) |
| Data-center GPU latency | 12.7 BPS (H100) | YOLO26n FP16 ~17.5 ms/frame (consumer) |
| Edge-level hardware latency | Seconds (M2 8GB, 10.6 s/complex query) | Milliseconds (17.5 ms median) |
| Memory footprint | Large for high-res; ~7.75 GB normal | Very low; nano runs on low-power edge |
| Construction-site ecosystem | Emerging; deployment tooling maturing | Extremely mature; extensive field validation |
| Class changes | Natural language, no retraining | Requires re-annotation + training |
| Dense / high-IoU grounding | Clear advantage (LVIS F1@0.95 31.1) | Mature, but weaker at high-IoU fine granularity |
| Commercial license | Weights: NVIDIA non-commercial research | AGPL/enterprise, clear commercial path |
Table 2. Multi-dimensional structured comparison between LocateAnything and YOLO.
The core message of Table 2 can be distilled into one sentence: YOLO's "fast" is industrial real-time detection fast; LocateAnything's "fast" is vision-language-model internal fast, on data-center GPUs, relative to comparable grounding models. The 1–3 order-of-magnitude latency gap will not be closed by PBD, because their architectural complexity, parameter scale, and design goals belong to different engineering problems.
5. Heavy Construction Vehicle Detection: Selection Guidance
5.1 Split the Scenario First, Not an Either/Or
Construction heavy vehicle detection in practice contains two qualitatively different subtasks:
Subtask A: real-time safety monitoring and dispatch (mainstream, 90%+ of demand). Detecting whether excavators, dump trucks, concrete pump trucks, and truck cranes enter hazard zones or operate in violation of rules. Requires 24/7 video-stream processing, with latency typically in the tens-to-hundreds of milliseconds range; deployment is usually on site edge boxes, NVRs, or low-power GPUs with limited compute and memory. Classes are essentially fixed at project kickoff.
Subtask B: complex semantic retrieval and open-scene review (auxiliary). For example, "find truck cranes currently lifting rebar" or "are any dump trucks leaving without tarps?" Classes cannot be enumerated in advance and rely on compositional semantics and long-tail descriptions. Seconds-level response is acceptable, typically for after-the-fact retrieval, inspection review, or low-frequency analysis.
5.2 Scenario-Based Selection Matrix
| Scenario | Recommendation | Rationale |
|---|---|---|
| Real-time safety monitoring (mainstream) | YOLO (v8s/m, v11, v26 light tiers) | Millisecond latency; YOLO26 reaches mAP@0.5 of 0.938 across three construction vehicle classes (wheel loader 0.978, excavator 0.914, dump truck 0.923) |
| Edge devices / low-power boxes | YOLO | LocateAnything needs ~7.75 GB for normal inference; edge real-time throughput insufficient. YOLO nano runs in real time on low-wattage hardware |
| Fixed classes, best cost-performance | YOLO | Transfer learning + pretrained weights; shortest engineering path; extensive mature construction-site solutions |
| Natural-language long-tail / compositional retrieval | LocateAnything as a supplement | Open-vocabulary and complex referring capability YOLO cannot replace |
| Offline post-hoc video analysis (GPU server available) | LocateAnything viable | Seconds-level latency acceptable for post-hoc analysis; gains open-vocabulary flexibility |
| Commercial closed-source productization | Prefer YOLO | LocateAnything weights are non-commercial research licensed; evaluate authorization |
Table 3. Scenario-based selection matrix for heavy construction vehicle detection.
5.3 Recommended Deployment Path: YOLO as Base, LocateAnything as Semantic Side-Path
Phase 1 (immediately usable). Use YOLO as the real-time detection base, choosing YOLOv8s/m or YOLO26 light tiers, with transfer learning on construction vehicle classes and TensorRT deployment on edge GPUs to handle millisecond-level video-stream alerts. Construction-site engineering practice has demonstrated that the YOLO family is reliable and mature at end-to-end latency under 200 ms and 0.9+ mAP@0.5.
Phase 2 (on-demand enhancement). Where a GPU server is available, integrate LocateAnything at low frequency and asynchronously to handle open-ended queries YOLO cannot cover. The typical pattern: YOLO guarantees multi-frame-per-second real-time detection and alerts; LocateAnything performs semantic review on key frames (e.g., "are there dump trucks without tarps in the frame?") as a second engine called on demand, rather than replacing YOLO on the main path.
A middle option worth evaluating in parallel is YOLOE. It preserves YOLO-level real-time performance while offering some open-vocabulary capability, suitable for scenarios that "need a dynamic vocabulary but cannot afford 3B VLM latency." Comparison experiments on Apple Silicon also suggest: YOLO26n suits continuous fixed-class perception, YOLOE is a practical intermediate layer between real-time detection and large VLMs, and LocateAnything is strongest at complex referring but has significantly higher speed and memory cost.
6. Conclusion
LocateAnything's technical contribution is real: PBD repairs, at the decoding-mechanism level, the long-standing pair of conflicting issues in VLM grounding—serial speed bottleneck and geometric consistency loss. Hybrid inference offers an actionable engineering compromise between speed and stability, and its lead on high-IoU benchmarks proves the practical benefit of atomic decoding. But its "fast" has a clear coordinate system—data-center GPU, within the VLM race, non-real-time video-stream scenarios.
For heavy construction vehicle detection, the first priority is always real-time performance, edge deployability, and long-run reliability. In that coordinate system, YOLO remains the most rational, lowest-risk primary detector today. LocateAnything's correct position is to complement YOLO's structural blind spot on long-tail semantic queries with open-vocabulary and complex referring capability—called on demand as a semantic side-path, not as a replacement for real-time detection.
The essence of selection is not comparing the absolute merits of two models, but identifying the engineering problem boundaries each is suited to, then letting them each do their job within the same system.
Questions
Is LocateAnything faster than YOLO?
Not in the sense that matters for real-time detection. LocateAnything reaches 12.7 boxes per second on an H100, which is fast relative to other vision-language grounding models (about 10x Qwen3-VL). YOLO runs at millisecond latency on consumer and edge hardware—YOLO26n at roughly 17.5 ms per frame, versus LocateAnything taking about 10.6 seconds for a complex query on an M2 8GB. They are one to three orders of magnitude apart and do not share a speed coordinate system.
Should I use YOLO or LocateAnything for construction-site heavy vehicle detection?
Use YOLO as the real-time base detector for safety monitoring and dispatch—it offers millisecond latency, low memory footprint, and a mature construction-site ecosystem. Use LocateAnything as an on-demand semantic side-path for open-ended queries such as "find trucks without tarps" or "cranes currently lifting rebar," where YOLO's fixed class set cannot help. The recommended architecture is YOLO on the main real-time path plus LocateAnything for asynchronous semantic review.
Can LocateAnything run on edge devices?
Only with significant constraints. Normal inference needs about 7.75 GB of memory, and high-resolution images can require far more under SDPA without flash-attn. On an M2 8GB it takes about 10.6 seconds for a complex query, approaching the memory ceiling. It is not suitable for real-time edge video streams; YOLO nano-class models remain the practical edge choice.
What is Parallel Box Decoding (PBD)?
PBD is LocateAnything's decoding mechanism that treats each bounding box as a fixed-length atomic unit and predicts all four coordinates in a single parallel step, instead of serializing coordinate tokens autoregressively. This improves throughput and preserves intra-box geometric consistency, with a hybrid mode that falls back to token-by-token decoding when parallel output is unstable.
Does LocateAnything's license allow commercial use?
The code is Apache 2.0, but the model weights use the NVIDIA non-commercial research license. Commercial productization requires evaluating the authorization path; YOLO's AGPL/enterprise licensing provides a clearer commercial route.
Zhcloudlink