frank/ruvnet--RuView
1
0
Fork 0
mirror of https://github.com/ruvnet/RuView synced 2026-08-09 20:21:43 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(benchmarks): edge optimization measured + measurement (b) blocked + 92.9% retraction

Browse Source 
Edge optimization (ADR-152 optimize track): ONNX Runtime fp32 is the CPU
latency win (3.2 ms/window, ~3.4x faster than torch, parity 2.4e-7); ORT
dynamic int8 reaches 2.44 MB (paper's ~2.2 MB claim plausible only via
conv-capable toolchains; -0.16pt PCK@20, +18% MPJPE, 2x slower); torch
dynamic quant converts 0% of this conv-only model; fp16 halves storage free
but is slower on CPU.

Measurement (b) BLOCKED-ON-DATA: only 1,077 paired ESP32 windows exist
(stop rule <2k). Forensic recheck of the surviving April holdout RETRACTS
the ADR-079 '92.9% PCK@20' figure: constant-output model, absolute (not
torso) threshold, 69 near-static frames — mean predictor scores 100% under
that protocol; torso-PCK@20 is 19.1%. Corroborates PR #535. Stale citations
removed from user-guide, readme-details, ADR-152 §2.1.3; no-citation rule
extended to ADR-079 accuracy claims. Unblock: >=2k-window multi-pose paired
session + torso-PCK re-baseline.

Co-Authored-By: claude-flow <ruv@ruv.net>
This commit is contained in:
ruv
2026-06-10 22:05:39 -04:00
parent 0e4b596803
commit 42b261f807
9 changed files with 981 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/readme-details.md
+15 -10
View File
@@ -50,7 +50,7 @@ See [PR #405](https://github.com/ruvnet/RuView/pull/405) for full details.
### What's New in v0.7.0
<details>
<summary><strong>Camera Ground-Truth Training — 92.9% PCK@20</strong></summary>
<summary><strong>Camera Ground-Truth Training</strong></summary>
**v0.7.0 adds camera-supervised pose training** using MediaPipe + real ESP32 CSI data:
@@ -76,15 +76,20 @@ node scripts/train-wiflow-supervised.js --data data/paired/*.jsonl --scale lite
node scripts/eval-wiflow.js --model models/wiflow-real/wiflow-v1.json --data data/paired/*.jsonl
```
**Result: 92.9% PCK@20** from a 5-minute data collection session with one ESP32-S3 and one webcam.
> **Accuracy retraction (2026-06-10):** the "92.9% PCK@20" figure previously
> shown here is retracted. A forensic recheck of the surviving eval holdout
> (69 samples) found a constant-output model scored with an absolute
> (non-torso-normalized) threshold on nearly-static frames — a protocol under
> which a trivial mean-pose predictor scores 100%. Torso-normalized PCK@20 on
> the same holdout is ~19% (from that degenerate predictor). No measured
> camera-supervised PCK@20 is currently published (CHANGELOG, PR #535).
| Metric | Before (proxy) | After (camera-supervised) |
|--------|----------------|--------------------------|
| PCK@20 | 0% | **92.9%** |
| Eval loss | 0.700 | **0.082** |
| Bone constraint | N/A | **0.008** |
| Training time | N/A | **19 minutes** |
| Model size | N/A | **974 KB** |
| Metric | Camera-supervised run (protocol retracted) |
|--------|--------------------------------------------|
| Eval loss | 0.082 |
| Bone constraint | 0.008 |
| Training time | 19 minutes |
| Model size | 974 KB |
Pre-trained model: [HuggingFace ruv/ruview/wiflow-v1](https://huggingface.co/ruv/ruview)
@@ -868,7 +873,7 @@ Download a pre-built binary — no build toolchain needed:
| Release | What's included | Tag |
|---------|-----------------|-----|
| [v0.7.0](https://github.com/ruvnet/RuView/releases/tag/v0.7.0) | **Latest** — Camera-supervised WiFlow model (92.9% PCK@20), ground-truth training pipeline, ruvector optimizations | `v0.7.0` |
| [v0.7.0](https://github.com/ruvnet/RuView/releases/tag/v0.7.0) | **Latest** — Camera-supervised WiFlow model (accuracy figure retracted 2026-06-10, see above), ground-truth training pipeline, ruvector optimizations | `v0.7.0` |
| [v0.6.0](https://github.com/ruvnet/RuView/releases/tag/v0.6.0-esp32) | [Pre-trained models on HuggingFace](https://huggingface.co/ruv/ruview), 17 sensing apps, 51.6% contrastive improvement, 0.008ms inference | `v0.6.0-esp32` |
| [v0.5.5](https://github.com/ruvnet/RuView/releases/tag/v0.5.5-esp32) | SNN + MinCut (#348 fix) + CNN spectrogram + WiFlow + multi-freq mesh + graph transformer | `v0.5.5-esp32` |
| [v0.5.4](https://github.com/ruvnet/RuView/releases/tag/v0.5.4-esp32) | Cognitum Seed integration ([ADR-069](docs/adr/ADR-069-cognitum-seed-csi-pipeline.md)), 8-dim feature vectors, RVF store, witness chain, security hardening | `v0.5.4-esp32` |