mirror of
https://github.com/ruvnet/RuView
synced 2026-07-27 18:11:43 +00:00
feat(cog-person-count): train count_v1.safetensors — honest v0.0.1 (ADR-103) (#695)
Phase 2 of ADR-103: trained count head on the existing 1,077 paired samples (the same data that produced pose_v1 yesterday). Honest result: 65.1% eval accuracy / 100% within ±1 / MAE 0.349 on the held-out time-window. Per-class: 100% on "empty room" / 0% on "1 person". The model overfit by epoch 100 (train_acc → 1.0, eval_loss climbed 0.67 → 7.8) and the "best" checkpoint is the snapshot that happened to predict the eval window's class distribution (140/215 = 65.1%, matches eval_acc exactly). Confidence head Spearman = 0.023 ⇒ uncalibrated. Same data-bound failure mode as pose_v1 (#645), bounded by single-session training data; same fix path (multi-room). What v0.0.1 still validates end-to-end: * PyTorch → safetensors → Candle Rust loads cleanly on first try. `cog-person-count health` reports `backend: candle-cpu` and emits real per-frame predictions instead of the stub backend's hard-coded {1 person, 0 confidence}. Architecture parity between train-count.py and src/inference.rs::CountNet is bit-exact. * ONNX export bit-clean (16 KB, opset 18, dynamic batch axis). * Training wall time: 5.6 s for 400 epochs on RTX 5080. * Binary size unchanged (2.36 MB stripped), model loads via mmap at runtime. This commit ships: * scripts/align-ground-truth.js: extended to emit n_persons_mode + n_persons_max per window so the training pipeline has count labels. Backwards-compatible (additive fields). * scripts/train-count.py: new — mirrors CountNet architecture exactly, loads paired.jsonl, trains 400 epochs with CE+BCE+Brier loss, exports safetensors + ONNX + per-epoch JSON. * v2/.../cog/artifacts/{count_v1.safetensors,count_v1.onnx, count_train_results.json}: the trained artifacts. * v2/.../cog/README.md: Status table updated with the v0.0.1 numbers + an Honest Caveat section explaining the data-bound result. * docs/benchmarks/person-count-cog.md: new — full v0.0.1 benchmark log mirroring the format docs/benchmarks/pose-estimation-cog.md established. Includes comparison to ADR-103 v0.1.0 acceptance gates and per-class breakdown. Still pending: * `run` subcommand wiring (long-running polling loop, same as pose) * Cross-compile + sign + GCS upload (mirror of pose cog pipeline) * Live install on cognitum-v0 * v0.2.0: re-train on multi-room data, LoRA per-room adapters, Stoer-Wagner min-cut clip in fusion stage
This commit is contained in:
@@ -27,19 +27,25 @@ Replaces the PR #491 slot heuristic (`subcarrier_diversity / dedup_factor`) with
|
||||
|
||||
Downstream consumers can render the **most-likely count** when confidence is high, or fall back to a `[lo, hi]` band with a "?" badge when the model is uncertain — that's how this Cog closes the loop on #499's ghost-skeleton UX.
|
||||
|
||||
## Status — v0.0.1 (this scaffold)
|
||||
## Status — v0.0.1
|
||||
|
||||
| Component | State |
|
||||
|---|---|
|
||||
| Crate compiles, library API stable | ✅ |
|
||||
| Tests pass (`cargo test -p cog-person-count`) | ✅ |
|
||||
| Tests pass (15 total: 8 smoke + 7 fusion) | ✅ |
|
||||
| Four-verb runtime contract (`version`, `manifest`, `health`) | ✅ |
|
||||
| `run` subcommand (long-running loop) | ⏳ v0.0.1 follow-up |
|
||||
| Trained `count_v1.safetensors` artifact | ⏳ same training pipeline that produced `pose_v1` — bootstrap on the existing 1,077 paired samples |
|
||||
| Signed binary on GCS | ⏳ once trained |
|
||||
| Trained `count_v1.safetensors` artifact | ✅ shipped at `cog/artifacts/count_v1.safetensors` (392 KB) |
|
||||
| ONNX export | ✅ `count_v1.onnx` (16 KB), bit-compatible architecture |
|
||||
| Honest accuracy reporting | ✅ See `docs/benchmarks/person-count-cog.md` — 65.1% eval acc on a single-session dataset; confidence head Spearman 0.023 ⇒ uncalibrated for v0.0.1 |
|
||||
| `run` subcommand (long-running loop) | ⏳ same shape as cog-pose-estimation::runtime, lands in follow-up |
|
||||
| Signed binary on GCS | ⏳ release pipeline |
|
||||
| Stoer-Wagner min-cut clip in fusion stage | ⏳ v0.2.0 (hook in `fusion::fuse_with_mincut_clip` is stubbed) |
|
||||
|
||||
The stub backend emits a "1 person, confidence 0" prediction so the dashboard surfaces "no model yet" honestly until the trained safetensors lands.
|
||||
### Honest v0.0.1 caveat
|
||||
|
||||
`count_v1` was trained on a single 30-minute solo recording. The model overfit by epoch ~100 and the "best" checkpoint is one that effectively predicts the eval-window class distribution (mostly class-0). Class-1 accuracy on the held-out tail = 0%. **This v0.0.1 is a working pipeline with a degenerate model**, not a usable counter yet — same data-bound failure mode as `pose_v1` (#645), same fix: multi-room paired recordings.
|
||||
|
||||
`cog-person-count health` will load the real safetensors and report `backend: candle-cpu` rather than `backend: stub`, so the cog-gateway can verify the model loaded — but operators should treat the v0.0.1 count outputs as scaffold-validation rather than production data. The 2.36 MB binary + 392 KB weights + 16 KB ONNX are all real and reusable as soon as more data lands.
|
||||
|
||||
## Security
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
Binary file not shown.
Binary file not shown.
Reference in New Issue
Block a user