mirror of
https://github.com/ruvnet/RuView
synced 2026-06-09 10:13:17 +00:00
rUv
3314c8db8d
* feat(cog-pose-estimation): scaffold first Cog from this repo (ADR-100 + ADR-101) Adds the foundation for the pose-estimation Cog that ships from this repo into Cognitum V0 appliances. Companion ADR-225 + crate land in cognitum-one/v0-appliance. ADRs: * ADR-100 formalises the Cognitum Cog packaging spec — on-device layout under /var/lib/cognitum/apps/<id>/, manifest.json schema (incl. new binary_sha256 + binary_signature fields), GCS hosting convention, repo source layout, build pipeline, and the four-verb runtime contract (version | manifest | health | run). Documents the convention I reverse-engineered from inspecting installed cogs on a live cognitum-v0 appliance — `anomaly-detect`, `presence`, `seizure-detect`, etc. * ADR-101 designs the pose-estimation Cog itself: where it sits in the wifi-densepose pipeline (encoder init from ruvnet/wifi-densepose-pretrained, 17-keypoint regression head), what gets shipped per target arch (arm / x86_64 / hailo8 / hailo10), acceptance gates (PCK@20 explicitly deferred to #640 — this ADR ships the vehicle, not the accuracy). Crate v2/crates/cog-pose-estimation/: * Cargo.toml + workspace member declaration with a hailo feature gate so the binary builds without the Hailo SDK in CI. * main.rs implements the four-verb CLI exactly per ADR-100. * config.rs / manifest.rs / publisher.rs / inference.rs / runtime.rs — small modules, each <100 lines. * publisher.rs emits ADR-100 structured JSON events. * inference.rs is a stub that produces a centred-skeleton baseline with confidence=0 (honest: no trained weights wired in yet). * runtime.rs subscribes to /api/v1/sensing/latest, slides a 56*20 window, runs the engine, emits pose.frame events. * cog/manifest.template.json + cog/config.schema.json define the release artifact + runtime config schemas. * cog/Makefile holds build / sign / upload targets. * tests/smoke.rs covers manifest roundtrip + engine I/O surface. Verified locally: * cargo check -p cog-pose-estimation: clean. * cargo test -p cog-pose-estimation: 4/4 pass. * ./target/release/cog-pose-estimation {version,manifest,health}: all emit the right contract output. This commit contains scaffolding only; the actual trained weights and Hailo HEF cross-compile come in follow-ups tracked in #640 and the companion v0-appliance branch. * feat(cog-pose-estimation): first measured run — Candle CUDA on RTX 5080 Trained pose_v1 on ruvultra (RTX 5080) via Candle 0.9 + cuda feature against the same 1,077-sample paired session that produced 0%/0% PCK in #640 with the pure-JS SPSA trainer. First real numbers: PCK@20 = 3.0% (up from 0.0%) PCK@50 = 18.5% (up from 0.0%) MPJPE = 0.093 (down from 0.66, ~7x improvement) 400 epochs in 2.1 s wall time, full-batch, ~5 ms/epoch. Loss curve 0.181 -> 0.014 over the run, eval 0.010. Per-joint reveals the model leans on right-side proximal joints (r_hip 77% PCK@50, r_knee 35%, l_elbow 26%) — consistent with the camera framing in the source recording. Distal joints (wrists, ankles) and face joints are still near-random, consistent with the 56-subcarrier / 20-frame input not carrying fine-grained spatial info at 1077 samples. This commit: * Adds v2/crates/cog-pose-estimation/cog/artifacts/{pose_v1.safetensors, train_results.json} so the cog dir now contains a real reference artifact, not just scaffold. * Updates cog/README.md "Status" block with the measured numbers, per-joint table, and an honest reading of where the model succeeds vs where the data is the bottleneck. * Adds docs/benchmarks/pose-estimation-cog.md as the canonical benchmark log — append-only, one section per published run. * Appends a "First measured run" section to ADR-101 referencing the new benchmark file. Still pending in the follow-up: * Wire pose_v1.safetensors into src/inference.rs (replace stub). * ONNX export (Candle lacks a writer — needs external conversion). * Hailo HEF cross-compile + cluster deploy. The data-bound gap to PCK@20 >= 35% is tracked in #640. * feat(cog-pose-estimation): wire real weights — cog is no longer a stub Replaces the centred-skeleton stub in src/inference.rs with a real Candle-based loader that reads cog/artifacts/pose_v1.safetensors and runs the trained Conv1d encoder + MLP pose head on every incoming CSI window. What changes: * src/inference.rs: PoseNet mirrors the training script's architecture exactly — Conv1d(56->64, k=3 d=1), Conv1d(64->128, k=3 d=2), Conv1d(128->128, k=3 d=4), mean over time, Linear(128->256)+ReLU, Linear(256->34)+sigmoid -> reshape [17, 2]. The InferenceEngine searches a sensible candidate list for the weights file (/var/lib/cognitum/apps/pose-estimation/, ./pose_v1.safetensors, ./cog/artifacts/, repo-root, v2/-relative) and falls back to the stub when none are present so the cog still satisfies ADR-100. * Cargo.toml: adds candle-core 0.9 + candle-nn 0.9 (no-default-features, CPU build by default) + safetensors 0.4. New `cuda` feature opt-in for GPU inference on hosts that have it. Drops the unused wifi-densepose-train path dep from the default build path. * src/main.rs + src/publisher.rs: health.ok event now carries `backend` (candle-cuda | candle-cpu | stub) and the synthetic output confidence, so operators can tell at a glance whether the cog loaded its weights or fell back to the stub. * tests/smoke.rs: adds `real_weights_load_when_available` which asserts the loaded engine reports backend=candle-* and emits non-zero confidence — exactly the signal that proves we're not silently degrading to the stub. Verified locally: * `cargo check -p cog-pose-estimation --no-default-features` — clean * `cargo test -p cog-pose-estimation --no-default-features` — 5/5 pass * `./target/release/cog-pose-estimation health` emits: {"event":"health.ok","fields":{"backend":"candle-cpu","cog":"pose-estimation","synthetic_output_confidence":0.185}} — 0.185 is the published PCK@50 from cog/artifacts/train_results.json, emitted by the real Candle inference path (would be 0.0 if it had fallen back to the stub). The cog now runs the trained pose_v1 model end-to-end. Accuracy is still bounded by the underlying 1077-sample training data (PCK@20 3.0%, PCK@50 18.5% per docs/benchmarks/pose-estimation-cog.md) — that gap is data-bound and tracked in #640. ONNX export + Hailo HEF cross-compile remain follow-ups. * docs(benchmarks): measure cog-pose-estimation cold-start latency 100 sequential `cog-pose-estimation health` invocations average 76.2 ms each on a Windows x86_64 host using the `candle-cpu` backend. Each invocation re-loads pose_v1.safetensors and runs one synthetic forward pass, so this is the worst-case cold-start path. Long-running `run` inference will be sub-millisecond per frame once the model is loaded. Updates the benchmarks doc accordingly. * feat(cog-pose-estimation): ONNX export — pose_v1.onnx + scripts/export-onnx.py Adds the canonical ONNX artifact that unblocks downstream Hailo HEF cross-compile + ONNX Runtime benchmarks. Generated on ruvultra (torch 2.12.0 + CUDA), 12,059 bytes, opset 18, dynamic batch axis. * scripts/export-onnx.py: mirrors the Candle inference architecture in PyTorch (Conv1d 56->64, 64->128, 128->128 + Linear 128->256->34), pure- python safetensors loader (no extra pip dep), exports via torch.onnx.export, then verifies via onnx.checker.check_model and numerical parity against the torch reference. * Verified parity vs torch: max |torch - onnx| = 8.94e-8 (1e-5 threshold). Effectively bit-perfect. * v2/crates/cog-pose-estimation/cog/artifacts/pose_v1.onnx — the artifact itself, 12 KB. * docs/benchmarks/pose-estimation-cog.md — adds an ONNX export section with the verification numbers. Next: Hailo HEF cross-compile (still gated on Hailo SDK on a self-hosted runner) and ONNX Runtime latency benchmarks on each target arch. * feat(cog-pose-estimation): release v0.0.1 — signed aarch64 binary on GCS End-to-end deploy: cross-compiled to aarch64-unknown-linux-gnu on ruvultra, ran via qemu-aarch64-static, then smoke-tested on a real cognitum-v0 Pi 5. Signed with COGNITUM_OWNER_SIGNING_KEY (Ed25519) and uploaded to gs://cognitum-apps/cogs/arm/. Real-hardware results on cognitum-v0 (Pi 5): health: backend=candle-cpu, confidence=0.185, real weights loaded 30x sequential `health`: 0.251 s total -> 8.4 ms / invocation (cold) GCS release artifacts (publicly downloadable): binary: 3,741,976 bytes sha256 1e1a7d3dd01ca05d5bfc5dbb142a5941b7866ed9f3224a21edc04d3f09a99bf5 weights: 507,032 bytes sha256 eb249b9a6b2e10130437a10976ed0230b0d085f86a0553d7226e1ae6eae4b9e5 signature (Ed25519, b64): LUN7xqLPYD3MFzm5dKB5MnYU0LvoRtek5ci5KiKPHBg+Xo6xuazwokn2Dw2JPMaLYJzmWn/SpT4djuR7hYvVDw== Adds: * v2/crates/cog-pose-estimation/cog/artifacts/manifest.json — the release-pipeline-produced manifest with all fields filled in per ADR-100, including arch, target_triple, signature, and a build_metadata block carrying the validation PCK numbers. * docs/benchmarks/pose-estimation-cog.md — new sections covering the real Pi 5 smoke (8.4 ms cold-start) and the signed GCS release artifacts. Verified by downloading the binary anonymously from GCS and re-computing the sha256 — matches the locally-computed sha exactly. Signature decoded to the expected 64-byte Ed25519 length. Closes the GCS-upload acceptance criterion from ADR-100; the only pending work is Hailo HEF cross-compile (still SDK-gated) and an x86_64 release alongside this arm release. * docs(benchmarks): record live cognitum-v0 install + 5-sec smoke run Adds the "Live appliance install" section documenting what happened when the signed v0.0.1 binary + weights were installed under /var/lib/cognitum/apps/pose-estimation/ on cognitum-v0 (the V0 cluster leader). * Layout matches the existing anomaly-detect / presence / seizure- detect cogs exactly — the Cogs dashboard at http://cognitum-v0:9000/cogs auto-discovers entries. * `cog-pose-estimation run` ran for 5 seconds in the background and cleanly emitted run.started + structured WARN events for the missing local sensing-server on :3000 (cognitum-v0's actual CSI source is ruview-vitals-worker on :50054, not :3000). No crashes, no NaN, no leaks. * Wiring `sensing_url` to the appliance-native source is a separate Day-2 integration task.
Architecture Decision Records
This folder contains 44 Architecture Decision Records (ADRs) that document every significant technical choice in the RuView / WiFi-DensePose project.
Why ADRs?
Building a system that turns WiFi signals into human pose estimation involves hundreds of non-obvious decisions: which signal processing algorithms to use, how to bridge ESP32 firmware to a Rust pipeline, whether to run inference on-device or on a server, how to handle multi-person separation with limited subcarriers.
ADRs capture the context, options considered, decision made, and consequences for each of these choices. They serve three purposes:
-
Institutional memory — Six months from now, anyone (human or AI) can read why we chose IIR bandpass filters over FIR for vital sign extraction, not just see the code.
-
AI-assisted development — When an AI agent works on this codebase, ADRs give it the constraints and rationale it needs to make changes that align with the existing architecture. Without them, AI-generated code tends to drift — reinventing patterns that already exist, contradicting earlier decisions, or optimizing for the wrong tradeoffs.
-
Review checkpoints — Each ADR is a reviewable artifact. When a proposed change touches the architecture, the ADR forces the author to articulate tradeoffs before writing code, not after.
ADRs and Domain-Driven Design
The project uses Domain-Driven Design (DDD) to organize code into bounded contexts — each with its own language, types, and responsibilities. ADRs and DDD work together:
- ADRs define boundaries: ADR-029 (RuvSense) established multistatic sensing as a separate bounded context from single-node CSI. ADR-042 (CHCI) defined a new aggregate root for coherent channel imaging.
- DDD models define the language: The RuvSense domain model defines terms like "coherence gate", "dwell time", and "TDM slot" that ADRs reference precisely.
- Together they prevent drift: An AI agent reading ADR-039 knows that edge processing tiers are configured via NVS keys, not compile-time flags — because the ADR says so. The DDD model tells it which aggregate owns that configuration.
How ADRs are structured
Each ADR follows a consistent format:
- Context — What problem or gap prompted this decision
- Decision — What we chose to do and how
- Consequences — What improved, what got harder, and what risks remain
- References — Related ADRs, papers, and code paths
Statuses: Proposed (under discussion), Accepted (approved and/or implemented), Superseded (replaced by a later ADR).
ADR Index
Hardware and firmware
| ADR | Title | Status |
|---|---|---|
| ADR-012 | ESP32 CSI Sensor Mesh for Distributed Sensing | Accepted (partial) |
| ADR-018 | ESP32 Development Implementation Path | Proposed |
| ADR-028 | ESP32 Capability Audit and Witness Record | Accepted |
| ADR-029 | RuvSense Multistatic Sensing Mode (TDM, channel hopping) | Proposed |
| ADR-032 | Multistatic Mesh Security Hardening | Accepted |
| ADR-039 | ESP32-S3 Edge Intelligence Pipeline (on-device vitals) | Accepted (hardware-validated) |
| ADR-040 | WASM Programmable Sensing (Tier 3) | Accepted |
| ADR-041 | WASM Module Collection (65 edge modules) | Accepted (hardware-validated) |
| ADR-044 | Provisioning Tool Enhancements | Proposed |
Signal processing and sensing
| ADR | Title | Status |
|---|---|---|
| ADR-013 | Feature-Level Sensing on Commodity Gear | Accepted |
| ADR-014 | SOTA Signal Processing Algorithms | Accepted |
| ADR-021 | Vital Sign Detection (breathing, heart rate) | Partial |
| ADR-030 | Persistent Field Model and Drift Detection | Proposed |
| ADR-033 | CRV Signal Line Sensing Integration | Proposed |
| ADR-037 | Multi-Person Pose Detection from Single ESP32 | Proposed |
| ADR-042 | Coherent Human Channel Imaging (beyond CSI) | Proposed |
Machine learning and training
| ADR | Title | Status |
|---|---|---|
| ADR-005 | SONA Self-Learning for Pose Estimation | Partial |
| ADR-006 | GNN-Enhanced CSI Pattern Recognition | Partial |
| ADR-015 | Public Dataset Strategy (MM-Fi, Wi-Pose) | Accepted |
| ADR-016 | RuVector Training Pipeline Integration | Accepted |
| ADR-017 | RuVector Signal + MAT Integration | Proposed |
| ADR-020 | Migrate AI Inference to Rust (ONNX Runtime) | Accepted |
| ADR-023 | Trained DensePose Model with RuVector Pipeline | Proposed |
| ADR-024 | Project AETHER: Contrastive CSI Embeddings | Required |
| ADR-027 | Project MERIDIAN: Cross-Environment Generalization | Proposed |
Platform and UI
| ADR | Title | Status |
|---|---|---|
| ADR-019 | Sensing-Only UI with Gaussian Splats | Accepted |
| ADR-022 | Windows WiFi Enhanced Fidelity (multi-BSSID) | Partial |
| ADR-025 | macOS CoreWLAN WiFi Sensing | Proposed |
| ADR-031 | RuView Sensing-First RF Mode | Proposed |
| ADR-034 | Expo React Native Mobile App | Accepted |
| ADR-035 | Live Sensing UI Accuracy and Data Transparency | Accepted |
| ADR-036 | Training Pipeline UI Integration | Proposed |
| ADR-043 | Sensing Server UI API Completion (14 endpoints) | Accepted |
Architecture and infrastructure
| ADR | Title | Status |
|---|---|---|
| ADR-001 | WiFi-Mat Disaster Detection Architecture | Accepted |
| ADR-002 | RuVector RVF Integration Strategy | Superseded |
| ADR-003 | RVF Cognitive Containers for CSI | Proposed |
| ADR-004 | HNSW Vector Search for Fingerprinting | Partial |
| ADR-007 | Post-Quantum Cryptography for Sensing | Proposed |
| ADR-008 | Distributed Consensus for Multi-AP | Proposed |
| ADR-009 | RVF WASM Runtime for Edge Deployment | Proposed |
| ADR-010 | Witness Chains for Audit Trail Integrity | Proposed |
| ADR-011 | Proof-of-Reality and Mock Elimination | Proposed |
| ADR-026 | Survivor Track Lifecycle (MAT crate) | Accepted |
| ADR-038 | Sublinear GOAP for Roadmap Optimization | Proposed |
| ADR-095 | rvCSI — Edge RF Sensing Runtime Platform | Proposed |
| ADR-096 | rvCSI — Crate Topology, the napi-c Shim, and the napi-rs Node Surface | Proposed |
| ADR-097 | Adopt rvCSI as RuView's primary CSI runtime (phased adoption) | Proposed |
| ADR-098 | Evaluate ruvnet/midstream for RuView's CSI / WebSocket / mesh pipeline |
Rejected |
| ADR-099 | Adopt midstream as RuView's real-time introspection + low-latency tap | Proposed |
Related
- DDD Domain Models — Bounded context definitions, aggregate roots, and ubiquitous language
- User Guide — Setup, API reference, and hardware instructions
- Build Guide — Building from source