Compare commits

...

2 Commits

Author SHA1 Message Date
rUv 573b00fd98 perf(ci): drop dead uvicorn start from perf job (#917)
Since #915 the perf job gates only on test_frame_budget.py, which drives
the CSIProcessor pipeline in-process and makes no HTTP calls. The
"Start application" step (uvicorn + `sleep 10`) was therefore dead weight:
it existed only for the now-excluded api_throughput/inference_speed tests,
wasted ~10-15 s per main-push run, and dumped ~50 misleading
"router requires hardware setup" ERROR lines into every CI log for a
server no test touched. MOCK_POSE_DATA is server-only, unused here.

Removed the step and the vestigial env. The gated test is unchanged and
passes (verified locally, 3/3).
2026-06-02 19:01:08 +02:00
rUv 91b0e625bd docs(#882): complete the "100% presence" retraction across all docs (#916)
The v1 "100% presence accuracy" headline was already retracted in the
README / user-guide intro / proof-of-capabilities — but 6 secondary
spots still flatly claimed "100% accuracy, never false alarms", which
made proof-of-capabilities.md's "replaced everywhere" assertion untrue.

Completed the retraction in-place with the honest label-free metric
(82.3% held-out temporal-triplet; v1 was a single-class recording where
a constant "yes" scores ~99.98%):

- docs/readme-details.md — 2 benchmark tables + the pre-trained-model row
- docs/user-guide.md — capability table, model-file comment, applications list
- CHANGELOG.md — annotated the historical entry in-place (kept as public
  record per built-in-public ethos, not rewritten)

Verified: no remaining flat "100% presence/accuracy" claim lacks a
retraction marker; proof-of-capabilities.md "replaced everywhere" is now
accurate.
2026-06-02 18:50:39 +02:00
4 changed files with 13 additions and 18 deletions
+6 -11
View File
@@ -267,20 +267,15 @@ jobs:
pip install -r requirements.txt
pip install pytest # the perf suite is pytest, not locust
- name: Start application
working-directory: archive/v1
env:
# No CSI hardware in CI — serve mock pose data so the pose endpoints
# respond 200 under load instead of erroring "requires real CSI data".
MOCK_POSE_DATA: "true"
run: |
uvicorn src.api.main:app --host 0.0.0.0 --port 8000 &
sleep 10
# No "Start application" step: the gated test (test_frame_budget.py) drives
# the CSIProcessor pipeline in-process and makes no HTTP calls, so the old
# uvicorn server + `sleep 10` were dead weight — they only existed for the
# now-excluded api_throughput/inference_speed tests, and on every run dumped
# ~50 misleading "router requires hardware setup" ERROR lines for a server
# no test touched. MOCK_POSE_DATA is server-only and unused here.
- name: Run performance tests
working-directory: archive/v1
env:
MOCK_POSE_DATA: "true"
run: |
# Gate only on the genuine, deterministic perf guard:
# test_frame_budget.py times the *real* CSIProcessor pipeline against
+1 -1
View File
@@ -430,7 +430,7 @@ Model release (no new firmware binary). Firmware remains at v0.6.0-esp32.
- Security fix merged via PR #310.
### Performance
- Presence detection: 100% accuracy on 60,630 overnight samples.
- Presence detection: 100% accuracy on 60,630 overnight samples. *(Retracted — that recording was single-class (one sleeping person, 6,062/6,063 frames "present"), so a constant "yes" scores ~99.98%. Superseded by the honest 82.3% held-out temporal-triplet metric; see [#882](https://github.com/ruvnet/RuView/issues/882). Kept here as the in-place public record.)*
- Inference: 0.008 ms per sample, 164K embeddings/sec.
- Contrastive self-supervised training: 51.6% improvement over baseline.
+3 -3
View File
@@ -122,7 +122,7 @@ node scripts/benchmark-ruvllm.js --model models/csi-ruvllm # benchmark
| What we measured | Result | Why it matters |
|-----------------|--------|---------------|
| **Presence detection** | **100% accuracy** | Never misses a person, never false alarms |
| **CSI embedding quality** | **82.3% held-out temporal-triplet** | Honest label-free metric on the last 20% by time (v1's "100% presence" was a single-class recording — retracted, [#882](https://github.com/ruvnet/RuView/issues/882)) |
| **Inference speed** | **0.008 ms** per embedding | 125,000x faster than real-time |
| **Throughput** | **164,183 embeddings/sec** | One Mac Mini handles 1,600+ ESP32 nodes |
| **Contrastive learning** | **51.6% improvement** | Strong pattern learning from real overnight data |
@@ -233,7 +233,7 @@ python firmware/esp32-csi-node/provision.py --port COM9 --hop-channels "1,6,11"
| **kNN similarity search** | "Find the 10 most similar states to right now" — anomaly detection, fingerprinting | Cognitum Seed |
| **Witness chain** | SHA-256 tamper-evident audit trail for every measurement (1,747 entries validated) | Cognitum Seed |
| **Camera-free pose training** | 17 COCO keypoints from 10 sensor signals — PIR, RSSI triangulation, subcarrier asymmetry, vibration, BME280 | 2x ESP32 + Seed |
| **Pre-trained model** | 82.8 KB (8 KB at 4-bit quantization), 100% presence accuracy, 0 skeleton violations | Download from release |
| **Pre-trained model** | 82.8 KB (8 KB at 4-bit quantization), 82.3% held-out temporal-triplet accuracy (v1's "100% presence" was single-class — retracted, [#882](https://github.com/ruvnet/RuView/issues/882)) | Download from release |
| **Sub-ms inference** | 0.012 ms latency, 171,472 embeddings/sec on M4 Pro | Any machine with Node.js |
| **SONA adaptation** | Adapts to new rooms in <1ms without retraining | ruvllm runtime |
| **LoRA room adapters** | Per-node fine-tuning with 2,048 parameters per adapter | Automatic |
@@ -262,7 +262,7 @@ node scripts/benchmark-ruvllm.js --model models/csi-ruvllm
| What we measured | Result | Why it matters |
|-----------------|--------|---------------|
| **Presence detection** | **100% accuracy** | Never misses a person, never false alarms |
| **CSI embedding quality** | **82.3% held-out temporal-triplet** | Honest label-free metric (v1's "100% presence" was single-class — retracted, [#882](https://github.com/ruvnet/RuView/issues/882)) |
| **Person counting** | **24/24 correct** (MinCut) | Fixed the #1 user-reported issue |
| **Inference speed** | **0.012 ms** per embedding | 83,000x faster than real-time |
| **Throughput** | **171,472 embeddings/sec** | One Mac Mini handles 1,700+ ESP32 nodes |
+3 -3
View File
@@ -1119,7 +1119,7 @@ What it ships (and what it does not):
| Capability | Status |
|------------|--------|
| Presence detection (occupied / empty) | ✅ Trained head — 100% accuracy on validation |
| Presence detection (occupied / empty) | ✅ Trained head — v2 encoder reports 82.3% held-out temporal-triplet acc (v1's "100% on validation" was a single-class recording — retracted, [#882](https://github.com/ruvnet/RuView/issues/882)) |
| 128-dim CSI embeddings (re-ID, similarity, downstream training) | ✅ Trained encoder |
| Single-person breathing / heart-rate | ⚠️ Server still uses heuristic DSP — model does not replace this yet |
| 17-keypoint full-body pose | 🔬 No keypoint weights shipped yet — pose pipeline runs but without a learned head |
@@ -1824,7 +1824,7 @@ huggingface-cli download ruvnet/wifi-densepose-pretrained --local-dir models/pre
# model.safetensors — 48 KB contrastive encoder
# model-q4.bin — 8 KB quantized (recommended)
# model-q2.bin — 4 KB ultra-compact (ESP32 edge)
# presence-head.json — presence detection head (100% accuracy)
# presence-head.json — presence detection head (v2 encoder: 82.3% held-out triplet acc)
# node-1.json — LoRA adapter for room 1
# node-2.json — LoRA adapter for room 2
```
@@ -1833,7 +1833,7 @@ huggingface-cli download ruvnet/wifi-densepose-pretrained --local-dir models/pre
The pre-trained encoder converts 8-dim CSI feature vectors into 128-dim embeddings. These embeddings power all 17 sensing applications:
- **Presence detection** — 100% accuracy, never misses, never false alarms
- **Presence detection** — v2 encoder: 82.3% held-out temporal-triplet accuracy (v1's "100%" was a single-class recording — retracted, [#882](https://github.com/ruvnet/RuView/issues/882))
- **Environment fingerprinting** — kNN search finds "states like this one"
- **Anomaly detection** — embeddings that don't match known clusters = anomaly
- **Activity classification** — different activities cluster in embedding space