Compare commits

..

12 Commits

Author SHA1 Message Date
rUv 573b00fd98 perf(ci): drop dead uvicorn start from perf job (#917)
Since #915 the perf job gates only on test_frame_budget.py, which drives
the CSIProcessor pipeline in-process and makes no HTTP calls. The
"Start application" step (uvicorn + `sleep 10`) was therefore dead weight:
it existed only for the now-excluded api_throughput/inference_speed tests,
wasted ~10-15 s per main-push run, and dumped ~50 misleading
"router requires hardware setup" ERROR lines into every CI log for a
server no test touched. MOCK_POSE_DATA is server-only, unused here.

Removed the step and the vestigial env. The gated test is unchanged and
passes (verified locally, 3/3).
2026-06-02 19:01:08 +02:00
rUv 91b0e625bd docs(#882): complete the "100% presence" retraction across all docs (#916)
The v1 "100% presence accuracy" headline was already retracted in the
README / user-guide intro / proof-of-capabilities — but 6 secondary
spots still flatly claimed "100% accuracy, never false alarms", which
made proof-of-capabilities.md's "replaced everywhere" assertion untrue.

Completed the retraction in-place with the honest label-free metric
(82.3% held-out temporal-triplet; v1 was a single-class recording where
a constant "yes" scores ~99.98%):

- docs/readme-details.md — 2 benchmark tables + the pre-trained-model row
- docs/user-guide.md — capability table, model-file comment, applications list
- CHANGELOG.md — annotated the historical entry in-place (kept as public
  record per built-in-public ethos, not rewritten)

Verified: no remaining flat "100% presence/accuracy" claim lacks a
retraction marker; proof-of-capabilities.md "replaced everywhere" is now
accurate.
2026-06-02 18:50:39 +02:00
rUv 88b835dd89 fix(ci): perf job gates on the real frame-budget guard, not TDD stubs (#915)
After #914 fixed collection, the perf job actually ran the suite and
exposed that test_api_throughput.py / test_inference_speed.py are TDD
red-phase stubs (every test suffixed `_should_fail_initially`) that time
a *mock that sleeps* — not a real perf signal. They carry machine-
dependent wall-clock asserts (actual_rps >= 40, batch_time < individual_time)
that are inherently flaky on shared CI runners, plus a cross-class
fixture-scope bug (`fixture 'standard_model' not found`). Result: 3 failed,
10 errored — by design, not a regression.

Forcing those green would manufacture a false signal. Instead, gate only
on test_frame_budget.py, which times the *real* CSIProcessor pipeline
against the ADR 50 ms per-frame budget (single-frame, p95/100-frames,
+Doppler) — a genuine regression guard. Verified locally: 3 passed.

The stub files remain in-repo for local TDD; they re-enter CI when their
features are implemented and the mock-timing asserts are made deterministic.
2026-06-02 18:31:55 +02:00
rUv f8f08076eb fix(ci): perf tests — use python -m pytest so src import resolves (#914)
The Performance Tests job collected 26 items then aborted with
`ModuleNotFoundError: No module named 'src'` on test_frame_budget.py,
which does `from src.core.csi_processor import CSIProcessor`. The bare
`pytest` console script does not put the cwd (archive/v1) on sys.path;
`python -m pytest` does. pytest aborts the whole session on a collection
error, so this one import masked the entire (otherwise mock-based,
self-contained) perf suite.

Verified locally: bare-script path reproduces the exact error; `-m`
resolves it and test_frame_budget.py passes 3/3. The other two files
(test_api_throughput.py mock server, test_inference_speed.py MockPoseModel
+psutil) are fully self-contained — no test hits the running server.

Closes the last red job in the v1-API CI chain (#910/#911/#913).
2026-06-02 18:12:00 +02:00
rUv 55f6a74e1e Merge pull request #913 from ruvnet/fix/ci-v1-api-perms-locust
ci(v1-api): fix gh-pages 403 + run real pytest perf suite
2026-06-02 17:36:43 +02:00
ruv b5a91c5635 ci(v1-api): install pytest, drop root --cov addopts for perf suite, ascii comment 2026-06-02 17:29:04 +02:00
ruv 308d2fc89d ci(v1-api): fix gh-pages 403 + run real perf suite — green main CI
Two more latent v1-API CI bugs surfaced once #910/#911 let the jobs reach
their later steps:

- API Documentation: openapi generation now succeeds (psutil fix), but the
  gh-pages deploy failed with HTTP 403 — the job had no `permissions` block
  and GITHUB_TOKEN is read-only by default. Add `permissions: contents:
  write`, and make the deploy `continue-on-error` (the openapi generation is
  the real validation; Pages may be disabled).
- Performance Tests: ran `locust -f tests/performance/locustfile.py`, but
  there is no locustfile — the suite is pytest (test_api_throughput.py,
  test_frame_budget.py, test_inference_speed.py). Run pytest instead, with
  working-directory: archive/v1 and MOCK_POSE_DATA=true.

ci.yml validated as well-formed YAML.
2026-06-02 17:26:39 +02:00
rUv 5038e3c8e1 Merge pull request #911 from ruvnet/fix/ci-v1-api-mock-mode
ci(v1-api): MOCK_POSE_DATA + declare psutil — green Performance Tests & API Docs
2026-06-02 06:20:21 -04:00
ruv e239af3636 fix(deps): declare psutil in requirements.txt — green API Documentation CI
The API Documentation job (and any env without locust) failed with
`ModuleNotFoundError: No module named 'psutil'` when importing the app:
psutil is imported by src/api/routers/health.py, services/metrics.py,
commands/status.py, and tasks/monitoring.py, but was never declared as a
dependency — it only happened to be present where locust (Performance
Tests) pulled it in transitively. Declare it explicitly (psutil>=5.9.0).

Verified locally: `from src.api.main import app; app.openapi()` (the exact
docs-job operation) now succeeds.
2026-06-02 12:11:55 +02:00
ruv 4856afbd0c ci(v1-api): run Performance Tests + API Docs with MOCK_POSE_DATA=true
After the DensePoseHead startup fix (#910), the v1 API starts, but the
Performance Tests load-hit the pose endpoints which error "requires real
CSI data" (no hardware in CI, mock_pose_data defaults False), and the
API-docs job imports the app the same way. Set MOCK_POSE_DATA=true on both
jobs so they exercise the mock path. Verified: the env var maps to
settings.mock_pose_data=True (pydantic, no env_prefix).

(Note: Performance Tests is continue-on-error so this is cleanup, not a
run-blocker; the run-level red on main has been transient Docker Hub pull
timeouts on Tests/docker-build, which are infra flakes that pass on re-run.)
2026-06-02 12:04:58 +02:00
rUv 4d205a05c4 Merge pull request #910 from ruvnet/fix/v1-pose-service-densepose-config
fix(v1-api): pass required config to DensePoseHead — green main CI
2026-06-02 05:50:25 -04:00
ruv bc42ae7903 fix(v1-api): pass required config to DensePoseHead — green main CI
The "Continuous Integration" workflow (Performance Tests + API
Documentation jobs) has failed on every main commit since the API start
path was exercised: pose_service._initialize_models() called
`DensePoseHead()` with no args, but DensePoseHead.__init__ requires a
config dict → "TypeError: DensePoseHead.__init__() missing 1 required
positional argument: 'config'" → uvicorn "Application startup failed".

Pass a config: input_channels=256 (matches the modality translator's
output), num_body_parts=24 (DensePose standard), num_uv_coordinates=2.
Both call sites (with/without pose_model_path) fixed.

Verified locally: DensePoseHead(config) + ModalityTranslationNetwork(config)
both construct + eval, clearing the startup TypeError.
2026-06-02 11:42:52 +02:00
6 changed files with 55 additions and 18 deletions
+35 -8
View File
@@ -265,23 +265,45 @@ jobs:
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install locust
pip install pytest # the perf suite is pytest, not locust
- name: Start application
working-directory: archive/v1
run: |
uvicorn src.api.main:app --host 0.0.0.0 --port 8000 &
sleep 10
# No "Start application" step: the gated test (test_frame_budget.py) drives
# the CSIProcessor pipeline in-process and makes no HTTP calls, so the old
# uvicorn server + `sleep 10` were dead weight — they only existed for the
# now-excluded api_throughput/inference_speed tests, and on every run dumped
# ~50 misleading "router requires hardware setup" ERROR lines for a server
# no test touched. MOCK_POSE_DATA is server-only and unused here.
- name: Run performance tests
working-directory: archive/v1
run: |
locust -f tests/performance/locustfile.py --headless --users 50 --spawn-rate 5 --run-time 60s --host http://localhost:8000
# Gate only on the genuine, deterministic perf guard:
# test_frame_budget.py times the *real* CSIProcessor pipeline against
# the ADR 50 ms per-frame budget (single-frame, p95 over 100 frames,
# +Doppler) — a true regression signal.
#
# test_api_throughput.py / test_inference_speed.py are excluded: every
# test there is a TDD red-phase stub (suffix `_should_fail_initially`)
# that times a *mock that sleeps* — meaningless as a perf signal, with
# machine-dependent wall-clock asserts (e.g. `actual_rps >= 40`,
# `batch_time < individual_time`) that are inherently flaky on shared
# CI runners, plus a cross-class fixture-scope bug. Forcing them green
# would be manufacturing a false signal; they stay in-repo for local
# TDD but do not gate CI until the underlying features are implemented.
#
# `python -m pytest` (not the bare `pytest` script) puts the cwd
# (archive/v1) on sys.path so `from src.core...` resolves — the bare
# script omits cwd and raises ModuleNotFoundError: No module named 'src'.
# -o addopts="" drops the root pyproject's --cov/--cov-fail-under=100.
python -m pytest tests/performance/test_frame_budget.py \
-o addopts="" -v --junitxml=perf-junit.xml
- name: Upload performance results
if: always()
uses: actions/upload-artifact@v4
with:
name: performance-results
path: locust_report.html
path: archive/v1/perf-junit.xml
# Docker Build and Test
# NOTE: the canonical Docker build for the sensing-server is now
@@ -367,6 +389,8 @@ jobs:
runs-on: ubuntu-latest
needs: [docker-build]
if: github.ref == 'refs/heads/main'
permissions:
contents: write # gh-pages deploy needs write (GITHUB_TOKEN is read-only by default -> 403)
steps:
- name: Checkout code
uses: actions/checkout@v4
@@ -384,6 +408,8 @@ jobs:
- name: Generate OpenAPI spec
working-directory: archive/v1
env:
MOCK_POSE_DATA: "true" # no CSI hardware in CI
run: |
python -c "
from src.api.main import app
@@ -394,6 +420,7 @@ jobs:
- name: Deploy to GitHub Pages
uses: peaceiris/actions-gh-pages@v4
continue-on-error: true # openapi generation above is the real validation; deploy is best-effort (Pages may be disabled)
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./docs
+1 -1
View File
@@ -430,7 +430,7 @@ Model release (no new firmware binary). Firmware remains at v0.6.0-esp32.
- Security fix merged via PR #310.
### Performance
- Presence detection: 100% accuracy on 60,630 overnight samples.
- Presence detection: 100% accuracy on 60,630 overnight samples. *(Retracted — that recording was single-class (one sleeping person, 6,062/6,063 frames "present"), so a constant "yes" scores ~99.98%. Superseded by the honest 82.3% held-out temporal-triplet metric; see [#882](https://github.com/ruvnet/RuView/issues/882). Kept here as the in-place public record.)*
- Inference: 0.008 ms per sample, 164K embeddings/sec.
- Contrastive self-supervised training: 51.6% improvement over baseline.
+12 -3
View File
@@ -107,16 +107,25 @@ class PoseService:
async def _initialize_models(self):
"""Initialize neural network models."""
try:
# Initialize DensePose model
# Initialize DensePose model. DensePoseHead requires a config
# dict — input_channels matches the modality translator's output
# (256), with the standard DensePose 24 body parts and 2 (U,V)
# coordinates. (Previously called with no args → TypeError at
# startup, which broke the API service.)
densepose_config = {
'input_channels': 256,
'num_body_parts': 24,
'num_uv_coordinates': 2,
}
if self.settings.pose_model_path:
self.densepose_model = DensePoseHead()
self.densepose_model = DensePoseHead(densepose_config)
# Load model weights if path is provided
# model_state = torch.load(self.settings.pose_model_path)
# self.densepose_model.load_state_dict(model_state)
self.logger.info("DensePose model loaded")
else:
self.logger.warning("No pose model path provided, using default model")
self.densepose_model = DensePoseHead()
self.densepose_model = DensePoseHead(densepose_config)
# Initialize modality translation
config = {
+3 -3
View File
@@ -122,7 +122,7 @@ node scripts/benchmark-ruvllm.js --model models/csi-ruvllm # benchmark
| What we measured | Result | Why it matters |
|-----------------|--------|---------------|
| **Presence detection** | **100% accuracy** | Never misses a person, never false alarms |
| **CSI embedding quality** | **82.3% held-out temporal-triplet** | Honest label-free metric on the last 20% by time (v1's "100% presence" was a single-class recording — retracted, [#882](https://github.com/ruvnet/RuView/issues/882)) |
| **Inference speed** | **0.008 ms** per embedding | 125,000x faster than real-time |
| **Throughput** | **164,183 embeddings/sec** | One Mac Mini handles 1,600+ ESP32 nodes |
| **Contrastive learning** | **51.6% improvement** | Strong pattern learning from real overnight data |
@@ -233,7 +233,7 @@ python firmware/esp32-csi-node/provision.py --port COM9 --hop-channels "1,6,11"
| **kNN similarity search** | "Find the 10 most similar states to right now" — anomaly detection, fingerprinting | Cognitum Seed |
| **Witness chain** | SHA-256 tamper-evident audit trail for every measurement (1,747 entries validated) | Cognitum Seed |
| **Camera-free pose training** | 17 COCO keypoints from 10 sensor signals — PIR, RSSI triangulation, subcarrier asymmetry, vibration, BME280 | 2x ESP32 + Seed |
| **Pre-trained model** | 82.8 KB (8 KB at 4-bit quantization), 100% presence accuracy, 0 skeleton violations | Download from release |
| **Pre-trained model** | 82.8 KB (8 KB at 4-bit quantization), 82.3% held-out temporal-triplet accuracy (v1's "100% presence" was single-class — retracted, [#882](https://github.com/ruvnet/RuView/issues/882)) | Download from release |
| **Sub-ms inference** | 0.012 ms latency, 171,472 embeddings/sec on M4 Pro | Any machine with Node.js |
| **SONA adaptation** | Adapts to new rooms in <1ms without retraining | ruvllm runtime |
| **LoRA room adapters** | Per-node fine-tuning with 2,048 parameters per adapter | Automatic |
@@ -262,7 +262,7 @@ node scripts/benchmark-ruvllm.js --model models/csi-ruvllm
| What we measured | Result | Why it matters |
|-----------------|--------|---------------|
| **Presence detection** | **100% accuracy** | Never misses a person, never false alarms |
| **CSI embedding quality** | **82.3% held-out temporal-triplet** | Honest label-free metric (v1's "100% presence" was single-class — retracted, [#882](https://github.com/ruvnet/RuView/issues/882)) |
| **Person counting** | **24/24 correct** (MinCut) | Fixed the #1 user-reported issue |
| **Inference speed** | **0.012 ms** per embedding | 83,000x faster than real-time |
| **Throughput** | **171,472 embeddings/sec** | One Mac Mini handles 1,700+ ESP32 nodes |
+3 -3
View File
@@ -1119,7 +1119,7 @@ What it ships (and what it does not):
| Capability | Status |
|------------|--------|
| Presence detection (occupied / empty) | ✅ Trained head — 100% accuracy on validation |
| Presence detection (occupied / empty) | ✅ Trained head — v2 encoder reports 82.3% held-out temporal-triplet acc (v1's "100% on validation" was a single-class recording — retracted, [#882](https://github.com/ruvnet/RuView/issues/882)) |
| 128-dim CSI embeddings (re-ID, similarity, downstream training) | ✅ Trained encoder |
| Single-person breathing / heart-rate | ⚠️ Server still uses heuristic DSP — model does not replace this yet |
| 17-keypoint full-body pose | 🔬 No keypoint weights shipped yet — pose pipeline runs but without a learned head |
@@ -1824,7 +1824,7 @@ huggingface-cli download ruvnet/wifi-densepose-pretrained --local-dir models/pre
# model.safetensors — 48 KB contrastive encoder
# model-q4.bin — 8 KB quantized (recommended)
# model-q2.bin — 4 KB ultra-compact (ESP32 edge)
# presence-head.json — presence detection head (100% accuracy)
# presence-head.json — presence detection head (v2 encoder: 82.3% held-out triplet acc)
# node-1.json — LoRA adapter for room 1
# node-2.json — LoRA adapter for room 2
```
@@ -1833,7 +1833,7 @@ huggingface-cli download ruvnet/wifi-densepose-pretrained --local-dir models/pre
The pre-trained encoder converts 8-dim CSI feature vectors into 128-dim embeddings. These embeddings power all 17 sensing applications:
- **Presence detection** — 100% accuracy, never misses, never false alarms
- **Presence detection** — v2 encoder: 82.3% held-out temporal-triplet accuracy (v1's "100%" was a single-class recording — retracted, [#882](https://github.com/ruvnet/RuView/issues/882))
- **Environment fingerprinting** — kNN search finds "states like this one"
- **Anomaly detection** — embeddings that don't match known clusters = anomaly
- **Activity classification** — different activities cluster in embedding space
+1
View File
@@ -36,3 +36,4 @@ scikit-learn>=1.2.0
# Monitoring dependencies
prometheus-client>=0.16.0
psutil>=5.9.0 # system metrics — imported by health.py / metrics.py / status.py / monitoring.py