Browser sign-in worked, but the settings panel kept offering "Sign in with
Cognitum" afterwards; only a hard reload showed the true state. The server was
correct throughout.
Root cause: `ui/sw.js` routed cache-first as its CATCH-ALL for every path
outside `/api/` and `/health/`. `/oauth/status` therefore had its first
(signed-out) response stored in the Cache API and replayed to the page forever.
The Cache API is not the HTTP cache and ignores `Cache-Control` entirely, so the
`no-store, no-cache, must-revalidate` the server already sends could not prevent
it. A hard reload bypasses the service worker, which is why that alone appeared
to fix it, and why signing out re-poisoned the entry.
Fixes, in order of blast radius:
- `/oauth/` is never handled by the worker. Not network-first — that still
writes a copy, which would be replayed the moment the server is briefly
unreachable, silently reinstating a stale sign-in state.
- Cache-first is now an ALLOWLIST (navigations and static asset extensions)
rather than the catch-all. This is the underlying defect: any endpoint added
outside `/api/` was frozen on its first response. Unrecognised paths now go
to the network untouched.
- `CACHE_NAME` bumped to `ruview-v2`, so `activate` evicts the poisoned
`ruview-v1` from browsers that already ran the old worker. Verified: the
cache list went from `[ruview-v1]` to `[ruview-v2]` on update.
- Shell lookups use `ignoreSearch` and store a search-less key, so the
`?signed_in=<ms>` the callback redirects to does not mint a fresh, never-hit
cache entry per sign-in.
Also: re-check sign-in state on `pageshow` (bfcache restore, where no script
re-runs) and on `visibilitychange` (signed in or out in another tab). Opening
the panel alone is not sufficient — it may already be open.
Verification, in a real browser driven end-to-end:
- Reproduced with a valid session cookie: server returned `signed_in: true` to
curl while the page's own fetch got a cached `signed_in: false`, and the
request never appeared in the server log at all.
- Confirmed the cached entry existed: `caches.open('ruview-v1').match(
'/oauth/status')` returned the signed-out body.
- After the fix, on a NORMAL (not hard) reload the panel reads
"Signed in as <account> - sensing:read" with Sign out shown.
Tests: `ui/sw.test.mjs` loads the real `sw.js` with stubbed worker globals and
asserts the routing decision per path. 4 of its 10 tests fail against the
pre-fix worker and pass after; the other 6 pin pre-existing guards (non-GET,
websocket upgrade, cross-origin, API paths, static assets, navigation) and pass
in both, so they track behaviour rather than the rewrite.
CI: adds a `ui-tests` job. Nothing ran the UI JavaScript before, so both this
suite and the ADR-272 ws-ticket suite would have rotted unexecuted — which is
the same blind spot that let this defect ship.
Removes the temporary `/oauth/status` cookie logging and the panel console
diagnostic added while tracking this down.
Co-Authored-By: Ruflo & AQE
`refreshSignInPanel` was exported and never invoked. The Cognitum Account panel
would have rendered "Checking…" forever and the Sign in button would never have
reflected a completed sign-in.
That is the FOURTH defect of this exact shape today — implemented, unit-tested,
and never called. The others: the refresh path no command invoked, scope
resolved only at write time, and a WebSocket listener with no auth layer. The
pattern is consistent: the logic was fine, the caller was missing, and a green
suite could not see it.
Now refreshed on every panel open — not once at construction — because the
session may have been established in another tab or expired since page load.
Buttons are also bound at construction so a click works before the first status
fetch resolves.
VERIFIED END TO END IN A REAL BROWSER (Chrome, http://127.0.0.1:8099/ui/), which
is the gap that has been open all session:
1. panel showed "This server requires sign-in." + Sign in with Cognitum
2. clicked -> auth.cognitum.one -> approved
3. server: "browser sign-in complete" sub=ed3efb51-…
4. DevTools: ruview_session cookie on 127.0.0.1, Path=/, HttpOnly, SameSite=Lax
5. after reload the panel reads:
"Signed in as UyShaIh1B1gL7km9Xli9kCDSdRT2 — sensing:read" + Sign out
So the browser half of gate G-3 is now proven, not argued: PKCE, consent, the
server-side code exchange, ES256 verification with audience and scope checks,
the signed session cookie, and the UI reading it back.
Diagnosis note worth keeping: the first attempt appeared to fail because the
browser tab had been open since BEFORE the fix — an already-loaded ES module
stays in memory regardless of `Cache-Control: no-store`. The cookie had been
stored correctly the whole time; the panel simply never re-queried. The server
log showed zero /oauth/status probes, which is what distinguished "fetch never
fired" from "fetch got the wrong answer". Temporary cookie diagnostics added to
find that are removed here.
Tests: 547 sensing-server lib, unchanged.
Co-Authored-By: Ruflo & AQE
Three loose ends from the browser sign-in work.
--- 1. The last mile: a button ---
The server endpoints existed but nothing in ui/ linked to them, so the feature
was unreachable. QuickSettings gains a "Cognitum Account" panel that renders
from `GET /oauth/status` and offers Sign in / Sign out.
`/oauth/status` is a new endpoint and is deliberately UNGATED — a signed-OUT
browser cannot ask a gated API whether sign-in is available. It returns only
capability flags and, when a session exists, who it belongs to. Never a
credential.
The panel distinguishes four states rather than showing a button that might
404: signed in; sign-in available; auth on but OAuth off (points at the
existing static-token panel); no auth required. A 404 from /oauth/status means
a server predating this work and says so plainly.
Sign-in is a full-page navigation, not fetch(): the server replies 302 and the
browser must follow it carrying the transaction cookie. An XHR would follow the
redirect invisibly and land nowhere.
--- 2. RUVIEW_SESSION_SECRET no longer required ---
Previously `/oauth/start` returned 503 unless an operator invented a secret —
a footgun, since they set RUVIEW_OAUTH_ISSUER, expect sign-in, and get a 503
naming an env var they have never heard of.
Now resolved in order: env var, then `<data_dir>/session-secret`, then generate
one and persist it 0600 (temp file, chmod before rename — the discipline used
for the CLI's credentials). Persisted rather than in-memory so a restart does
not silently sign everyone out.
The env var still wins, which is what a multi-instance deployment needs: several
servers must share a secret or a session issued by one is rejected by the next.
If the file cannot be written we log the reason and continue with an in-memory
secret rather than refusing sign-in outright.
Verified with NO configuration at all: secret generated, file mode 0600,
/oauth/start returns 302, /oauth/status reports browser_signin: true.
--- 3. The JavaScript is now executed by tests ---
`ui/services/ws-ticket.test.mjs`, 9 tests, node:test built-in — no new
dependency and no package.json needed. Run: `node --test ui/services/`.
Covers: no token means no fetch and an unchanged URL; the bearer reaches the
Authorization header and NEVER the URL; `?` vs `&` when a query already exists;
ticket URL-encoding; 404 treated as a pre-ADR-272 server (the property that
lets one UI work against old and new servers, and therefore lets the legacy
escape hatch be removed later); 503 and network failure swallowed rather than
breaking the connect path; and that a fresh ticket is minted per call, since
tickets are single-use and caching one fails on the second reconnect.
STATED PRECISELY, because the distinction matters: this EXECUTES the module in
Node with stubbed fetch/localStorage. It is more than the `node --check` it
replaces and less than a browser — no real WebSocket upgrade, no real cookies,
no page wiring. "The UI JavaScript has never been run" is no longer true of
this module. "Browser-tested" still is not.
Tests: 547 sensing-server lib, 5 wiring integration, 87 ruview-auth, 9 JS.
Co-Authored-By: Ruflo & AQE
Completes ADR-272. Three parts.
1. PREFIX MATCHING, not an allowlist. Anything under `/ws/` is a WebSocket path,
plus `/api/v1/stream/pose` which lives outside it. An allowlist means every
WebSocket route added later ships ungated until someone remembers to extend
it — the same bug reintroduced on a delay. Not hypothetical:
`/ws/train/progress` (ADR-186, arriving with PR #1387) is ALREADY referenced
by ui/services/training.service.js and would have shipped unauthenticated.
Pinned by a test that asserts it is gated before it exists.
2. UI wiring. A shared `withWsTicket()` helper mints a ticket immediately before
each connection attempt — never cached, because a ticket is single-use and
expires in seconds, so reusing one across reconnects fails on the second
attempt. Wired into the three sites that open gated sockets:
sensing.service.js, websocket-client.js, observatory/js/main.js.
It degrades in both directions on purpose: no stored token means auth is off
and no ticket is needed; a 404 from /api/v1/ws-ticket means a server
predating this ADR, which still exempts WebSockets, so connecting without a
ticket is correct there. The same UI therefore works against old and new
servers, which is what makes the escape hatch removable later rather than
permanent.
The long-lived bearer token is still never put in a URL — only the ticket is.
3. ADR-272 itself. Previously cited in five places without existing; the same
dangling-reference mistake made with ADR-271 earlier today, so it is written
before this lands rather than after someone notices.
It records the measured before/after, why a credential in a URL is
acceptable here specifically (single use, seconds-long, not the credential),
why the escape hatch exists and why it is deliberately uncomfortable, and
what is deliberately NOT done — including that /health/metrics stays ungated,
with the caveat that this should be revisited if metrics ever carry
occupancy-derived values, since that would make them sensing data wearing an
ops label.
Tests: 526 sensing-server (4 new path-matching), 82 ruview-auth. JS
syntax-checked with `node --check`; there is no UI test suite to extend.
Co-Authored-By: Ruflo & AQE
- sdkconfig.defaults.devkitc: header build command idf v5.2 -> v5.4
(source needs esp_driver_uart, IDF >=5.3 — same reason the PR fixed
the README refs)
- engine/lib.rs: separate doc comments — set_room_adapter's ADR-150
adapter/witness doc had merged into set_multistatic_config's doc
- ui: apply stored bearer token at api.service.js module load instead
of QuickSettings.init — services + dashboard tab dispatch their first
/api/v1/* requests before initializeEnhancements() runs, so the first
requests on every load went out without the Authorization header
- CHANGELOG: Unreleased entries for #1308/#1309/#1310
Co-Authored-By: claude-flow <ruv@ruv.net>
Browsers cannot attach an Authorization header to a WebSocket upgrade,
so with RUVIEW_API_TOKEN set the Live Demo pose stream at
/api/v1/stream/pose always failed with 401 — the same reason
/ws/sensing is already exempted (see bearer_auth module docs). Adds a
narrow EXEMPT_PATHS list plus a regression test that the exemption
does not leak to other /api/v1/* paths. Query-string tokens remain
rejected (CWE-598 test untouched).
Also adds an 'API Access' bearer-token field to the QuickSettings
panel: ui/services/api.service.js had setAuthToken() but nothing ever
called it, so enabling RUVIEW_API_TOKEN broke every /api/v1/* call
from the bundled dashboard. The token is stored in localStorage and
applied before the first request.
Fixes#1310
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix(ui): unbreak viz.html — OrbitControls importmap, WS URL, toast NPE (#760)
Three independent bugs were stacking to make ui/viz.html unusable from `main`:
1. Three.js r160 removed `examples/js/OrbitControls.js`, so the script-tag
load 404'd and `new THREE.OrbitControls(...)` threw. Switch to an
importmap that pulls the ES module build, then re-expose
`window.THREE` and `THREE.OrbitControls` so the existing component
modules (scene.js, body-model.js, …) keep working without a wider
refactor.
2. The WebSocket client was hardcoded to `ws://localhost:8000/ws/pose`,
but the sensing-server listens on `--ws-port` (8765 default, 3001 in
the Docker image) at `/ws/sensing`. Reuse the existing
`buildSensingWsUrl()` helper from `sensing.service.js` so port
pairings are handled centrally, and add a `?ws=…` query-string
override for non-standard setups. The websocket-client.js default is
also updated to derive from `window.location` instead of the dead
`:8000/ws/pose` literal.
3. `ToastManager.show()` called `this.container.appendChild(...)` even
when `init()` had never been called, throwing a TypeError that
killed the rest of page initialization. Auto-init the container
lazily on first show (patch from issue reporter).
Closes#760.
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix(ui): single module script + mutable THREE — OrbitControls validated
Browser validation against the previous commit caught two stacked issues:
1. `import * as THREE from 'three'` returns a frozen Module Namespace
Object — assignment `THREE.OrbitControls = OrbitControls` silently
no-ops, so the global never gets the OrbitControls reference.
2. Two separate `<script type="module">` blocks (one installing the
THREE global, one consuming it via Scene) are independently
async-resolved. The second can finish dependency loading first and
call `new THREE.OrbitControls(...)` before the first script has run.
Fixed by spreading the namespace into a plain mutable object and merging
all initialization into a single module script with `await import()` for
component modules. Order is now strictly: import THREE → install
window.THREE → import components → run init().
Validated via agent-browser: page logs `[VIZ] Initialization complete`,
WebSocket targets the correct `ws://127.0.0.1:3001/ws/sensing` endpoint
(derived from buildSensingWsUrl), toast lazy-init confirmed via eval.
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat(ui): add keyboard shortcuts, perf monitor, toast system, theme toggle, and WCAG accessibility
- Keyboard shortcuts overlay (press ? for help, 1-8 for tabs, T for theme, P for perf)
- Real-time performance monitor with FPS, memory, latency sparklines (draggable)
- Enhanced toast notification system with stacking, auto-dismiss, progress bars
- Dark/light theme toggle with localStorage persistence and system preference detection
- WCAG accessibility: skip-to-content link, ARIA roles/attributes on tabs and panels,
arrow key navigation in tab bar, focus-visible outlines
- ESLint config for UI directory with security and quality rules
* feat(ui): add command palette, activity log, data export, fullscreen mode, connection status
- Command palette (Ctrl+K / Cmd+K) with fuzzy search across tabs and actions
- Activity log panel (L key) with real-time console interception, filters, resizable
- Data export utility (E key) for sensor data as JSON/CSV with dialog
- Fullscreen mode (F key / F11) for visualization tabs with exit button
- Connection status widget in header showing WebSocket state and reconnect
* feat(ui): add mobile hamburger nav, PWA support, and 40 unit tests
- Mobile hamburger navigation: slide-out drawer replacing tab bar on <768px,
swipe-to-close, animated hamburger icon, auto-sync with tab manager
- PWA manifest + service worker: installable dashboard, offline shell caching
(cache-first for static, network-first for API), auto-cleanup of old caches
- 40 unit tests for ToastManager, ThemeToggle, KeyboardShortcuts, PerfMonitor,
TabManager - browser-based test runner at ui/tests/unit-tests.html
- PWA meta tags: theme-color, apple-mobile-web-app-capable, manifest link
- Icon generator page for creating PWA icons (ui/icons/generate.html)
* feat(ui): add URL routing, onboarding tour, idle detection, notification center
- Hash router: tabs are bookmarkable/shareable via URL (#demo, #sensing, etc.),
syncs with TabManager, supports browser back/forward navigation
- Onboarding tour: interactive 6-step first-run walkthrough with spotlight
highlighting, step indicators, skip/back/next controls, localStorage persistence
- Idle detection: pauses health polling and reduces CSS animations after 3 min
of inactivity, resumes on user interaction, integrates with Page Visibility API
- Notification center: bell icon in header with unread badge, event history panel
with mark-read/clear, persists across page views via sessionStorage
* feat(ui): add i18n (EN/PL), screenshot tool, settings panel, reduced motion, uptime clock
- i18n: English/Polish translations with auto-detection, language selector
in header, data-i18n attributes on dashboard elements, localStorage persistence
- Screenshot tool (S key): captures active tab to clipboard or downloads PNG,
flash effect, canvas rendering with watermark, fallback for tainted canvases
- Quick settings panel (gear icon): reduced motion toggle, high contrast mode,
compact layout mode, health polling toggle, clear data, reset onboarding
- Uptime clock: current time + session duration in header
- prefers-reduced-motion: system-level and manual toggle, disables all
animations and transitions for vestibular accessibility
- High contrast mode: WCAG AAA compliant colors for both light and dark themes
- Compact mode: condensed layout for dense information display
When two render frames land in the same performance.now() tick,
`currentTime - lastFrameTime === 0`, so `fps = 1000 / 0 = Infinity`,
and `averageFps = averageFps * 0.9 + Infinity * 0.1 = Infinity` poisons
the EMA forever after a single zero-dt tick. The UI then displays
"Infinity FPS" until reload.
Floor deltaTime at 1 ms before the division. That caps displayed FPS at
1000 (far above any real render rate so the cap is never observed in
practice) but keeps the EMA finite.
Reported in #519 ("Bug 2 — FPS shows Infinity") by @kapilsoni2013 on a
3-node ESP32-S3-WROOM multi-node setup with edge-tier 1 + 2.
* security: pin GitHub Actions to SHAs and bump vulnerable npm deps (#442)
Addresses confirmed findings from issue #442 (Pentesterra/DevGuard).
GitHub Actions — pin all third-party Action references in
security-scan.yml and ci.yml to verified commit SHAs (with the
matching version in a trailing comment for legibility):
* snyk/actions/python -> v1.0.0
* aquasecurity/trivy-action -> v0.36.0 (security-scan.yml + ci.yml)
* bridgecrewio/checkov-action -> v12.1347.0
* tenable/terrascan-action -> v1.4.1
* checkmarx/kics-github-action -> v2.1.20 (the action #442 named)
* trufflesecurity/trufflehog -> v3.95.2
Verification:
grep -rE 'uses:.*@(main|master|latest)$' .github/workflows/
returns no matches.
npm deps in ui/mobile — add `overrides` forcing patched versions of
the three packages flagged by the DevGuard scanner, regenerate
package-lock.json:
* @xmldom/xmldom@0.8.11 -> 0.8.13
* node-forge@1.3.3 -> ^1.4.0 (closes 3 HIGH advisories)
* picomatch@2.3.1 -> ^2.3.2 (transitive in jest tooling)
npm audit totals: 25 -> 22 advisories (5 HIGH -> 2 HIGH).
Out of scope for this PR (tracked separately):
* Sensing-server unauth REST API surface — opened as #443
pending design-intent confirmation from @ruvnet.
* Bearer-token-shaped string in git history — confirmed test
seed per repo owner; no rotation required.
Refs: #442
Co-Authored-By: claude-flow <ruv@ruv.net>
* chore: add Dependabot config for github-actions and ui/mobile npm (#442)
Pairs with the SHA pinning from the previous commit so the pinned
versions get automated weekly bumps rather than drifting back to
mutable refs over time.
Scoped to the two ecosystems #442 surfaced findings in:
* github-actions (root) — the supply-chain risk
* npm (ui/mobile) — the @xmldom/xmldom, node-forge, picomatch
advisories
Other ecosystems (pip, cargo, desktop UI npm) deliberately omitted —
they can be added in a separate PR if desired.
Refs: #442
Co-Authored-By: claude-flow <ruv@ruv.net>
* chore(dependabot): expand to pip, cargo, and desktop UI npm (#442)
Broadens the Dependabot config from the initial 2 ecosystems
(github-actions + ui/mobile npm) to cover all 5 package surfaces
in the repo so pinned dependencies stay current across the board:
+ npm /v2/crates/wifi-densepose-desktop/ui (vite advisory live)
+ pip / (requirements.txt loose pins)
+ cargo /v2 (no cargo audit in CI yet)
Marginal cost is zero — Dependabot only opens PRs when an upstream
bump exists, and per-ecosystem pull-request limits cap the noise.
Each ecosystem labelled distinctly so PRs route cleanly.
Refs: #442
Co-Authored-By: claude-flow <ruv@ruv.net>
---------
Co-authored-by: claude-flow <ruv@ruv.net>
The Rust port lived two directories deep (rust-port/wifi-densepose-rs/)
without any sibling under rust-port/ that warranted the extra level.
Move the whole workspace up to v2/ to match v1/ (Python) at the same
depth and shorten every cd / build command across the repo.
git mv preserves history for all tracked files. 60 files updated for
path references (CI workflows, ADRs, docs, scripts, READMEs, internal
.claude-flow state). Two manual fixes for relative-cd paths in
CLAUDE.md and ADR-043 that became wrong after the depth change
(cd ../.. → cd ..).
Validated:
- cargo check --workspace --no-default-features → clean (after target/
nuke; the gitignored target/ was carried by the OS rename and had
hard-coded old paths in build scripts)
- cargo test --workspace --no-default-features → 1,539 passed, 0 failed,
8 ignored (same totals as pre-rename)
- ESP32-S3 on COM7 → still streaming live CSI (cb #40300, RSSI -64 dBm)
After-merge follow-up: contributors should `rm -rf v2/target` once and
let cargo regenerate from the new path.
Address all 5 P0 issues from QE analysis (55/100 score):
- P0-1: Rate limiter bypass — validate X-Forwarded-For against trusted proxy list
- P0-2: Exception detail leak — generic 500 messages, exception_type gated by dev mode
- P0-3: WebSocket JWT in URL (CWE-598) — first-message auth pattern replaces query param
- P0-4: Rust tests not in CI — add rust-tests job gating docker-build and notify
- P0-5: WebSocket path mismatch — use WS_PATH constant instead of hardcoded /ws/sensing
Includes ADR-080 remediation plan and 9 QE reports (4,914 lines).
Firmware validated on ESP32-S3 (COM8): CSI collecting, calibration OK.
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat(server): cross-node RSSI-weighted feature fusion + benchmarks
Adds fuse_multi_node_features() that combines CSI features across all
active ESP32 nodes using RSSI-based weighting (closer node = higher weight).
Benchmark results (2 ESP32 nodes, 30s, ~1500 frames):
Metric | Baseline | Fusion | Improvement
---------------------|----------|---------|------------
Variance mean | 109.4 | 77.6 | -29% noise
Variance std | 154.1 | 105.4 | -32% stability
Confidence | 0.643 | 0.686 | +7%
Keypoint spread std | 4.5 | 1.3 | -72% jitter
Presence ratio | 93.4% | 94.6% | +1.3pp
Person count still fluctuates near threshold — tracked as known issue.
Verified on real hardware: COM6 (node 1) + COM9 (node 2) on ruv.net.
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix(ui): add client-side lerp smoothing to pose renderer
Keypoints now interpolate between frames (alpha=0.25) instead of
jumping directly to new positions. This eliminates visual jitter
that persists even with server-side EMA smoothing, because the
renderer was drawing every WebSocket frame at full rate.
Applied to skeleton, keypoints, and dense body rendering paths.
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat: DynamicMinCut person separation + UI lerp smoothing
- Added ruvector-mincut dependency to sensing server
- Replaced variance-based person scoring with actual graph min-cut on
subcarrier temporal correlation matrix (Pearson correlation edges,
DynamicMinCut exact max-flow)
- Recalibrated feature scaling for real ESP32 data ranges
- UI: client-side lerp interpolation (alpha=0.25) on keypoint positions
- Dampened procedural animation (noise, stride, extremity jitter)
- Person count thresholds retuned for mincut ratio
Co-Authored-By: claude-flow <ruv@ruv.net>
* docs: update CHANGELOG with v0.5.1-v0.5.3 releases
Co-Authored-By: claude-flow <ruv@ruv.net>
Complements #326 (per-node state pipeline) with additional features:
- Dynamic adaptive classifier: discover activity classes from training
data filenames instead of hardcoded array. Users add classes via
filename convention (train_<class>_<desc>.jsonl), no code changes.
- Per-node UI cards: SensingTab shows individual node status with
color-coded markers, RSSI, variance, and classification per node.
- Colored node markers in 3D gaussian splat view (8-color palette).
- Per-node RSSI history tracking in sensing service.
- XSS fix: UI uses createElement/textContent instead of innerHTML.
- RSSI sign fix: ensure dBm values are always negative.
- GET /api/v1/nodes endpoint for per-node health monitoring.
- node_features field in WebSocket SensingUpdate messages.
- Firmware watchdog fix: yield after every frame to prevent IDLE1 starvation.
Addresses #237, #276, #282
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add null-safe optional chaining for embPoints and rssiDbm in diagnostic log
- Handle Blob data in _handleLiveFrame (convert to ArrayBuffer before processing)
- Bump cache busters to v=13
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat: dual-modal WASM browser pose estimation demo (ADR-058)
Live webcam video + WiFi CSI fusion for real-time pose estimation.
Two parallel CNN pipelines (ruvector-cnn-wasm) with attention-weighted
fusion and dynamic confidence gating. Three modes: Dual, Video-only,
CSI-only. Includes pre-built WASM package (~52KB) for browser deployment.
- ADR-058: Dual-modal architecture design
- ui/pose-fusion.html: Main demo page with dark theme UI
- 7 JS modules: video-capture, csi-simulator, cnn-embedder, fusion-engine,
pose-decoder, canvas-renderer, main orchestrator
- Pre-built ruvector-cnn-wasm WASM package for browser
- CSI heatmap, embedding space visualization, latency metrics
- WebSocket support for live ESP32 CSI data
- Navigation link added to main dashboard
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix: motion-responsive skeleton + through-wall CSI tracking
- Pose decoder now uses per-cell motion grid to track actual arm/head
positions — raising arms moves the skeleton's arms, head follows
lateral movement
- Motion grid (10x8 cells) tracks intensity per body zone: head,
left/right arm upper/mid, legs
- Through-wall mode: when person exits frame, CSI maintains presence
with slow decay (~10s) and skeleton drifts in exit direction
- CSI simulator persists sensing after video loss, ghost pose renders
with decreasing confidence
- Reduced temporal smoothing (0.45) for faster response to movement
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix: video fills available space + correct WASM path resolution
- Remove fixed aspect-ratio and max-height from video panel so it
fills the available viewport space without scrolling
- Grid uses 1fr row for content area, overflow:hidden on main grid
- Fix WASM path: resolve relative to JS module file using import.meta.url
instead of hardcoded ./pkg/ which resolved incorrectly on gh-pages
- Responsive: mobile still gets aspect-ratio constraint
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat: live ESP32 CSI pipeline + auto-connect WebSocket
- Add auto-connect to local sensing server WebSocket (ws://localhost:8765)
- Demo shows "Live ESP32" when connected to real CSI data
- Add build_firmware.ps1 for native Windows ESP-IDF builds (no Docker)
- Add read_serial.ps1 for ESP32 serial monitor
Pipeline: ESP32 → UDP:5005 → sensing-server → WS:8765 → browser demo
Co-Authored-By: claude-flow <ruv@ruv.net>
* docs: add ADR-059 live ESP32 CSI pipeline + update README with demo links
- ADR-059: Documents end-to-end ESP32 → sensing server → browser pipeline
- README: Add dual-modal pose fusion demo link, update ADR count to 49
- References issue #245
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat: RSSI visualization, RuVector attention WASM, cache-bust fixes
- Add animated RSSI Signal Strength panel with sparkline history
- Fix RuVector WasmMultiHeadAttention retptr calling convention
- Wire up RuVector Multi-Head + Flash Attention in CNN embedder
- Add ambient temporal drift to CSI simulator for visible heatmap animation
- Fix embedding space projection (sparse projection replaces cancelling sum)
- Add auto-scaling to embedding space renderer
- Add cache busters (?v=4) to all ES module imports to prevent stale caches
- Add diagnostic logging for module version verification
- Add RSSI tracking with quality labels and color-coded dBm display
- Includes ruvector-attention-wasm v2.0.5 browser ESM wrapper
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat: 26-keypoint dexterous pose + full RuVector attention pipeline
Pose Decoder (17 → 26 keypoints):
- Add finger approximations: thumb, index, pinky per hand (6 new)
- Add toe tips: left/right foot index (2 new)
- Add neck keypoint (1 new)
- Hand openness driven by arm motion intensity
- Finger positions computed from wrist-elbow axis angles
CNN Embedder (full RuVector WASM pipeline):
- Stage 1: Multi-Head Attention (global spatial reasoning)
- Stage 2: Hyperbolic Attention (hierarchical body-part tree)
- Stage 3: MoE Attention (3 experts: upper/lower/extremities, top-2)
- Blended 40/30/30 weighting → final embedding projection
Canvas Renderer:
- Magenta finger joints with distinct glow
- Cyan toe tips
- White neck keypoint
- Thinner limb lines for hand/foot connections
- Joint count shown in overlay label
CSI Simulator:
- Skip synthetic person state when live ESP32 connected
- Only simulate CSI data in demo mode (was already correct)
Embedding Space:
- Fixed projection: sparse 8-dim projection replaces cancelling sum
- Auto-scaling normalizes point spread to fill canvas
Cache busters bumped to v=5 on all imports.
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix: centroid-based pose tracking for responsive limb movement
Rewrites pose decoder from intensity-based to position-based tracking:
- Arms now track toward motion centroid in each body zone
- Elbow/wrist positions computed along shoulder→centroid vector
- Legs track toward lower-body zone centroids
- Smoothing reduced from 0.45 to 0.25 for responsiveness
- Zone centroids blend 30% old / 70% new each frame
6 body zones with overlapping coverage:
- Head (top 20%, center cols)
- Left/Right Arm (rows 10-60%, outer cols)
- Torso (rows 15-55%, center cols)
- Left/Right Leg (rows 50-100%, half cols each)
Hand openness now driven by arm spread distance + raise amount.
Cache busters v=6.
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix: remove duplicate lAnkleX/rAnkleX declarations in pose-decoder
Stale code block from old intensity-based tracking was left behind,
re-declaring variables already defined by centroid-based tracking.
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat(demo): wire all 6 RuVector WASM attention mechanisms into pose fusion
- Add WasmLinearAttention and WasmLocalGlobalAttention to browser ESM wrapper
- Add 6 WASM utility functions (batch_normalize, pairwise_distances, etc.)
- Extend CnnEmbedder to 6-stage pipeline: Flash → MHA → Hyperbolic → Linear → MoE → L+G
- Use log-energy softmax blending across all 6 stages
- Wire WASM cosine_similarity and normalize into FusionEngine
- Add RuVector pipeline stats panel to UI (energy, refinement, pose impact)
- Compute embedding-to-joint mapping stats without modifying joint positions
- Center camera prompt with flexbox layout
- Add cache busters v=12
Co-Authored-By: claude-flow <ruv@ruv.net>
The ambient light color 0x446688 (dark blue-gray) was too dim to produce
visible brightness changes. Changed to 0xccccdd (bright neutral) with 5x
multiplier. Bumped SETTINGS_VERSION to force fresh defaults.
Co-Authored-By: claude-flow <ruv@ruv.net>
The ambient light was initialized with intensity * 3.0 but the slider
and preset callbacks set raw value without the multiplier, making the
setting appear to do nothing.
Co-Authored-By: claude-flow <ruv@ruv.net>
Add environment-tuned activity classification that learns from labeled
ESP32 CSI recordings, replacing brittle static thresholds.
- Adaptive classifier: 15-feature logistic regression trained from JSONL
recordings (variance, motion band, subcarrier stats: skew, kurtosis,
entropy, IQR). Trains in <1s, persists as JSON, auto-loads on restart.
- Three-stage signal smoothing: adaptive baseline subtraction (α=0.003),
EMA + trimmed-mean median filter (21-frame window), hysteresis debounce
(4 frames). Motion classification now stable across seconds, not frames.
- Vital signs stabilization: outlier rejection (±8 BPM HR, ±2 BPM BR),
trimmed mean, dead-band (±2 BPM HR), EMA α=0.02. HR holds steady for
10+ seconds instead of jumping 50 BPM every frame.
- Observatory auto-detect: always probes /health on startup, connects
WebSocket to live ESP32 data automatically.
- New API endpoints: POST /api/v1/adaptive/train, GET /adaptive/status,
POST /adaptive/unload for runtime model management.
- Updated user guide with Observatory, adaptive classifier tutorial,
signal smoothing docs, and new troubleshooting entries.
The web UI had persistent 404 errors on model, recording, and training
endpoints, and the sensing WebSocket never connected on Dashboard/Live
Demo tabs because sensingService.start() was only called lazily on
Sensing tab visit.
Server (main.rs):
- Add 14 fully-functional Axum handlers: model CRUD (7), recording
lifecycle (4), training control (3)
- Scan data/models/ and data/recordings/ at startup
- Recording writes CSI frames to .jsonl via tokio background task
- Model load/unload lifecycle with state tracking
Web UI (app.js):
- Import and start sensingService early in initializeServices() so
Dashboard and Live Demo tabs connect to /ws/sensing immediately
Mobile (ws.service.ts):
- Fix WebSocket URL builder to use same-origin port instead of
hardcoded port 3001
Mobile (jest.config.js):
- Fix testPathIgnorePatterns that was ignoring the entire test directory
Mobile (25 test files):
- Replace all it.todo() placeholder tests with real implementations
covering components, services, stores, hooks, screens, and utils
ADR-043 documents all changes.
* feat: RVF training pipeline & UI integration (ADR-036)
Implement full model training, management, and inference pipeline:
Backend (Rust):
- recording.rs: CSI recording API (start/stop/list/download/delete)
- model_manager.rs: RVF model loading, LoRA profile switching, model library
- training_api.rs: Training API with WebSocket progress streaming, simulated
training mode with realistic loss curves, auto-RVF export on completion
- main.rs: Wire new modules, recording hooks in all CSI paths, data dirs
UI (new components):
- ModelPanel.js: Dark-mode model library with load/unload, LoRA dropdown
- TrainingPanel.js: Recording controls, training config, live Canvas charts
- model.service.js: Model REST API client with events
- training.service.js: Training + recording API client with WebSocket progress
UI (enhancements):
- LiveDemoTab: Model selector, LoRA profile switcher, A/B split view toggle,
training quick-panel with 60s recording shortcut
- SettingsPanel: Full dark mode conversion (issue #92), model configuration
(device, threads, auto-load), training configuration (epochs, LR, patience)
- PoseDetectionCanvas: 10-frame pose trail with ghost keypoints and motion
trajectory lines, cyan trail toggle button
- pose.service.js: Model-inference confidence thresholds
UI (plumbing):
- index.html: Training tab (8th tab)
- app.js: Panel initialization and tab routing
- style.css: ~250 lines of training/model panel dark-mode styles
191 Rust tests pass, 0 failures. Closes#92.
Refs: ADR-036, #93
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix: real RuVector training pipeline + UI service fixes
Training pipeline (training_api.rs):
- Replace simulated training with real signal-based training loop
- Load actual CSI data from .csi.jsonl recordings or live frame history
- Extract 180 features per frame: subcarrier amplitudes, temporal variance,
Goertzel frequency analysis (9 bands), motion gradients, global stats
- Train calibrated linear CSI-to-pose mapping via mini-batch gradient descent
with L2 regularization (ridge regression), Xavier init, cosine LR decay
- Self-supervised: teacher targets from derive_pose_from_sensing() heuristics
- Real validation metrics: MSE and PCK@0.2 on 80/20 train/val split
- Export trained .rvf with real weights, feature normalization stats, witness
- Add infer_pose_from_model() for live inference from trained model
- 16 new tests covering features, training, inference, serialization
UI fixes:
- Fix double-URL bug in model.service.js and training.service.js
(buildApiUrl was called twice — once in service, once in apiService)
- Fix route paths to match Rust backend (/api/v1/train/*, /api/v1/recording/*)
- Fix request body formats (session_name, nested config object)
- Fix top-level await in LiveDemoTab.js blocking module graph
- Dynamic imports for ModelPanel/TrainingPanel in app.js
- Center nav tabs with flex-wrap for 8-tab layout
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix: WebSocket onOpen race condition, data source indicators, auto-start pose detection
- Fix WebSocket onOpen race condition in websocket.service.js where
setupEventHandlers replaced onopen after socket was already open,
preventing pose service from receiving connection signal
- Add 4-state data source indicator (LIVE/SIMULATED/RECONNECTING/OFFLINE)
across Dashboard, Sensing, and Live Demo tabs via sensing.service.js
- Add hot-plug ESP32 auto-detection in sensing server (auto mode runs
both UDP listener and simulation, switches on ESP32_TIMEOUT)
- Auto-start pose detection when backend is reachable
- Hide duplicate PoseDetectionCanvas controls when enableControls=false
- Add standalone Demo button in LiveDemoTab for offline animated demo
- Add data source banner and status styling
Co-Authored-By: claude-flow <ruv@ruv.net>
- All buttons converted to dark translucent style with colored accents:
Start (green), Stop (red), Reconnect (blue), Demo (purple)
- Header, wrapper, status badge all use dark backgrounds
- Controls in single flat row (no wrapping)
- Mode select dropdown styled for dark theme
- HTML entity icons on all buttons
Co-Authored-By: claude-flow <ruv@ruv.net>
- Heatmap: Gaussian radial blobs per keypoint with per-person hue,
faint skeleton overlay at 25% opacity
- Dense: body region segmentation with colored filled polygons for
head, torso, arms, legs — thick strokes + joint circles
- Keypoints: now also renders bounding box and confidence
- Previously both heatmap and dense were stubs falling back to skeleton
Co-Authored-By: claude-flow <ruv@ruv.net>
- Convert all Live Demo sidebar panels to dark theme matching rest of UI
- Fix pose_source not reaching LiveDemoTab (was lost in
convertZoneDataToRestFormat — now passes through from WS message)
- Dark backgrounds, muted text, consistent border opacity throughout
- Estimation Mode badge colors adjusted for dark background contrast
Co-Authored-By: claude-flow <ruv@ruv.net>
Reflects current state: Rust sensing server as primary backend,
sensing tab with 3D signal field, data source indicators, estimation
mode badge, setup guide, Docker deployment with CSI_SOURCE, and
updated file tree with all components/services.
Co-Authored-By: claude-flow <ruv@ruv.net>
- Docker default changed from --source simulated to --source auto
(auto-detects ESP32 on UDP 5005, falls back to simulation)
- Pose derivation now driven by real sensing features: motion_band_power,
breathing_band_power, variance, dominant_freq_hz, change_points
- Temporal feature extraction: 100-frame circular buffer, Goertzel
breathing rate estimation (0.1-0.5 Hz), frame-to-frame L2 motion
detection, SNR-based signal quality metric
- Signal field driven by subcarrier variance spatial mapping instead
of fixed animation circle
- UI data source indicators: LIVE/RECONNECTING/SIMULATED banner on
sensing tab, estimation mode badge on live demo tab
- Setup guide panel explaining ESP32 count requirements for each
capability level (1x: presence, 3x: localization, 4x+: full pose)
- Tick rate improved from 500ms to 100ms (2fps to 10fps)
- Fixed Option<f64> division bug from PR #83
- ADR-035 documents all decisions
Closes#86
Co-Authored-By: claude-flow <ruv@ruv.net>
- Added IosRssiService to handle synthetic RSSI data for iOS.
- Created WebRssiService to simulate RSSI scanning on the web.
- Defined shared types for WifiNetwork and RssiService in rssi.service.ts.
- Introduced simulation service to generate synthetic sensing data.
- Implemented WebSocket service for real-time data handling with reconnection logic.
- Established Zustand stores for managing application state related to MAT and pose data.
- Developed theme context and utility functions for consistent styling and formatting.
- Added type definitions for various application entities including API responses and sensing data.
- Created utility functions for color mapping and URL validation.
- Configured TypeScript settings for the mobile application.
The UI had hardcoded localhost:8080 for HTTP and localhost:8765 for
WebSocket, causing "Backend unavailable" when served from Docker
(port 3000) or any non-default port.
Changes:
- api.config.js: BASE_URL now uses window.location.origin instead
of hardcoded localhost:8080
- api.config.js: buildWsUrl() uses window.location.host instead of
hardcoded localhost:8080
- sensing.service.js: WebSocket URL derived from page origin instead
of hardcoded localhost:8765
- main.rs: Added /ws/sensing route to the HTTP server so WebSocket
and REST are reachable on a single port
Fixes#55
Co-Authored-By: claude-flow <ruv@ruv.net>
process.env does not exist in vanilla browser ES modules (no bundler).
Use window.location.protocol check only for WSS detection.
Co-Authored-By: claude-flow <ruv@ruv.net>
Replace Python FastAPI + WebSocket servers with a single 2.1MB Rust binary
(wifi-densepose-sensing-server) that serves all UI endpoints:
- REST: /health/*, /api/v1/info, /api/v1/pose/current, /api/v1/pose/stats,
/api/v1/pose/zones/summary, /api/v1/stream/status
- WebSocket: /api/v1/stream/pose (pose_data with 17 COCO keypoints),
/ws/sensing (raw sensing_update stream on port 8765)
- Static: /ui/* with no-cache headers
WiFi-derived pose estimation: derive_pose_from_sensing() generates 17 COCO
keypoints from CSI/WiFi signal data with motion-driven animation.
Data sources: ESP32 CSI via UDP :5005, Windows WiFi via netsh, simulation
fallback. Auto-detection probes each in order.
UI changes:
- Point all endpoints to Rust server on :8080 (was Python :8000)
- Fix WebSocket sensing URL to include /ws/sensing path
- Remove sensingOnlyMode gating — all tabs init normally
- Remove api.service.js sensing-only short-circuit
- Fix clearPingInterval bug in websocket.service.js
Also removes obsolete k8s/ template manifests.
Co-Authored-By: claude-flow <ruv@ruv.net>
Dragan Spiridonov