Compare commits

...

29 Commits

Author SHA1 Message Date
ruv 13f43004c8 docs(meridian): iteration 3 plan + GPU pre-train wiring stub (#68)
Closes the prototype's "iter 3 = plan + wiring documented" item (ADR-027 §2.0):

  - scripts/pretrain-mae-gcloud.sh — GCloud GPU driver for the MAE pre-train: a
    thin, reviewable mirror of scripts/gcloud-train.sh that provisions a VM in
    cognitum-20260110, builds wifi-densepose-train --features tch-backend,cuda,
    runs the `pretrain-mae` binary, downloads the .ot variable store, tears the
    VM down. Currently drives SyntheticCsiDataset (the smoke path); the one TODO
    is the --data-dir/--datasets plumbing for the real heterogeneous corpus.
    NOT run as part of this prototype. Also supports --dry-run (local synthetic
    pre-train, needs LibTorch).
  - ADR-027 §2.0 — added the "Iteration 3 plan" subsection: heterogeneous-CSI
    ingest (own recordings + MM-Fi + Wi-Pose + multi-band virtual sub-carriers,
    normalised to 56 sub-carriers), the GPU run, lifting the v0 limits
    (per-sample masking, transformer blocks, circular phase loss), the fine-tune
    handoff (load the CsiMae encoder into WiFiDensePoseModel via a
    `--init-encoder <mae.ot>` flag, then train the §2.x heads as regularisers),
    cross-domain eval (§4.6 protocol), and shipping the encoder as an RVF segment.
  - wifi-densepose-train/README.md — new "MERIDIAN-MAE" section pointing at the
    csi_mae module, the pretrain-mae binary, the gcloud script, and ADR-027 §2.0.
  - csi_mae.rs module doc — updated the iteration-status block.

cargo test -p wifi-densepose-train --no-default-features → 121 lib tests pass.

This completes the MERIDIAN CSI-MAE *prototype* (iter 1 masking pipeline +
iter 2 tch model/pretrain loop/bin + iter 3 plan/wiring). Real cross-domain
results need the heterogeneous ingest + a GPU pre-train run (iter 3 execution),
out of scope for the prototype.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-05-11 13:09:49 -04:00
ruv dcfa922518 docs(adr-027): mark MERIDIAN iter 2 complete (CI-verified tch path, #68)
Iteration status block: iter 1 + 2a + 2b done; iter 3 plan listed (heterogeneous-CSI ingest, real GPU pre-train, per-sample masking + transformer blocks, fine-tune §2.x heads, cross-domain eval, RVF segment).

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-05-11 13:05:28 -04:00
ruv 7d26b15eef feat(train): MERIDIAN-MAE — csi_mae::model + pretrain loop + pretrain-mae bin (iter 2b, #68)
Real CSI masked-autoencoder behind feature `tch-backend` (ADR-027 §2.0):

  - CsiMae: dual-stream per-token amp+phase embed → fuse → residual-MLP encoder
    over the visible tokens → flatten-to-latent bottleneck → learned per-position
    query + broadcast latent → residual-MLP decoder → dec_amp_head / dec_ph_head
    → index_select the masked positions. (MLP-based v0; self-attention transformer
    blocks are iter 3.)
  - CsiMae::reconstruction_loss(pred_amp, pred_phase, tgt_amp, tgt_phase, phase_w)
    = MSE(amp) + phase_w * MSE(phase).
  - MaeBatch::from_windows — partition computed once from window 0 and reused
    across the batch (the bottleneck fixes n_tokens), ndarray → tch conversion.
  - pretrain_step(model, opt, batch) -> f64 — one Adam step, returns the loss.
  - src/bin/pretrain_mae.rs — synthetic-data pre-train driver (required-features
    = ["tch-backend"]); clap args for epochs/batch/samples/lr/mask-ratio/save.
  - #[cfg(feature="tch-backend")] smoke test: loss halves when overfitting one
    batch over 60 steps; also asserts model.n_visible/n_masked match
    mask_csi_window's clamping.

v0 limits (documented in the module): fixed n_tokens; batch-shared masking;
MSE on unwrapped phase (vs a circular loss). The dev box has no LibTorch, so the
tch path is CI-verified (`--features tch-backend`), not locally. The default
`cargo test -p wifi-densepose-train --no-default-features` stays green (121 lib
tests) — the model module and the bin are both feature-gated.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-05-11 13:05:27 -04:00
ruv 48c7d03250 feat(train): MERIDIAN-MAE — information-guided masking (iter 2a, #68)
csi_mae::mask_csi_window now dispatches on MaskStrategy:
  - Random:      uniform Fisher–Yates (as before).
  - InfoGuided:  CIG-MAE-style — preferentially mask high-information tokens.
                 A token's "information" = variance of its amplitude values +
                 variance of its phase values (token_information()); near-constant
                 tokens are trivially in-painted so masking them teaches less.
                 Selection is weighted-without-replacement (Efraimidis–Spirakis:
                 key_i = u_i^(1/w_i), ranked by ln(u_i)/w_i) — exact, and
                 deterministic given `seed` (the u_i come from SplitMix64).

Replaces the iteration-1 "InfoGuided falls back to Random with a warning" stub.
+3 unit tests (info-guided skews ≥7.5/10 toward high-info tokens; deterministic
in seed; token_information ≈ 0 for constant tokens). `cargo test -p
wifi-densepose-train --no-default-features` → 121 lib tests pass.

Still to do (iter 2b, next loop tick): the real csi_mae::model (tch encoder/
decoder + reconstruction_loss + pretrain_step), bin/pretrain_mae.rs, a gated
"loss decreases" smoke test.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-05-11 12:57:42 -04:00
ruv 603ad585b6 feat(train): MERIDIAN-MAE — CSI masked-autoencoder masking pipeline (iter 1, #68)
New `wifi-densepose-train::csi_mae` module (ADR-027 §2.0):

  - MaeConfig (+ validate), MaskStrategy {Random, InfoGuided}
  - TokenLayout — flattens a [T,tx,rx,sub] CSI window to [N=T*tx*rx, sub] tokens
    (the same layout model.rs::ModalityTranslator consumes)
  - mask_csi_window — deterministic visible/masked token partition + amplitude &
    phase reconstruction targets; reproducible via a tiny inline SplitMix64 PRNG
    (no extra dependency); clamps so both partitions are non-empty
  - reassemble_tokens — round-trips encoder-visible + decoder-predicted tokens
    back to a full [N, sub] grid (for reconstruction eval/viz)
  - model submodule (gated behind `tch-backend`): v0 skeleton — the
    encoder/decoder networks, reconstruction loss, and pretrain_step land in
    iteration 2 (transformer blocks, per-sample masking, info-guided masking,
    a `pretrain-mae` bin)

8 new unit tests; builds and tests green under
`cargo test -p wifi-densepose-train --no-default-features` (118 lib tests pass).
The tch-gated `model` submodule is not exercised by the default workspace test
job — compile-checking it needs a LibTorch toolchain.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-05-11 12:45:11 -04:00
ruv 1d4f23bd41 docs(adr-027): re-scope MERIDIAN to MAE foundation pre-training (§2.0, #68)
Adds §2.0 — the primary MERIDIAN path is now a three-stage pipeline:
  1. pre-train a CIG-MAE-style dual-stream (amplitude+phase) masked autoencoder
     on heterogeneous CSI (data breadth > pose-net capacity — arXiv:2511.18792);
  2. fine-tune the existing §2.1–§2.6 heads (17-kpt/DensePose, AETHER, domain-
     adversarial, geometry-conditioned) on top of the pre-trained encoder;
  3. adapt per-room with source-free unsupervised domain adaptation behind
     coherence_gate.rs::Recalibrate (separate ADR).

§2.1+ is retained but re-framed as the fine-tune-stage head, not a from-scratch
design. Adds the supporting references (2511.18792, 2512.04723, 2605.01369,
2506.12052, ACM TOSN 10.1145/3715130) and points at the 2026-Q2 SOTA survey.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-05-11 12:45:11 -04:00
ruv 8b2a2d94e8 docs(research): 2026-Q2 agentic-AI & RF/WiFi-sensing SOTA survey
Surveys the relevant slice of the 2026-Q2 agentic-AI literature (long-horizon
agents, agent-memory discipline, self-improving/continual learning, the ESP32
mesh-as-a-swarm framing, the "agent harness on the MCU" pattern, retrieval/
quantization incl. the CoDEQ verdict, agentic verification) and the related
RF/WiFi-sensing SOTA (CSI foundation models — the 1.3M-sample MAE scaling study
showing data > capacity, AM-FM, CIG-MAE amplitude+phase MAE; source-free domain
adaptation; the DensePose-from-WiFi lineage; multistatic fusion; mmWave+WiFi
vitals; adversarial/privacy; through-wall). Maps every finding to a RuView ADR
with impact/effort/horizon. Headline recommendation: re-scope MERIDIAN (ADR-027)
as a heterogeneous-CSI MAE pre-train -> small task head.

Lives under docs/research/sota/ alongside 2026-Q2-rf-sensing-and-edge-rust.md.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-05-11 12:45:10 -04:00
rUv 19ee207d51 Merge pull request #528 from ruvnet/fix/update-submodules-workflow
ci: fix "Update vendor submodules" workflow (git identity + drop --merge)
2026-05-11 12:34:20 -04:00
ruv 8aa7fb9e9f ci: fix "Update vendor submodules" workflow (identity + drop --merge)
The scheduled job has been failing on every run with:

    fatal: empty ident name (...) not allowed
    fatal: Unable to merge '...' in submodule path 'vendor/ruvector'

Two bugs:
1. `git config user.name/email` was only set inside the "Create PR" step,
   but `git submodule update --remote --merge` runs first and the merge
   inside vendor/ruvector needs a committer when the pinned commit isn't a
   fast-forward of upstream `main` → "Committer identity unknown".
2. `--merge` is the wrong operation here. We only want to bump the
   superproject's gitlink to the latest upstream commit on each submodule's
   tracked branch — there's no reason to create merge commits inside the
   vendored repos, and `--merge` breaks whenever the current pin has diverged.

Fix:
- Add a "Configure git identity" step before any commit-creating operation.
- Replace `git submodule update --remote --merge` with
  `git submodule sync --recursive && git submodule update --remote --recursive`
  (detached checkout at each `.gitmodules` branch tip).
- Log the pointer diff in the "Check for changes" step for reviewability.
- Tidy the PR-creation step (identity now set globally; clearer commit/PR text).

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-05-11 12:33:40 -04:00
rUv f2e3a6a392 Merge pull request #526 from ruvnet/fix/esp32-issues-505-517-521
fix: ESP32 CSI 0pps (#521), aggregator sibling magics (#517), version.txt (#505) + fix-marker CI guard
2026-05-11 11:40:36 -04:00
ruv eda45a6857 ci: fix-marker regression guard (witness-style)
Adds a fast per-PR gate that asserts previously-shipped fixes are still
present in the tree — the CI analogue of the ruflo witness fix-marker
system, but self-contained (no plugin dependency, reviewable as plain
JSON). Complements the heavier checks (firmware build, deterministic
pipeline proof, release witness bundle) by catching the silent-revert
class of regression that build+test wouldn't.

  - scripts/fix-markers.json   manifest: 11 markers (RuView#396, #521,
    #517, #505, #354, #263, #266/#321, #265, #232/#375/#385/#386/#390,
    ADR-028 proof + witness bundle). Each has files / require (literal
    substring or /regex/) / optional forbid / rationale / ref.
  - scripts/check_fix_markers.py  stdlib-only checker. Exit 0 clean /
    1 regression / 2 bad manifest. Modes: --list, --json, --only ID.
  - .github/workflows/fix-regression-guard.yml  runs on PR + push to
    main/master; gates on the checker and writes the result table into
    the run summary + an artifact.

If a fix is intentionally removed, update scripts/fix-markers.json in the
same PR with a rationale — the diff becomes the audit trail.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-05-11 10:48:14 -04:00
ruv a1cb6bd8e5 fix(firmware): bump version.txt to 0.6.4 + CI guard for tag/version match (#505)
version.txt on main was still 0.6.2. CMake reads PROJECT_VER from it, so
esp_app_get_description()->version (and the boot log line) reported 0.6.2
for any source build — and v0.6.3-esp32 shipped a release binary that
internally identified as 0.6.2 because the bump never landed on main.

  - version.txt: 0.6.2 -> 0.6.4 (matches the latest release tag)
  - firmware-ci.yml: new `version-guard` job that runs on v*-esp32 tag
    pushes and fails the run if the tag's X.Y.Z != version.txt, so a
    future release can't ship a mislabeled binary.

Closes #505

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-05-11 10:48:14 -04:00
ruv 4d0521ca08 fix(hardware): aggregator tolerates sibling RuView UDP packet magics (#517)
The ESP32 firmware multiplexes several wire packet types onto the same
UDP port as ADR-018 raw CSI frames (magic 0xC5110001):

  0xC5110002  ADR-039 edge vitals (32 B)
  0xC5110003  ADR-069 feature vector
  0xC5110004  ADR-063 fused vitals
  0xC5110005  ADR-039 compressed CSI
  0xC5110006  ADR-081 feature state
  0xC5110007  ADR-095/#513 temporal classification

Esp32CsiParser only knew 0xC5110001, so the standalone `aggregator`
binary printed "parse error: Invalid magic: expected 0xc5110001, got
0xc5110002" for every vitals packet. No CSI data was lost — just noise.

Add the sibling-magic constants + ruview_sibling_packet_name(), classify
recognized siblings before the CSI-frame length gate, and return a new
ParseError::NonCsiPacket { magic, kind } instead of InvalidMagic. The
`aggregator` CLI now skips them quietly (logs "[skipped ADR-039 edge
vitals packet — not a CSI frame]" only with --verbose); the library-level
CsiAggregator already dropped them silently. New regression tests cover
all seven magics.

Closes #517

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-05-11 10:48:00 -04:00
ruv 3f55c95b34 fix(esp32): disable WiFi modem sleep so CSI capture isn't starved (#521)
csi_collector_init() never called esp_wifi_set_ps(), leaving the radio on
the ESP-IDF STA default WIFI_PS_MIN_MODEM. The modem then sleeps between
DTIM beacons; combined with the MGMT-only promiscuous filter (#396) the
CSI callback is starved and the per-second yield collapses toward 0 pps,
which is what users on a clean multi-node setup were seeing
(motion=0.00 presence=0.00 yield=0pps).

Force WIFI_PS_NONE before enabling promiscuous mode — the textbook
requirement for reliable CSI capture (every ESP-IDF CSI example does it).
New boot line: "csi_collector: WiFi modem sleep disabled (WIFI_PS_NONE)
for CSI capture". Battery duty-cycling is unaffected: power_mgmt_init()
runs after this and re-enables modem sleep when provision.py is given
--duty-cycle <100.

Builds clean for esp32s3 (idf.py build, 48% flash free).

Closes #521

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-05-11 10:47:48 -04:00
rUv e7904786f0 Update README.md
Added Spatial Intelligence to readme, since that seems to be a common description
2026-05-03 11:48:12 -04:00
ruv 9a078e4ac8 fix(pointcloud): exponential backoff on unreachable backend + status banner
When ?backend=<url> pointed at a server that wasn't running (e.g. user
forgot to start ruview-pointcloud serve before clicking Connect ESP32),
the viewer was retrying 10 Hz forever — flooding the console with
ERR_CONNECTION_REFUSED and offering no guidance about what was wrong.

Two fixes:

1. Replace setInterval(fetchCloud, 100) with self-rescheduling
   setTimeout. On success: 250 ms steady cadence. On failure for an
   explicit backend: 250 ms → 500 → 1 s → 2 s → 4 s → 8 s → 16 s →
   capped at 30 s. Resets to 250 ms the moment the backend comes back.
   Auto mode (Pages with no backend) still disables network entirely
   after the first 404. Strict-live mode (?live=1) also backs off so
   it doesn't spam.

2. Show an actionable status banner in the info panel when the chosen
   backend is unreachable: the URL, the actual error string, the next
   retry time, and the exact `cargo run` command to start the server.
   Visitor sees the diagnosis instead of staring at a 'demo' badge
   wondering why their ESP32 feed isn't visible.

The scene keeps animating (face mesh / synthetic) while the viewer
waits, so the tab never goes blank.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-04-29 23:03:05 -04:00
ruv 0e39faac73 feat(pointcloud): overlay browser face mesh on top of ESP32 backend feed
Lets the visitor enable their browser webcam face mesh in addition to
(not instead of) a connected ESP32 backend. Both render in the same
Three.js scene — the live ESP32-driven splats from /api/splats plus the
visitor's own face as a 478-vertex MediaPipe point cloud. Use cases:

- Local development: see your face overlaid on the camera+CSI fusion
  output to debug coordinate-frame alignment.
- Demos: show 'this is the room as ESP32 sees it, and this is me as
  MediaPipe sees me' side-by-side in one scene.

Implementation:
- Extract pushFaceSplats(splats) — pushes the 478 face vertices plus
  ~8000 edge-interpolated samples into the array, with no Foundation
  context. Reused by faceMeshFrame (demo path) and handleData (overlay
  path) so there is one source of truth for face-splat geometry.
- handleData now appends pushFaceSplats output to data.splats when the
  source is not 'face-mesh' AND the user has clicked the camera CTA.
  Sets data._faceOverlay so the badge can show '+ face overlay'.
- Camera CTA is no longer hidden in remote/live modes — it relabels to
  '▶ Add face overlay' so the affordance is clear. Strict-live mode
  (?live=1) still hides it because the offline panel takes over.
- Splat count in the info panel reflects the rendered total (backend +
  overlay) when the overlay is active.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-04-29 20:37:36 -04:00
ruv ad41a89960 feat(pointcloud): integrate ESP32 CSI as optional data stream from hosted viewer
The hosted GitHub Pages viewer can now act as a thin client for a
locally-running ruview-pointcloud serve instance — flip a button, the
ESP32's CSI fusion (camera depth + WiFi CSI + mmWave) renders inside
the same Three.js scene that previously only showed the face mesh
demo. No clone, no rebuild, no toolchain on the visitor's side.

Server (stream.rs):
- Add tower_http::cors::CorsLayer with a deliberate allowlist:
  https://ruvnet.github.io, http://localhost:*, http://127.0.0.1:*,
  and 'null' (for file:// origins). Anything else is denied — not a
  wildcard CORS. Modern browsers (Chrome 94+, Firefox 116+, Safari
  16.4+) treat 127.0.0.1 as a "potentially trustworthy" origin so
  HTTPS Pages → HTTP loopback is permitted. The new layer wraps the
  existing /api/cloud, /api/splats, /api/status, /health routes.
- Cargo.toml: pull in workspace tower-http (cors feature already on).

Viewer:
- New "📡 Connect ESP32…" CTA bottom-right. Clicking prompts for a
  ruview-pointcloud serve URL (default http://127.0.0.1:9880),
  persists the last-used value in localStorage, and reloads with
  ?backend=<url> so the existing remote-mode fetch path takes over.
  When already connected the button toggles to "disconnect" and
  reloads back to the demo.
- Reuses the existing transport selector — no new code path to
  maintain. The face mesh / synthetic demo render path is unaffected;
  this is purely an additive UI affordance over the ?backend= query.

Docs:
- ADR-094 §2.3 expanded with the local-ESP32 workflow and the CORS
  posture rationale.
- Workflow README documents ?backend=http://127.0.0.1:9880 as the
  intended local-ESP32 path.

Tests: cargo test -p wifi-densepose-pointcloud → 15/15 passed.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-04-29 20:33:00 -04:00
ruv e3021c777c chore(pointcloud): inline amber-dot favicon to silence /favicon.ico 404
Browsers auto-request /favicon.ico when none is declared in <head>.
On a static GitHub Pages host that's a guaranteed 404 in the console.
Inline a 32x32 SVG amber dot via data: URL so the browser is satisfied
without an extra network round-trip.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-04-29 20:27:44 -04:00
ruv b4c2f7d20b fix(pointcloud): stop polling /api/splats on Pages after first 404
When the viewer is hosted on a static origin (GitHub Pages, S3) it has
no backend at /api/splats. The default ?backend=auto path was issuing
a fetch every 100 ms, getting a 404, falling back to the demo, and
flooding the console with one 404 per tick. Cosmetic on the surface
but real network/CPU waste over time.

After the first 404 in auto mode, set networkDisabled=true and skip
fetch on subsequent ticks — the interval still fires but goes straight
to pickDemoFrame() so the face mesh / synthetic render path keeps
animating. Remote (?backend=<url>) and live (?live=1) modes keep
retrying so a transient outage doesn't permanently downgrade them.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-04-29 20:24:38 -04:00
ruv aea9892aed Revert "feat(pointcloud): Hollywood face fx — webcam texture, wireframe, scan line"
This reverts commit 347ad4bb11.
2026-04-29 20:21:27 -04:00
ruv 347ad4bb11 feat(pointcloud): Hollywood face fx — webcam texture, wireframe, scan line
Adds optional cinematic effects to the face-mesh demo, all toggleable
via a new ?fx= URL param. Default is 'all' (texture + mesh + scan +
halo). Lightweight modes available: ?fx=clean (texture only) or
?fx=points (original solid amber).

- Texture: per-frame webcam → hidden 2D canvas → getImageData lookup
  at each landmark (and each interpolated edge sample). Splats now
  carry the visitor's actual skin tone, not solid amber. Sampling is
  mirrored on x to match the selfie convention used by the face mesh
  vertex placement. All on-device — no frames leave the browser.
- Mesh: persistent THREE.LineSegments overlay drawn from
  FACEMESH_TESSELATION (~1300 edges). Translucent (opacity 0.35),
  amber, additive blending, depthWrite off — gives a holographic
  wireframe wrapping the point cloud. Geometry is updated in place
  each frame; only positions get re-uploaded.
- Scan: vertical bright slab sweeps top→bottom every 4 seconds,
  amplifying splat color up to 2.6× when within ±0.08 world units of
  the line. Westworld-style scanning.
- Halo: existing 60-particle ring around the face is now opt-in via
  FX_HALO. Cleaner default for the texture-mesh combination.

Info panel surfaces active fx list in face-mesh mode. Synthetic
fallback hides the wireframe overlay so it doesn't render against an
empty figure. Workflow README updated with the new ?fx= options.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-04-29 20:18:15 -04:00
ruv 5d7fccce79 feat(pointcloud): fix upside-down face, densify mesh, add Foundation aesthetic
Three fixes in one pass to address visitor feedback:

1. Face was rendering upside down — MediaPipe's lm.y is image-down (0=top
   of frame, 1=bottom) and the existing updateSplats() already does a
   y-negate to convert to Three.js Y-up. Pre-flipping in lmToCenter was a
   double flip. Use lm.y directly so the renderer's single flip lands the
   head at the top of the screen.

2. Density and fidelity — interpolate 6 splats per FACEMESH_TESSELATION
   edge (~1300 edges → ~8000 face splats vs 478 vertex-only). Amplify
   lm.z mapping (×8 vs ×4) so eye sockets, nose, and chin show real 3D
   depth. Smaller splat scale (0.006 surface, 0.010 vertices) for finer
   point appearance.

3. Foundation-inspired aesthetic — the demo now renders the subject
   (face mesh OR procedural fallback) inside a Hari Seldon time-vault:

   * Holographic surveyor grid in amber, breathing brightness pattern.
   * Slow-rotating two-arm galactic spiral receding behind the subject
     (~640 stars, warm core to cool edges, Trantor-evocation).
   * 800-star deterministic distant starfield on a spherical shell
     (fixed LCG seed so visitors don't see noise flicker).
   * 60-particle holographic halo orbiting the subject plane.

   Shared pushFoundationContext() drives both face-mesh and synthetic
   paths. Synthetic procedural figure densified 4x (240 vs 60 points)
   and re-oriented (head→top, feet→bottom) so the y-down convention is
   internally consistent.

Camera pulled back to (0, 0.2, -3.5) to frame the galactic context.
Poll cadence 4 Hz → 10 Hz so the spiral animates smoothly. Info panel
gets a Seldon quote and "Seldon Vault" branding. CTA copy reframed to
"Project Subject — render your face into the Vault".

ADR-094 already documents the dual-transport intent; the aesthetic
choices here are content, not architecture, so no ADR update needed.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-04-29 19:51:12 -04:00
ruv cbedbce9e3 feat(pointcloud): use MediaPipe Face Mesh for the live demo (ADR-094)
The previous synthetic procedural demo did not represent what the local
fusion pipeline produces — a real depth-backprojected point cloud of
the user's face and surroundings. This commit ports the closest browser
equivalent: MediaPipe Face Mesh runs in-browser at ~30 fps and emits
478 3D landmarks per frame. Each visitor now sees the outline of their
own face rendered as a point cloud, with a small floor + back wall for
spatial context.

- Adds MediaPipe Face Mesh + Camera Utils via jsdelivr CDN.
- Adds an "▶ Enable camera" CTA so getUserMedia is gated on a user
  gesture (required by some browsers and good UX regardless).
- New face-mesh frame generator uses the same splat shape as the live
  /api/splats payload, so a single render path drives both modes.
- Mirrors x to match selfie convention; maps lm.z (relative depth) to
  the world-coord range used by the live pipeline.
- Falls back automatically to the procedural floor + walls + figure
  when the camera is denied, dismissed, or unavailable.
- Badge surfaces the new state: '● DEMO Your Face (MediaPipe)'.
- Bumps poll cadence to 4 Hz so face mesh updates feel live.
- ADR-094 updated to reflect the new default behavior.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-04-29 19:42:51 -04:00
ruv 7343bdc4dd docs(readme): retarget Live 3D Point Cloud link to hosted demo
Now that ADR-094 is deployed, point the README's demo link at
https://ruvnet.github.io/RuView/pointcloud/ instead of the
docs/readme-details.md anchor. Matches the pattern of the sibling
Observatory and Pose Fusion demo links.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-04-29 19:37:11 -04:00
rUv 21b2b3352f feat(pointcloud): GitHub Pages demo with optional live backend (ADR-094) (#495)
Publishes the live 3D point cloud viewer to gh-pages/pointcloud/ so it
can be linked from the README alongside the Observatory and Dual-Modal
Pose Fusion demos. The viewer auto-selects its transport from URL
parameters:

- default / ?backend=auto — try /api/splats, fall back to synthetic demo
- ?backend=demo — synthetic in-browser only, no network
- ?backend=<url> — fetch from a CORS-permitting host running
  ruview-pointcloud serve
- ?live=1 — strict mode, show offline panel instead of demo fallback

The synthetic frame matches the live API JSON shape (splats, count,
frame, live, pipeline.{skeleton,vitals}) so a single render path drives
both modes. New workflow uses keep_files: true to preserve the existing
observatory/, pose-fusion/, and nvsim/ deployments on gh-pages.

See docs/adr/ADR-094-pointcloud-github-pages-deployment.md for the full
decision record and 6 acceptance gates.
2026-04-29 19:35:41 -04:00
ruv e11d569a39 docs(readme): split details to docs/readme-details.md and reorganize
- Move Latest Additions, Key Features, and everything from Installation
  through Changelog (1855 lines) into docs/readme-details.md.
- Keep README focused on overview, capability table, How It Works,
  Use Cases, Documentation, License, and Support.
- Add per-row emojis to the top capability table.
- Add 3D point cloud row noting optional camera + WiFi CSI + mmWave
  fusion with link to the live viewer demo.
- Move Documentation table closer to the bottom (just above License).
- Collapse Edge Intelligence (ADR-041) into a <details> block matching
  the sibling Use Case sections.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-04-29 19:34:24 -04:00
Dragan Spiridonov 36e70bf229 security: pin GitHub Actions to SHAs and bump vulnerable npm deps (#442)
* security: pin GitHub Actions to SHAs and bump vulnerable npm deps (#442)

Addresses confirmed findings from issue #442 (Pentesterra/DevGuard).

GitHub Actions — pin all third-party Action references in
security-scan.yml and ci.yml to verified commit SHAs (with the
matching version in a trailing comment for legibility):

  * snyk/actions/python              -> v1.0.0
  * aquasecurity/trivy-action        -> v0.36.0  (security-scan.yml + ci.yml)
  * bridgecrewio/checkov-action      -> v12.1347.0
  * tenable/terrascan-action         -> v1.4.1
  * checkmarx/kics-github-action     -> v2.1.20  (the action #442 named)
  * trufflesecurity/trufflehog       -> v3.95.2

  Verification:
    grep -rE 'uses:.*@(main|master|latest)$' .github/workflows/
  returns no matches.

npm deps in ui/mobile — add `overrides` forcing patched versions of
the three packages flagged by the DevGuard scanner, regenerate
package-lock.json:

  * @xmldom/xmldom@0.8.11  ->  0.8.13
  * node-forge@1.3.3       ->  ^1.4.0   (closes 3 HIGH advisories)
  * picomatch@2.3.1        ->  ^2.3.2   (transitive in jest tooling)

  npm audit totals: 25 -> 22 advisories (5 HIGH -> 2 HIGH).

Out of scope for this PR (tracked separately):
  * Sensing-server unauth REST API surface — opened as #443
    pending design-intent confirmation from @ruvnet.
  * Bearer-token-shaped string in git history — confirmed test
    seed per repo owner; no rotation required.

Refs: #442

Co-Authored-By: claude-flow <ruv@ruv.net>

* chore: add Dependabot config for github-actions and ui/mobile npm (#442)

Pairs with the SHA pinning from the previous commit so the pinned
versions get automated weekly bumps rather than drifting back to
mutable refs over time.

Scoped to the two ecosystems #442 surfaced findings in:
  * github-actions (root)  — the supply-chain risk
  * npm (ui/mobile)        — the @xmldom/xmldom, node-forge, picomatch
                             advisories

Other ecosystems (pip, cargo, desktop UI npm) deliberately omitted —
they can be added in a separate PR if desired.

Refs: #442

Co-Authored-By: claude-flow <ruv@ruv.net>

* chore(dependabot): expand to pip, cargo, and desktop UI npm (#442)

Broadens the Dependabot config from the initial 2 ecosystems
(github-actions + ui/mobile npm) to cover all 5 package surfaces
in the repo so pinned dependencies stay current across the board:

  + npm  /v2/crates/wifi-densepose-desktop/ui   (vite advisory live)
  + pip  /                                     (requirements.txt loose pins)
  + cargo /v2                                  (no cargo audit in CI yet)

Marginal cost is zero — Dependabot only opens PRs when an upstream
bump exists, and per-ecosystem pull-request limits cap the noise.
Each ecosystem labelled distinctly so PRs route cleanly.

Refs: #442

Co-Authored-By: claude-flow <ruv@ruv.net>

---------

Co-authored-by: claude-flow <ruv@ruv.net>
2026-04-28 08:46:51 -04:00
rUv f06d0c6ab5 fix(firmware): SPI cache crash fix + node_id/filter_mac defensive copies + esptool v5 (rebased #397)
* fix(firmware): move defensive node_id capture before wifi_init_sta()

The original defensive copy in csi_collector_init() (line 172 of main.c)
runs AFTER wifi_init_sta() (line 147), which on some ESP32-S3 devices
corrupts g_nvs_config.node_id back to the Kconfig default of 1.

Reproduced on device 80:b5:4e:c1:be:b8 (ESP32-S3 QFN56 rev v0.2):
  - NVS provisioned with node_id=5
  - Release firmware (no fix): seed receives node_id=1 (clobbered)
  - This patch: seed receives node_id=5 (correct)

Changes:
  - Add csi_collector_set_node_id() called from main.c immediately
    after nvs_config_load(), before wifi_init_sta() runs
  - csi_collector_init() now detects and logs the clobber if early
    capture disagrees with current g_nvs_config value
  - Fallback path preserved: if set_node_id() is never called,
    init() still captures from g_nvs_config (backwards compatible)

Co-Authored-By: claude-flow <ruv@ruv.net>

* fix(firmware): defensive copy of filter_mac to prevent callback crash

The CSI callback reads g_nvs_config.filter_mac_set and filter_mac on
every invocation (100-500 Hz). If wifi_init_sta() corrupts g_nvs_config
(same root cause as the node_id clobber), the callback reads garbage
from the struct, leading to Core 0 LoadProhibited panic after ~2400
callbacks (~70 seconds of operation).

Extends the early-capture pattern from the node_id fix to also copy
filter_mac_set and filter_mac into module-local statics before WiFi
init runs. Adds canary logging to detect filter_mac corruption.

Observed on device 80:b5:4e:c1:be:b8 via serial:
  CSI cb #2400 → Guru Meditation Error: Core 0 panic'ed (LoadProhibited)
  → TG0WDT_SYS_RST → reboot → crash again at ~2900 callbacks

Refs #232 #375 #385 #386 #390

Co-Authored-By: Ruflo & AQE

* fix(firmware): MGMT-only promiscuous filter to prevent SPI cache crash

The WiFi driver's wDev_ProcessFiq interrupt handler crashes with
LoadProhibited in cache_ll_l1_resume_icache when promiscuous mode
captures MGMT+DATA frames (100-500 interrupts/sec). The high interrupt
rate races with SPI flash cache operations, corrupting cache state.

Changes:
- Promiscuous filter: MGMT+DATA → MGMT-only (~10 Hz beacons)
- CSI config: disable htltf_en and stbc_htltf2_en (LLTF-only)

LLTF provides 64 subcarriers (HT20) — sufficient for presence,
breathing, and fall detection. The 10 Hz beacon rate eliminates
the SPI flash cache contention that caused the crash.

Verified on device 80:b5:4e:c1:be:b8:
- Before: LoadProhibited crash at ~1600-2400 callbacks (every ~70s)
- After: 2700+ callbacks over 4.7 minutes, zero crashes

Backtrace decode confirmed crash in ESP-IDF closed-source WiFi blob:
  _xt_lowint1 → wDev_ProcessFiq → spi_flash_restore_cache
  → cache_ll_l1_resume_icache → EXCVADDR=0x00000004 (NULL deref)

Co-Authored-By: Ruflo & AQE

* fix(provision): write-flash → write_flash for esptool v5 compat

esptool v5+ rejects hyphenated subcommands. The provision script
used 'write-flash' which fails with "invalid choice". Changed to
'write_flash' (underscore) which works with both old and new esptool.

Co-Authored-By: Ruflo & AQE

* fix(firmware): 50 Hz callback rate gate + sdkconfig extra IRAM opt

- Add early rate gate in wifi_csi_callback at 50 Hz (defense-in-depth,
  does not prevent crash alone but reduces callback execution time)
- Add null-data injection timer infrastructure (disabled — TX adds
  interrupt pressure that triggers the SPI cache crash, RuView#396)
- sdkconfig.defaults: add CONFIG_ESP_WIFI_EXTRA_IRAM_OPT=y
- sdkconfig.defaults: document SPIRAM XIP attempt (crashes differently)

Co-Authored-By: Ruflo & AQE

* fix(firmware): address PR #397 review feedback

Applies @ruvnet's five review requests on PR #397 (RuView#397 comment
4289417527):

1. **Inline comment on `provision.py` `write_flash`** — ESP-IDF v5.4
   bundles esptool 4.10.0 (underscore-only). #391's hyphen swap broke
   the documented venv flow; kept the underscore form and added a
   three-line comment warning future maintainers not to "re-fix" it.

2. **Correct `edge_processing.c` sample_rate** (blocking) — changed
   hard-coded `20.0f` → `10.0f` at line 718 so
   `estimate_bpm_zero_crossing()` matches the MGMT-only CSI rate.
   Without this, breathing and heart-rate reports were 2× the true
   value. Added a comment tying the constant to the callback rate gate.

3. **Removed disabled probe-injection infrastructure** — dropped the
   forward declaration, the `CSI_PROBE_INTERVAL_MS` define, six static
   variables (`s_probe_timer`, `s_probe_tx_count`, `s_probe_tx_fail`,
   `s_ap_bssid`, `s_ap_bssid_known`), and three functions
   (`csi_send_probe_request`, `probe_timer_cb`,
   `csi_collector_start_probe_timer`). None were reachable.
   `csi_inject_ndp_frame()` reverted to the original ADR-029 stub.
   Can be revived from this commit's parent if needed.

4. **Cleaned `sdkconfig.defaults`** — removed the SPIRAM prose and
   commented-out `# CONFIG_SPIRAM is not set` line. Kept only the live
   `CONFIG_ESP_WIFI_EXTRA_IRAM_OPT=y` with a concise rationale.

5. **Bumped firmware version 0.6.1 → 0.6.2** and added four
   `[Unreleased]` CHANGELOG entries covering the SPI cache crash fix,
   the `filter_mac` / `node_id` clobber defense, the sample-rate
   correction, and the `write_flash` command-form revert.

Net: +39 / -128 across six files.

Validation in this devcontainer:
- Static sanity on modified C files: braces balance (csi_collector.c
  59/59; edge_processing.c 96/96), zero dangling references to removed
  probe-injection symbols.
- Rust workspace tests and Python proof not executed here — cargo not
  installed and pip blocked by PEP 668. Deferring hardware build +
  flash + miniterm verification to @ruvnet's COM7 per his offer in
  the review comment.

Co-Authored-By: claude-flow <ruv@ruv.net>

---------

Co-authored-by: Dragan Spiridonov <spiridonovdragan@gmail.com>
2026-04-28 08:41:49 -04:00
37 changed files with 5162 additions and 1995 deletions
+58
View File
@@ -0,0 +1,58 @@
version: 2
updates:
# Keep all third-party GitHub Actions on verified, pinned commit SHAs.
# Pairs with the SHA pinning in security-scan.yml and ci.yml so that
# future bumps stay automated and reviewable rather than drifting back
# to mutable @master / @main refs. See issue #442.
- package-ecosystem: github-actions
directory: /
schedule:
interval: weekly
open-pull-requests-limit: 5
labels:
- dependencies
- github-actions
# Mobile app npm deps. Includes the @xmldom/xmldom, node-forge, and
# picomatch advisories from #442 plus axios and any future surface.
- package-ecosystem: npm
directory: /ui/mobile
schedule:
interval: weekly
open-pull-requests-limit: 10
labels:
- dependencies
- mobile
# Desktop UI npm deps. Direct vite devDep currently has a HIGH advisory
# (dev-server-only path traversal); track future bumps automatically.
- package-ecosystem: npm
directory: /v2/crates/wifi-densepose-desktop/ui
schedule:
interval: weekly
open-pull-requests-limit: 5
labels:
- dependencies
- desktop
# Python deps used by v1/ and the FastAPI service. requirements.txt is
# only loosely pinned; let Dependabot surface upstream CVE bumps.
- package-ecosystem: pip
directory: /
schedule:
interval: weekly
open-pull-requests-limit: 10
labels:
- dependencies
- python
# Rust workspace (15+ crates). cargo audit is not currently wired into
# any workflow, so Dependabot is the primary automated bump path.
- package-ecosystem: cargo
directory: /v2
schedule:
interval: weekly
open-pull-requests-limit: 10
labels:
- dependencies
- rust
+1 -1
View File
@@ -255,7 +255,7 @@ jobs:
docker stop test-container
- name: Run container security scan
uses: aquasecurity/trivy-action@master
uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25 # v0.36.0
with:
image-ref: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
format: 'sarif'
+26
View File
@@ -2,6 +2,11 @@ name: Firmware CI
on:
push:
branches:
- '**'
tags:
# ESP32 firmware release tags — build + version-consistency guard (RuView#505).
- 'v*-esp32'
paths:
- 'firmware/**'
- '.github/workflows/firmware-ci.yml'
@@ -11,6 +16,27 @@ on:
- '.github/workflows/firmware-ci.yml'
jobs:
version-guard:
name: Verify version.txt matches release tag
runs-on: ubuntu-latest
if: github.ref_type == 'tag'
steps:
- uses: actions/checkout@v4
- name: Check firmware version.txt == tag
run: |
# Tag form: vX.Y.Z-esp32 → expect version.txt to contain X.Y.Z
TAG="${GITHUB_REF_NAME}"
EXPECTED="${TAG#v}"
EXPECTED="${EXPECTED%-esp32}"
ACTUAL="$(tr -d '[:space:]' < firmware/esp32-csi-node/version.txt)"
echo "Tag: $TAG → expected version.txt: $EXPECTED | actual: $ACTUAL"
if [ "$EXPECTED" != "$ACTUAL" ]; then
echo "::error::firmware/esp32-csi-node/version.txt is '$ACTUAL' but tag '$TAG' expects '$EXPECTED'."
echo "::error::Bump version.txt and re-tag so esp_app_get_description()->version is correct (RuView#505)."
exit 1
fi
echo "version.txt matches the release tag."
build:
name: Build ESP32-S3 Firmware (${{ matrix.variant }})
runs-on: ubuntu-latest
@@ -0,0 +1,54 @@
name: Fix-Marker Regression Guard
# Asserts that previously-shipped fixes are still present in the tree.
# Manifest: scripts/fix-markers.json Checker: scripts/check_fix_markers.py
# Run locally: python scripts/check_fix_markers.py (also --list / --json)
#
# This complements the heavyweight checks (firmware build, deterministic
# pipeline proof, witness bundle) with a fast per-PR "did someone revert a
# known fix?" gate — the CI analogue of the ruflo witness fix-marker system.
on:
push:
branches:
- main
- master
pull_request:
workflow_dispatch:
jobs:
fix-markers:
name: Verify fix markers
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Validate the manifest is well-formed JSON
run: python -c "import json; json.load(open('scripts/fix-markers.json')); print('manifest OK')"
- name: Check fix markers
run: python scripts/check_fix_markers.py
- name: Emit machine-readable result (for the run summary)
if: always()
run: |
python scripts/check_fix_markers.py --json > fix-markers-result.json || true
{
echo '### Fix-marker regression guard'
echo ''
echo '```'
python scripts/check_fix_markers.py || true
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
- name: Upload result artifact
if: always()
uses: actions/upload-artifact@v4
with:
name: fix-markers-result
path: fix-markers-result.json
retention-days: 30
+74
View File
@@ -0,0 +1,74 @@
name: Point Cloud Viewer → GitHub Pages
# Publishes the live 3D point cloud viewer to gh-pages/pointcloud/.
# The viewer defaults to a synthetic in-browser demo; users can append
# ?backend=<url> or ?backend=auto to point it at a real ruview-pointcloud
# server (CORS-permitting host required). See ADR-094.
#
# Uses keep_files: true to preserve the existing observatory/, pose-fusion/,
# nvsim/, and root index.html demos already on gh-pages.
on:
push:
branches: [main]
paths:
- 'v2/crates/wifi-densepose-pointcloud/src/viewer.html'
- '.github/workflows/pointcloud-pages.yml'
workflow_dispatch:
permissions:
contents: write
concurrency:
group: pointcloud-pages
cancel-in-progress: true
jobs:
build-and-deploy:
runs-on: ubuntu-latest
steps:
- name: Checkout main
uses: actions/checkout@v4
- name: Stage viewer for Pages
run: |
mkdir -p _site/pointcloud
cp v2/crates/wifi-densepose-pointcloud/src/viewer.html _site/pointcloud/index.html
# Drop a tiny README so direct browsers of the directory get context.
cat > _site/pointcloud/README.md <<'EOF'
# RuView — Live 3D Point Cloud Viewer
Hosted at: https://ruvnet.github.io/RuView/pointcloud/
## Modes
- Default — synthetic in-browser demo (no backend, no network calls).
- `?backend=auto` — fetch from `/api/splats` on the same origin
(only works when the viewer is served by `ruview-pointcloud serve`).
- `?backend=<url>` — fetch from `<url>/api/splats`. The intended
local-ESP32 use is `?backend=http://127.0.0.1:9880`: run
`ruview-pointcloud serve --bind 127.0.0.1:9880` on the same
machine with your ESP32 streaming CSI to UDP port 3333, then
visit the URL above. The local server's CorsLayer permits
requests from `https://ruvnet.github.io`, and modern browsers
permit HTTPS→127.0.0.1 mixed-content as a trustworthy origin.
The "📡 Connect ESP32" button in the viewer prompts for this
URL and persists it in localStorage.
- `?live=1` — require a live backend; show an offline message instead
of falling back to the synthetic demo.
See ADR-094 for the deployment design.
EOF
- name: Deploy to gh-pages/pointcloud/
uses: peaceiris/actions-gh-pages@v4
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./_site/pointcloud
destination_dir: pointcloud
# CRITICAL: preserves observatory/, pose-fusion/, nvsim/, and root
# index.html already on gh-pages.
keep_files: true
commit_message: 'deploy(pointcloud): ${{ github.sha }}'
user_name: 'github-actions[bot]'
user_email: 'github-actions[bot]@users.noreply.github.com'
+6 -6
View File
@@ -111,7 +111,7 @@ jobs:
continue-on-error: true
- name: Run Snyk vulnerability scan
uses: snyk/actions/python@master
uses: snyk/actions/python@9adf32b1121593767fc3c057af55b55db032dc04 # v1.0.0
env:
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
with:
@@ -163,7 +163,7 @@ jobs:
cache-to: type=gha,mode=max
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@master
uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25 # v0.36.0
with:
image-ref: 'wifi-densepose:scan'
format: 'sarif'
@@ -221,7 +221,7 @@ jobs:
uses: actions/checkout@v4
- name: Run Checkov IaC scan
uses: bridgecrewio/checkov-action@master
uses: bridgecrewio/checkov-action@99bb2caf247dfd9f03cf984373bc6043d4e32ebf # v12.1347.0
with:
directory: .
framework: kubernetes,dockerfile,terraform,ansible
@@ -238,7 +238,7 @@ jobs:
category: checkov
- name: Run Terrascan IaC scan
uses: tenable/terrascan-action@main
uses: tenable/terrascan-action@3a6e87da8e244513bd77b631e624552643f794c6 # v1.4.1
with:
iac_type: 'k8s'
iac_version: 'v1'
@@ -247,7 +247,7 @@ jobs:
sarif_upload: true
- name: Run KICS IaC scan
uses: checkmarx/kics-github-action@master
uses: checkmarx/kics-github-action@05aa5eb70eede1355220f4ca5238d96b397e30a6 # v2.1.20
with:
path: '.'
output_path: kics-results
@@ -277,7 +277,7 @@ jobs:
fetch-depth: 0
- name: Run TruffleHog secret scan
uses: trufflesecurity/trufflehog@main
uses: trufflesecurity/trufflehog@17456f8c7d042d8c82c9a8ca9e937231f9f42e26 # v3.95.2
with:
path: ./
base: main
+23 -6
View File
@@ -19,8 +19,24 @@ jobs:
fetch-depth: 0
token: ${{ secrets.GITHUB_TOKEN }}
- name: Update submodules to latest main
run: git submodule update --remote --merge
# Identity must be set BEFORE any operation that can create a commit.
# `git submodule update --remote --merge` used to fail here with
# "Committer identity unknown" because the merge inside vendor/ruvector
# needs an author when the pinned commit isn't a fast-forward of upstream.
- name: Configure git identity
run: |
git config --global user.name "github-actions[bot]"
git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com"
# Use a plain `--remote` checkout (detached HEAD at each submodule's
# configured `branch` tip from .gitmodules) rather than `--merge`. We only
# want to bump the superproject's gitlink to the latest upstream commit;
# there's no reason to create merge commits inside the vendored repos, and
# `--merge` breaks whenever the current pin has diverged from that branch.
- name: Update submodules to latest tracked branch
run: |
git submodule sync --recursive
git submodule update --remote --recursive
- name: Check for changes
id: check
@@ -29,21 +45,22 @@ jobs:
echo "changed=false" >> "$GITHUB_OUTPUT"
else
echo "changed=true" >> "$GITHUB_OUTPUT"
echo "--- submodule pointer changes ---"
git submodule status --recursive || true
git diff --submodule=log -- vendor/ || true
fi
- name: Create PR with updates
if: steps.check.outputs.changed == 'true'
run: |
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
BRANCH="chore/update-submodules-$(date +%Y%m%d-%H%M%S)"
git checkout -b "$BRANCH"
git add vendor/
git commit -m "chore: update vendor submodules to latest main"
git commit -m "chore: update vendor submodules to latest upstream"
git push origin "$BRANCH"
gh pr create \
--title "chore: update vendor submodules" \
--body "Automated submodule update to latest upstream main." \
--body "Automated submodule update to the latest upstream commit on each submodule's tracked branch (see \`.gitmodules\`). Review the pointer diff before merging." \
--base main \
--head "$BRANCH"
env:
+5 -1
View File
@@ -167,7 +167,11 @@ firing cleanly, HEALTH mesh packets sent.
Kconfig surface added under "Adaptive Controller (ADR-081)".
### Fixed
- **`provision.py` esptool v5 compat** (#391) — Stale `write_flash` (underscore) syntax in the dry-run manual-flash hint now uses `write-flash` (hyphenated) for esptool >= 5.x. The primary flash command was already correct.
- **Firmware: SPI flash cache crash under high CSI callback pressure** (RuView#396, #397) — ESP32-S3 nodes crashed in `cache_ll_l1_resume_icache` / `wDev_ProcessFiq` after ~2400 callbacks when the promiscuous filter admitted DATA frames at 100500 Hz. Fixed by narrowing the filter mask to `WIFI_PROMIS_FILTER_MASK_MGMT` (~10 Hz beacons), adding a 50 Hz early callback rate gate (`CSI_MIN_PROCESS_INTERVAL_US`) that drops excess callbacks before any processing work, and enabling `CONFIG_ESP_WIFI_EXTRA_IRAM_OPT=y` as defense-in-depth. Stability validated with a 4-min-per-node soak.
- **Firmware: `filter_mac` / `node_id` clobber by WiFi driver init** (#232, #375, #385, #386, #390, #397) — `g_nvs_config` can be corrupted during `wifi_init_sta()` on some devices (confirmed on `80:b5:4e:c1:be:b8`), reverting `node_id` to the Kconfig default and producing garbage MAC-filter reads in the CSI callback (100500 Hz). New `csi_collector_set_node_id()` API called from `app_main()` **before** `wifi_init_sta()` captures both fields into module-local statics (`s_node_id`, `s_filter_mac`, `s_filter_mac_set`). `csi_collector_init()` now runs a canary that distinguishes "early≠g_nvs_config" (corruption confirmed) from a no-op match. All CSI runtime paths use the defensive copies exclusively.
- **Firmware: `edge_processing` sample rate mismatch** (#397) — `estimate_bpm_zero_crossing()` was called with a hard-coded `sample_rate = 20.0f`, but MGMT-only promiscuous delivers ~10 Hz. Breathing and heart-rate reports were 2× too high. Corrected to `10.0f` with an explicit comment tying it to the callback rate.
- **`provision.py` esptool command form** (#391, #397) — ESP-IDF v5.4 bundles `esptool 4.10.0`, which only accepts `write_flash` (underscore). Standalone `pip install esptool` v5.x accepts both forms but prefers `write-flash`. #391 switched to `write-flash` which broke the documented ESP-IDF Python venv flow; #397 reverts to `write_flash` (works with both esptool 4.x and 5.x) with an inline comment warning future maintainers not to "re-fix" it.
- **`provision.py` esptool v5 dry-run hint** (#391) — Stale `write_flash` (underscore) syntax in the dry-run manual-flash hint now uses `write-flash` (hyphenated) for esptool >= 5.x. The primary flash command was already correct.
- **`provision.py` silent NVS wipe** (#391) — The script replaces the entire `csi_cfg` NVS namespace on every run, so partial invocations were silently erasing WiFi credentials and causing `Retrying WiFi connection (10/10)` in the field. Now refuses to run without `--ssid`, `--password`, and `--target-ip` unless `--force-partial` is passed. `--force-partial` prints a warning listing which keys will be wiped.
- **Firmware: defensive `node_id` capture** (#232, #375, #385, #386, #390) — Users on multi-node deployments reported `node_id` reverting to the Kconfig default (`1`) in UDP frames and in the `csi_collector` init log, despite NVS loading the correct value. The root cause (memory corruption of `g_nvs_config`) has not been definitively isolated, but the UDP frame header is now tamper-proof: `csi_collector_init()` captures `g_nvs_config.node_id` into a module-local `s_node_id` once, and `csi_serialize_frame()` plus all other consumers (`edge_processing.c`, `wasm_runtime.c`, `display_ui.c`, `swarm_bridge_init`) read it via the new `csi_collector_get_node_id()` accessor. A canary logs `WARN` if `g_nvs_config.node_id` diverges from `s_node_id` at end-of-init, helping isolate the upstream corruption path. Validated on attached ESP32-S3 (COM8): NVS `node_id=2` propagates through boot log, capture log, init log, and byte[4] of every UDP frame.
+28 -1876
View File
File diff suppressed because it is too large Load Diff
@@ -60,8 +60,59 @@ Five concurrent lines of research have converged on the domain generalization pr
## 2. Decision
### 2.0 — 2026-Q2 Re-scope: MERIDIAN-MAE foundation pre-training (primary path)
> **Status of this subsection:** Active. Supersedes the *training strategy* of §2.1–§2.6 (the dual-path / domain-adversarial / geometry-conditioned *architecture* is retained — it becomes the **fine-tune-stage head** on top of a pre-trained encoder, not a from-scratch network).
> **Driver:** `docs/research/sota/2026-Q2-agentic-ai-and-edge-for-ruview.md` (§B1) and the 2025→2026 evidence below.
**What changed.** The 2026 WiFi-sensing literature converged on a single result: **masked-autoencoder (MAE) pre-training on large, heterogeneous CSI pools beats supervised baselines on cross-domain tasks, and the bottleneck is data breadth, not model capacity.**
- *Scale What Counts, Mask What Matters* (arXiv:2511.18792): pre-trains/evaluates across **14 datasets, >1.3 M CSI samples, 4 device types, 2.4/5/6 GHz**; **log-linear** cross-domain gains with pre-training data (+2.2 % to +15.7 % over supervised), **marginal** gains from bigger models.
- **CIG-MAE** (arXiv:2512.04723): dual-stream MAE reconstructing **both amplitude and phase**, with information-guided masking — phase reconstruction is now SOTA-competitive (historically the hard part).
- **AM-FM** (2026; arXiv:2602.11200, already cited in §1.2): ~9.2 M samples, ~20 device types — the data-breadth thesis at scale.
- *A Tutorial-cum-Survey on SSL for Wi-Fi Sensing* (arXiv:2506.12052) and ACM TOSN (10.1145/3715130): MAE is the consistently strongest SSL choice for CSI.
**Revised decision.** The primary MERIDIAN program is now a **three-stage** pipeline:
1. **Pre-train** a CIG-MAE-style **dual-stream (amplitude + phase) masked autoencoder** on every CSI source RuView can reach — own recordings (`data/recordings/`, overnight captures), MM-Fi + Wi-Pose (ADR-015), public CSI corpora, and the multi-band virtual-subcarrier streams from `ruvsense/multiband.rs`. Thesis: *data breadth > pose-net capacity*.
2. **Fine-tune** the existing MERIDIAN heads — the 17-keypoint / DensePose-UV regression heads, the AETHER contrastive embedding (ADR-024), and the domain-adversarial / geometry-conditioned layers of §2.1–§2.6 — on top of the **frozen-then-unfrozen** pre-trained encoder. The §2.x machinery is now *regularisation on a good representation* rather than the load-bearing structure.
3. **Adapt** per room with **source-free unsupervised domain adaptation** (MU-SHOT-Fi, arXiv:2605.01369; Wi-SFDAGR) wired behind `ruvsense/coherence_gate.rs::Recalibrate` — a bounded MicroLoRA-delta + EWC++ pass on the head, triggered by the coherence z-score, logged via the witness chain. (Tracked separately; see the companion ADR referenced in the survey's Part C #2.)
**Why this is better than from-scratch (§2.1 as the primary path).** A model trained from scratch on one or two single-environment datasets *cannot* see enough multipath/hardware diversity to learn an environment-agnostic representation — that's the layout-overfitting / multipath-memorisation failure in §1.1. A pre-trained encoder front-loads that diversity, so the SISO-multistatic ESP32 input (§B3) has to carry far less, and the per-room work shrinks to adaptation (stage 3), not retraining.
**Token convention (implementation).** A CSI window `[T, tx, rx, sub]` → a sequence of `N = T·tx·rx` tokens, each a `sub`-dim *channel snapshot* — the same `[B, T·tx·rx, sub]` layout `model.rs::ModalityTranslator` already consumes. Amplitude and phase share the token grid, so one mask drives both streams.
**Implementation status & plan.**
-**Iteration 1**: `wifi-densepose-train::csi_mae``MaeConfig` (+`validate`), `MaskStrategy`, `TokenLayout`, deterministic `mask_csi_window` / `reassemble_tokens` (pure Rust, dependency-free PRNG, unit tests, builds & tests under `cargo test --no-default-features`); the re-scoped ADR (this section).
-**Iteration 2a**: information-guided masking — `MaskStrategy::InfoGuided` now masks high-information tokens (token "information" = variance of amplitude + variance of phase), weighted-without-replacement via EfraimidisSpirakis, deterministic in seed; replaces the iter-1 Random fallback. +3 tests.
-**Iteration 2b** (CI-verified): `csi_mae::model` behind `tch-backend``CsiMae` (dual-stream amp+phase per-token embed → fuse → residual-MLP encoder over visible tokens → flatten-to-latent bottleneck → learned per-position query + broadcast latent → residual-MLP decoder → `dec_amp_head`/`dec_ph_head``index_select` the masked positions); `CsiMae::reconstruction_loss` (MSE amp + `phase_w`·MSE phase); `MaeBatch::from_windows` (partition from window 0, reused across the batch — `n_tokens` is fixed); `pretrain_step`; `src/bin/pretrain_mae.rs` (synthetic-data driver, `required-features = ["tch-backend"]`); a gated "loss halves when overfitting one batch" smoke test. v0 limits noted in the module docs: fixed `n_tokens`, batch-shared masking, MSE on unwrapped phase. The dev box that wrote this had no LibTorch, so the tch path is verified by CI (`tch-backend` feature), not locally.
-**Iteration 3+**: pool & ingest heterogeneous CSI (own recordings + MM-Fi + Wi-Pose + multi-band virtual sub-carriers); real pre-train run (GPU — `scripts/gcloud-train.sh` / the cognitum project); per-sample masking + self-attention transformer blocks (lift the v0 limits); fine-tune the §2.x heads on top of the pre-trained encoder; cross-domain eval (§4.6 protocol); ship the encoder as an RVF segment (§4.7).
-**Out of scope here**: the per-room SFDA adaptation (stage 3) — its own ADR.
#### Iteration 3 plan — heterogeneous-CSI ingest, GPU pre-train, fine-tune handoff
The remaining prototype work (the parts that can't run on the dev box):
1. **Heterogeneous-CSI ingest.** A `csi_mae`-adjacent loader that pools every reachable CSI source into a uniform `[T, tx, rx, sub]` window stream, normalising sub-carrier count to 56 (via `wifi-densepose-train::subcarrier::interpolate_subcarriers`) and amplitude scale per-frame:
- own captures: `data/recordings/*.csi.jsonl`, overnight recordings;
- `MmFiDataset` (ADR-015, NeurIPS-2023 MM-Fi, 114 sub-carriers → interpolate);
- Wi-Pose (ADR-015);
- multi-band virtual sub-carriers from `ruvsense/multiband.rs` (3 channels × 56 → 168) — treated as extra tokens, not extra streams;
- public CSI corpora as available.
Implemented as a `CsiDataset` impl (e.g. `PooledCsiDataset`) that round-robins / weights sources; `pretrain-mae` gains a `--datasets <spec>` flag selecting it instead of `SyntheticCsiDataset`. *Thesis (arXiv:2511.18792): breadth of this pool — devices, bands, rooms — is what buys cross-domain generalisation; the model stays small.*
2. **GPU pre-train run.** `scripts/pretrain-mae-gcloud.sh` (added this iteration — a thin mirror of `scripts/gcloud-train.sh`): provisions a GCloud VM in `cognitum-20260110`, builds `wifi-densepose-train` with `--features tch-backend,cuda`, runs `pretrain-mae`, downloads the `.ot` variable store, tears the VM down. Currently drives `SyntheticCsiDataset` (the smoke path); the `--data-dir`/`--datasets` plumbing for the real corpus is the one TODO in that script. *Not run as part of this prototype.*
3. **Lift the v0 model limits.** Per-sample masking (gather/scatter so each window in a batch can have its own mask), self-attention transformer blocks in the encoder/decoder (replacing the residual MLPs and the flatten-to-latent bottleneck — this also removes the fixed-`n_tokens` constraint), a circular phase-reconstruction loss.
4. **Fine-tune handoff.** Load the pre-trained `CsiMae` encoder weights into the `model::WiFiDensePoseModel` front-end (the `ModalityTranslator` slot), freeze for a warm-up, then unfreeze; train the 17-keypoint / DensePose-UV heads, the AETHER contrastive embedding (ADR-024), and the §2.1–§2.6 domain-adversarial / geometry-conditioned layers *as regularisers on top of the pre-trained representation*. A `train` sub-command flag (`--init-encoder <mae.ot>`) wires this.
5. **Cross-domain eval.** Run §4.6's protocol (leave-one-room-out / leave-one-device-out) on the fine-tuned model vs. the from-scratch baseline; the win condition is the +2.2 %…+15.7 % cross-domain band that 2511.18792 reports for MAE pre-training.
6. **Ship the encoder** as an RVF segment (§4.7) so deployments load a pre-trained backbone and only carry the small task head + per-room adapter (stage 3 / the SFDA ADR).
The remainder of this ADR (§2.1 onward) describes the **fine-tune-stage architecture** — read it as "the head and regularisers that sit on top of the §2.0 pre-trained encoder", not as a from-scratch design.
### 2.1 Architecture: Environment-Disentangled Dual-Path Transformer
> *(Now the fine-tune-stage head — see §2.0.)*
MERIDIAN adds a domain generalization layer between the CSI encoder and the pose/embedding heads. The core insight is explicit factorization: decompose the latent representation into a **pose-relevant** component (invariant across environments) and an **environment** component (captures room geometry, hardware, layout):
```
@@ -546,3 +597,12 @@ ADR-011 Proof-of-Reality ──→ ⏳ Independent (Python v1 issue, high pr
8. Ramesh, S. et al. (2025). "LatentCSI: High-resolution efficient image generation from WiFi CSI using a pretrained latent diffusion model." arXiv:2506.10605. https://arxiv.org/abs/2506.10605
9. Ganin, Y. et al. (2016). "Domain-Adversarial Training of Neural Networks." JMLR 17(59):1-35. https://jmlr.org/papers/v17/15-239.html
10. Perez, E. et al. (2018). "FiLM: Visual Reasoning with a General Conditioning Layer." AAAI 2018. arXiv:1709.07871. https://arxiv.org/abs/1709.07871
**2026-Q2 re-scope (§2.0) — masked-autoencoder foundation pre-training:**
11. "Scale What Counts, Mask What Matters: Evaluating Foundation Models for Zero-Shot Cross-Domain Wi-Fi Sensing." arXiv:2511.18792. https://arxiv.org/html/2511.18792 — 14 datasets / >1.3 M CSI samples; data-breadth > model-capacity.
12. "CIG-MAE: Cross-Modal Information-Guided Masked Autoencoder for Self-Supervised WiFi Sensing." arXiv:2512.04723. https://arxiv.org/html/2512.04723v1 — dual-stream amplitude+phase MAE, information-guided masking.
13. "MU-SHOT-Fi: Self-Supervised Multi-User Wi-Fi Sensing with Source-free Unsupervised Domain Adaptation." arXiv:2605.01369. https://arxiv.org/html/2605.01369 — per-room SFDA (MERIDIAN stage 3).
14. "A Tutorial-cum-Survey on Self-Supervised Learning for Wi-Fi Sensing: Trends, Challenges, and Outlook." arXiv:2506.12052. https://arxiv.org/html/2506.12052
15. "Evaluating Self-Supervised Learning for WiFi CSI-Based Human Activity Recognition." ACM Trans. Sensor Networks. https://dl.acm.org/doi/10.1145/3715130
16. RuView 2026-Q2 SOTA survey — `docs/research/sota/2026-Q2-agentic-ai-and-edge-for-ruview.md` (§B1, Part C #1).
@@ -0,0 +1,203 @@
# ADR-094: Live 3D Point Cloud Viewer — GitHub Pages Deployment with Optional Real-Data Stream
| Field | Value |
|---|---|
| **Status** | Proposed (2026-04-29) |
| **Date** | 2026-04-29 |
| **Authors** | ruv |
| **Related** | ADR-092 (nvsim dashboard Pages deployment), ADR-059 (live ESP32 CSI pipeline), ADR-079 (camera ground-truth training) |
| **Branch** | `feat/pointcloud-pages-demo` |
---
## 1. Context
The `wifi-densepose-pointcloud` crate ships a Three.js-based viewer
(`v2/crates/wifi-densepose-pointcloud/src/viewer.html`) that renders the
fused camera-depth + WiFi CSI + mmWave point cloud produced by the
`ruview-pointcloud serve` binary. Today the viewer is local-only:
- It is served by the Axum binary on `127.0.0.1:9880`.
- It polls `/api/splats` every 500 ms expecting a backend on the same
origin.
- There is no GitHub Pages deployment, so the README's
"▶ Live 3D Point Cloud" link points at the moved-content section in
`docs/readme-details.md`, not at a hosted demo. The two sibling demos
(Live Observatory, Dual-Modal Pose Fusion) are already hosted at
`https://ruvnet.github.io/RuView/` and `…/pose-fusion.html`.
This is an asymmetry: a first-time visitor can preview the WiFi pose
demo and the Observatory in one click, but cannot preview the point
cloud without cloning the repo, building Rust, plugging in an ESP32,
and pointing a webcam at themselves. That gap suppresses the most
visually compelling demonstration of the v0.7+ sensor-fusion work.
A naive fix — drop the static HTML at `gh-pages/pointcloud/` — does
not work because the viewer's `fetch("/api/splats")` will 404 on Pages
and the canvas will hang at "Loading…". A second naive fix — bake in a
fixed sample dataset — solves the loading state but loses the live-data
story entirely, and forks the viewer into a "demo build" and a "real
build" that drift apart.
## 2. Decision
Ship **one** viewer that auto-selects its transport from URL parameters,
and publish it to `gh-pages/pointcloud/` alongside the other demos:
1. **Default mode** — when the viewer is opened with no query parameters
on `https://ruvnet.github.io/RuView/pointcloud/`, present a "▶ Enable
camera" CTA. On click the viewer requests webcam access, runs
**MediaPipe Face Mesh** in-browser (~30 fps, 478 refined landmarks),
and renders the visitor's own face as a point cloud — the closest
browser equivalent of the local pipeline's depth-backprojected face
geometry that motivated this ADR (`I could see the outline of my face
in points`). The viewer mirrors x to match selfie convention and
maps Face Mesh's relative-z to the same world-coordinate range the
live `/api/splats` payload uses, so a single render path drives both.
Badge reads `● DEMO Your Face (MediaPipe)`. If the user denies
camera permission, dismisses the prompt, or visits on a device
without a webcam, the viewer falls back automatically to a
procedural scaffold (floor grid, walls, breathing figure, 17-keypoint
skeleton). All processing is client-side; no frames leave the
browser. ~480-500 splats from the face plus ~110 floor/wall context
splats.
2. **Auto mode** (`?backend=auto`) — fetch from `/api/splats` on the same
origin. This is the local-development case (`ruview-pointcloud serve`
serves the viewer and the API together). On any failure (404, network
error, CORS), fall back silently to synthetic-demo rendering so the
tab never dies.
3. **Remote mode** (`?backend=<url>`) — fetch from `<url>/api/splats`.
This is the **integrated-ESP32** path: the user runs
`ruview-pointcloud serve --bind 127.0.0.1:9880` locally with an
ESP32-S3 streaming CSI to UDP port 3333, then opens
`https://ruvnet.github.io/RuView/pointcloud/?backend=http://127.0.0.1:9880`.
The hosted Pages viewer becomes a thin client for the local Rust
fusion pipeline (camera depth + WiFi CSI + mmWave) without a clone
or rebuild. The viewer also exposes a "📡 Connect ESP32" button that
prompts for the URL, persists it in `localStorage`, and reloads
with the query param.
For this to work the local server must answer the browser's CORS
preflight. `stream.rs` therefore installs a `tower_http` `CorsLayer`
that allows three origin classes:
- `https://ruvnet.github.io` — the published Pages demo.
- `http://localhost:*` and `http://127.0.0.1:*` — developer running
the bundled `viewer.html` directly.
- `null``file://` origins.
Mixed-content (HTTPS Pages → HTTP loopback) is permitted because
modern browsers (Chrome 94+, Firefox 116+, Safari 16.4+) classify
`127.0.0.1` and `localhost` as "potentially trustworthy" origins.
Any other origin (a public hostname, etc.) is denied — this is not
a wildcard CORS posture. Badge reads `● REMOTE <url>`. Same silent
demo fallback on failure.
4. **Strict-live mode** (`?live=1`) — disable the demo fallback. If the
chosen transport fails, replace the info panel with an explicit offline
message (`● OFFLINE — Live backend required but unreachable`). Useful
for embedding the viewer in a status page or kiosk.
The synthetic frame returned by the in-browser generator matches the
JSON shape of the live `/api/splats` payload exactly (`splats`, `count`,
`frame`, `live`, `pipeline.{skeleton,vitals,…}`), so a single render path
drives both modes. There is no demo build vs real build — only one HTML
file, one render path, and one set of bugs.
A new GitHub Actions workflow (`.github/workflows/pointcloud-pages.yml`)
copies the viewer to `gh-pages/pointcloud/index.html` on every push to
`main` that touches the viewer, using `peaceiris/actions-gh-pages@v4`
with `keep_files: true` to preserve the existing observatory, pose-fusion,
and nvsim deployments.
## 3. Consequences
### Positive
- **First-click demo.** Visitors clicking the README's
"▶ Live 3D Point Cloud" link land on a working Three.js scene in <1 s,
no toolchain required. Matches the parity of the other two demos.
- **Real-data on demand.** Users with their own `ruview-pointcloud serve`
host can use the same hosted viewer URL with
`?backend=https://their-host.example.com` — no clone, no rebuild. The
hosted demo doubles as a thin client for self-hosted backends.
- **Single render path.** Synthetic frames flow through the same
`handleData → updateSplats → drawSkeleton` pipeline as live frames, so
visual regressions surface in the demo and the live build at the same
time. This is the same dual-transport pattern ADR-092 chose for nvsim.
- **No backend deploy required.** Pages serves static HTML; the demo
works without standing up an Axum host on the public internet, and
there is no per-visitor CSI/camera plumbing to provision.
- **Preserves existing deployments.** `keep_files: true` plus the
`pointcloud/` destination means observatory/, pose-fusion/, nvsim/,
and the root index.html on gh-pages are untouched.
### Negative / tradeoffs
- **Face mesh ≠ CSI.** Browser webcam + MediaPipe gives real face
geometry but does not produce CSI-derived pose. Visitors who want to
see the *WiFi-driven* path still need `?backend=<their-host>`. The
procedural fallback is not WiFi-driven either; it is purely visual
scaffolding. We accept this — the goal of the hosted demo is to
convey the *shape* of what the local pipeline produces (a point
cloud of the user) rather than reproduce the WiFi physics in the
browser. The latter is a future ADR (WASM port of the fusion crate).
- **CORS burden on remote mode.** Users who want to share their backend
must add `Access-Control-Allow-Origin: https://ruvnet.github.io` (or
`*`) to their `ruview-pointcloud serve` config. We document this in the
workflow's generated README; we do **not** add a public proxy.
- **Synthetic generator lives in the viewer.** ~80 LOC of procedural JS
is now part of `viewer.html`. Acceptable: the file is already the
client-side render bundle, and the generator is bounded and inert
(deterministic, no I/O, no eval).
- **No replay-from-recording in this ADR.** A future ADR may add a
`?recording=<url>.jsonl` mode that replays captured frames at native
rate; that is out of scope here.
### Neutral
- The local-dev experience is unchanged. `ruview-pointcloud serve` still
serves `viewer.html` from the bundled asset and the viewer still hits
`/api/splats` because `?backend` defaults to `auto`. Nothing in the
Rust crate changes — this is HTML + workflow only.
## 4. Implementation
| File | Change |
|---|---|
| `v2/crates/wifi-densepose-pointcloud/src/viewer.html` | Add URL-param transport selector (`backend`, `live`), synthetic frame generator, demo-fallback path, transport-aware mode badge. ~120 LOC added, no removed behavior. |
| `.github/workflows/pointcloud-pages.yml` | New workflow: stage viewer to `_site/pointcloud/index.html`, deploy to `gh-pages/pointcloud/` with `keep_files: true`. Triggers on viewer changes and on manual dispatch. |
| `README.md` | Already updated — `▶ Live 3D Point Cloud` link will be retargeted to `https://ruvnet.github.io/RuView/pointcloud/` once the first deploy succeeds. (Tracked separately, not blocking this ADR.) |
| `docs/adr/README.md` | ADR index — add ADR-094 row. |
## 5. Acceptance Gates
This ADR is **Implemented** when all of the following hold:
1. Pushing to `main` with a viewer change triggers
`pointcloud-pages.yml`, which deploys to `gh-pages/pointcloud/` in
under 60 seconds.
2. `https://ruvnet.github.io/RuView/pointcloud/` loads, shows the
"Enable camera" CTA, and on accept renders the visitor's face as a
point cloud with badge `● DEMO Your Face (MediaPipe)` and non-zero
splat + frame counts. On camera denial, falls back to the
procedural scene with badge `● DEMO Synthetic`.
3. Existing demos at `https://ruvnet.github.io/RuView/` and
`…/pose-fusion.html` and `…/nvsim/` are still reachable after the
first deploy (smoke-tested manually).
4. `https://ruvnet.github.io/RuView/pointcloud/?live=1` shows the
`● OFFLINE` panel (because no same-origin backend exists on Pages).
5. `https://ruvnet.github.io/RuView/pointcloud/?backend=https://example.invalid`
falls back to demo within one poll interval (~500 ms) without
throwing in the console.
6. Running `./target/release/ruview-pointcloud serve` locally and
opening `http://127.0.0.1:9880/` (which serves the same HTML) still
shows live-mode rendering with the `● LIVE Local Backend` badge.
## 6. Out of Scope
- Replaying recorded JSONL frames in the browser (future ADR).
- WASM-side execution of the fusion pipeline in the browser (would
require porting the camera + mmWave path; deferred).
- Authentication / signed splats payloads — backend-side concern,
unaffected by this client-side change.
- Hosting a public CORS proxy for users without their own backend.
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,241 @@
# Agentic-AI Breakthroughs & Related SOTA — Applicability to RuView (2026-Q2)
> **Status:** Research note — non-binding survey. Nothing here is an accepted decision.
> **Date:** 2026-05-11 · **Author:** research pass (Claude Code) · **Scope owner:** ruv
> **Companion docs:** `docs/research/sota/2026-Q2-rf-sensing-and-edge-rust.md`,
> `docs/research/sota-surveys/wifi-sensing-ruvector-sota-2026.md`,
> `docs/research/sota-surveys/ruview-multistatic-fidelity-sota-2026.md`
> **Maps onto ADRs:** 015, 016, 017, 024, 027, 028, 029032, 039, 040, 069, 081, 084086, 095, 096
---
## 0. TL;DR — the eight findings that matter for RuView
| # | Breakthrough / SOTA result | Why it matters here | Lands against | Horizon |
|---|---|---|---|---|
| 1 | **WiFi-sensing foundation models scale with *data*, not capacity** — a 14-dataset / 1.3 M-CSI-sample MAE study (arXiv 2511.18792) shows log-linear cross-domain gains from pre-training breadth; larger models barely help. AM-FM (2026) pushes this to ~9.2 M samples / 20 device types. | This is the answer to **MERIDIAN (ADR-027)**: the path to cross-room generalisation is a CSI MAE pre-trained on heterogeneous capture, then a tiny task head — *not* a bigger pose net. RuView already has the data-collection plumbing (`scripts/collect-training-data.py`, overnight recordings, MM-Fi/Wi-Pose under ADR-015). | ADR-015, ADR-016, ADR-027 | Medium |
| 2 | **MAE on amplitude *and* phase** — CIG-MAE (arXiv 2512.04723) reconstructs both with a symmetric dual-stream encoder + information-guided masking; phase reconstruction is now SOTA-competitive. | RuView throws away most phase information after `phase_align.rs` / `coherence.rs`. A dual-stream amplitude+phase MAE pre-text task fits the existing `wifi-densepose-train` graph and the AETHER contrastive head (ADR-024). | ADR-016, ADR-024 | Medium |
| 3 | **Source-free unsupervised domain adaptation works for Wi-Fi** — MU-SHOT-Fi (arXiv 2605.01369), Wi-SFDAGR — adapt a deployed model to a new environment with *no source data and no labels*, just the target stream. | This is exactly the **recalibration gate** decision in `ruvsense/coherence_gate.rs` (`Recalibrate`). Today that gate only flags drift; SFDA gives it something to *do* — adapt the head online, on-device, without phoning home. Pairs naturally with SONA / MicroLoRA + EWC++ already in the stack. | ADR-027, ADR-081, ADR-095/096 | Medium |
| 4 | **The "agent harness on the MCU" pattern** — ESP-Claw (Espressif) and the broader hybrid edge/cloud consensus: heavy reasoning stays in the cloud, but the *loop* (sense → decide → act, plus skills/memory/routing) runs on the microcontroller. ESP32-P4 (dual RISC-V 400 MHz, 768 KB SRAM, 32 MB PSRAM); ESP32-S3 vector ISA accelerates NN kernels. | RuView's ESP32 firmware is already a tiny agent: `adaptive_controller.c` (ADR-081) is the decide loop, the WASM tier (ADR-040) is the skill sandbox, `temporal_task.c` (ADR-095/096) is the on-device model, `edge_processing.c` is sensor fusion. Worth re-framing the firmware explicitly as a *constrained autonomous agent* and stealing patterns (skill registry, memory budget, watchdog-as-supervisor). ESP32-P4 is a credible next hardware tier (vs. the S3's ~200 KB free heap). | ADR-040, ADR-081, ADR-095/096, ADR-028 | Short→Medium |
| 5 | **Agent memory has matured into a discipline** — hierarchical episodic/semantic/procedural memory, the LOCOMO benchmark, the ICLR-2026 *MemAgents* workshop, "Memory in the Age of AI Agents" survey, mem0's "State of Agent Memory 2026". | RuView/ruflo already runs a ReasoningBank-style loop (HNSW-indexed trajectories, verdict→distil→consolidate). The new framing adds: (a) *procedural* memory as a first-class type — the **fix-marker witness guard** (`scripts/fix-markers.json`, merged 2026-05-11) is a primitive instance of this; (b) typed-memory eval (LOCOMO-style) for the dev-loop memory. | ADR-016 (memory side), repo CI | Short |
| 6 | **Long-horizon agents + continual learning are converging into "the system, not the model, is the unit of progress"** — METR's task-duration-doubling (~1 h tasks early-2025 → multi-hour by late-2026); Q2-2026 long-horizon agents from the frontier labs; continual-learning + world-models maturing together. | Two reads for RuView: (a) the *dev* workflow — multi-PR, multi-day swarm work is now tractable; the witness/fix-marker guard is the kind of "system memory" that makes long-horizon dev safe (don't silently revert). (b) the *product* — a RuView deployment that *learns its room over weeks* (longitudinal biomechanics drift in `ruvsense/longitudinal.rs`, the persistent field model ADR-030) is the same "world model that adapts" story, just for RF. | ADR-030, ADR-027, repo workflow | Medium |
| 7 | **Streaming / drift-tolerant vector quantisation** — CoDEQ (arXiv 2512.18335, in ruvector): frozen kd-tree + live leaf centroids via Welford → O(1) updates, no k-means retrain, 7.5× faster build than PQ, ~7 % standalone recall@10 (coarse pre-filter only). RaBitQ / Extended-RaBitQ (arXiv 2405.12497 + follow-ups) remains the high-recall 1-bit workhorse. | RuView's vector tier is RaBitQ + HNSW (ADR-084/085, ADR-016). CoDEQ is **not** a replacement (recall too low) and its "no k-means" advantage is over PQ, which RuView doesn't use. It's a potential *new tier* only if (a) on-ESP32 adaptive quant becomes a goal, or (b) HNSW build cost shows up in a profile as the mesh scales. Deferred — see Part C. | ADR-016, ADR-084/085 | Defer |
| 8 | **Agentic verification / "show your work" is now an explicit research theme** (eval harnesses, trajectory judging, reproducibility gates). | RuView is ahead of the curve here: the ADR-028 deterministic-pipeline proof (`archive/v1/data/proof/verify.py` → SHA-256), the release witness bundle, and the new fix-marker regression guard *are* the "verifiable agent output" pattern applied to a codebase. Worth writing this up as a reference pattern (it's reusable beyond RuView). | ADR-028, repo CI | Done / extend |
**One-line recommendation:** the highest-leverage move is **#1 + #2 + #3 together** — a heterogeneous-CSI masked-autoencoder pre-train (amplitude+phase) plus a source-free online-adaptation hook on the recalibration gate. That's the credible path to closing MERIDIAN, and every piece of plumbing it needs already exists in the repo. Everything else is incremental or already in flight.
---
## 1. Method & scope
This note surveys two adjacent literatures as of 2026-Q2 and filters hard for RuView relevance:
- **Agentic AI** — long-horizon agents, agent memory architectures, self-improving / continual-learning agents, multi-agent coordination, on-device ("edge") agents, retrieval & memory compression, agentic evaluation/verification.
- **RF / WiFi sensing** — CSI foundation models & self-supervised pre-training, domain adaptation, the DensePose-from-WiFi lineage, multistatic / distributed sensing, mmWave + WiFi fusion, adversarial robustness & privacy, through-wall / stand-off radar.
Selection bias is deliberate: a result is in only if there's a concrete hook into an existing RuView crate, ADR, or workflow. Pure-LLM-application work (browser agents, code agents, RAG-over-docs) is out except where it has changed how *this project's* dev loop or firmware should be structured. References (Part E) carry arXiv IDs / DOIs where available.
---
## PART A — Agentic-AI breakthroughs (the relevant slice)
### A1. Long-horizon agents and "the system is the unit of progress"
The headline 2026 shift isn't a model — it's that agents can now hold a goal across hours and many steps. METR's measurement (AI task duration doubling roughly every seven months) crossed from ~1-hour tasks in early 2025 toward multi-hour workstreams by late 2026, and the frontier labs are explicitly targeting "long-horizon agents" for H1-2026. Commentary across the field (adaline labs' "Beyond Transformers", the *Architecture of Agency* guide, MLM's "7 trends") converges on: when continual learning + long-horizon planning + world models mature *simultaneously*, the unit of engineering stops being "a model" and becomes "a system" — harness + memory + tools + verification.
**RuView implications.** Two, on two different timescales:
1. *Dev workflow* (already realised this session). Multi-PR, multi-day swarm work is tractable, and the failure mode is *forgetting / silently reverting* a hard-won fix. The fix-marker witness guard (`scripts/check_fix_markers.py` + `scripts/fix-markers.json`, the `Fix-Marker Regression Guard` workflow, merged in PR #526) is exactly the "system memory that survives the agent" primitive that makes long-horizon dev safe. It's a procedural-memory artifact (see A5) and a verification artifact (see A7) at once. **Action:** keep growing the manifest; treat it as the project's "don't regress" ledger.
2. *Product* (medium horizon). A RuView install that *learns a room over weeks* — longitudinal biomechanics drift (`ruvsense/longitudinal.rs`, Welford stats), the persistent field-model eigenstructure (ADR-030, `ruvsense/field_model.rs`), cross-room fingerprinting (`ruvsense/cross_room.rs`) — is the same "world model that adapts" pattern, in the RF domain. The agentic literature's framing (episodic→semantic→procedural consolidation) is a useful lens for organising what the deployed node should remember about its environment vs. discard.
### A2. Agent memory has become a discipline
2026 is the year "agent memory" stopped being an implementation detail. Signals: the **LOCOMO** benchmark for long-term conversational memory (the first apples-to-apples comparison of memory architectures); the **ICLR-2026 MemAgents workshop**; the *"Memory in the Age of AI Agents: A Survey"* paper list; mem0's "State of AI Agent Memory 2026". The settled taxonomy: **episodic** (what happened), **semantic** (distilled facts), **procedural** (how to do things / skills) — with consolidation passes promoting episodic → semantic → procedural, hierarchical stores, and (newer) multi-agent *shared* memory.
**RuView/ruflo already runs a ReasoningBank-style loop** — HNSW-indexed trajectory store, verdict-judge → distil → consolidate, experience replay (this is in the `.claude-flow/` coordination layer and the `reasoningbank-*` skills). What the 2026 framing adds:
- **Procedural memory as a first-class artifact.** The fix-marker manifest is procedural memory for the *codebase*: "the way we do CSI capture is `WIFI_PS_NONE` before promiscuous (#521); the way we handle sibling UDP magics is `ruview_sibling_packet_name` (#517); …". It's checked-in, diffable, CI-enforced. That's exactly what the literature recommends and almost nobody does. **Action:** generalise the pattern — a `docs/research/`-adjacent "decisions ledger" that links ADRs ↔ code markers ↔ tests is a natural extension.
- **Typed-memory evaluation.** A LOCOMO-style harness for the *dev-loop* memory (does recall surface the right ADR/pattern for a given task?) would be cheap and would catch memory rot. Low priority but easy.
- **Sensor-side memory.** The deployed ESP32 node has its own (tiny) episodic/semantic split: `edge_processing.c` ring buffer (episodic), the rolling vitals/presence stats (semantic), the NVS config + adaptive-controller policy (procedural). Worth being explicit about the budget (the S3 has ~200 KB free heap) and what gets consolidated vs. dropped.
### A3. Self-improving / continual-learning agents
The "self-improving" thread: agents that analyse past outcomes and evolve their strategies (this is the verdict→distil loop again, plus online weight updates). The hard problem is doing it without catastrophic forgetting. RuView is *already living in this space*: **SONA** (self-optimising neural architecture) + **MicroLoRA** online adaptation + **EWC++** for forgetting-resistance is named in the project's own positioning, and ADR-024 (AETHER contrastive CSI embedding) + ADR-095/096 (on-ESP32 temporal head with sparse GQA) are the substrate.
**What's new and worth pulling in:**
- **Source-free domain adaptation as the *update rule*** (see B2) — MU-SHOT-Fi / Wi-SFDAGR show you can adapt to a new environment with target stream only. Wire that as the action behind `coherence_gate.rs::Recalibrate`: when the coherence z-score gate says "this room has drifted", run an SFDA pass on the temporal head (MicroLoRA delta, EWC++ regulariser, bounded steps) instead of just degrading to `PredictOnly`.
- **Bounded, auditable online learning.** The fix-marker / witness culture should extend to learned weights: every on-device adaptation event should be logged (the firmware already has the witness-chain primitives from the Cognitum/`brain` work and ADR-028) so "the model in this room is now divergent from the shipped checkpoint" is *observable*, not silent — the same lesson as #505 (mislabelled firmware), one layer up.
### A4. Multi-agent coordination at the edge — RuView's mesh *is* a swarm
A 46 node ESP32 RuView deployment is, structurally, a multi-agent system: each node senses, runs an edge-tier pipeline, makes local decisions (channel hop, role, send-rate via `adaptive_controller.c`), and they fuse via TDM + multistatic attention. The agentic-systems literature (the arXiv 2601.12560 architectures/taxonomies paper; Byzantine-consensus and market-based task-allocation work) maps cleanly:
- **Role assignment / task allocation** — which node is the coordinator, which compute the per-cluster π hop (ADR-083), which go `PredictOnly` when degraded — is a market/auction problem. RuView does this ad hoc today; the literature has cleaner protocols.
- **Byzantine robustness** — ADR-032 (multistatic mesh security hardening) and `ruvsense/adversarial.rs` (physically-impossible-signal detection, multi-link consistency) are RuView's answer to "a node lies / is spoofed". The 2026 framing: treat it as Byzantine-fault-tolerant sensor fusion with an explicit fault model, not just heuristics.
- **Shared memory across the swarm** — the multi-agent-shared-memory thread suggests the nodes should converge on a shared *field model* (ADR-030) the way agents converge on shared semantic memory: CRDT-style, eventually consistent, with the coordinator as a soft leader (raft-ish). Some of this is sketched in `radio_ops.rs` (mesh header, node status, anomaly alerts).
**Action:** a short ADR re-framing the firmware mesh as a BFT sensor swarm with explicit role-auction + shared-field-model-as-CRDT would unify ADR-029/030/032/081/083 and give the implementation a literature to lean on. Low urgency, high coherence value.
### A5. The "agent harness on the MCU" pattern (most directly actionable)
Espressif's **ESP-Claw** crystallised something RuView half-built already: keep the LLM in the cloud, but run the *whole agent loop* — skills, tools, memory, routing, the sense→decide→act cycle — on the microcontroller. The hardware backs it: **ESP32-P4** (dual RISC-V @ 400 MHz, ~768 KB internal SRAM, 32 MB PSRAM, HW H.264) is Espressif's AI-focused part; the **ESP32-S3** vector ISA already accelerates NN kernels (which is why ADR-095/096's on-device temporal head is viable at all). The broader 2026 consensus is *hybrid*: MCU runs lightweight models + rule-based agents for real-time decisions, offload heavy reasoning only when needed.
**RuView's firmware is already a constrained agent — name it as one:**
| Agent component (ESP-Claw-ish) | RuView firmware equivalent |
|---|---|
| Decide loop | `adaptive_controller.c` (ADR-081) — fast/med/slow ticks, channel/role/send-rate policy |
| Skill sandbox | WASM tier (ADR-040) — uploadable `.wasm` modules, `on_timer()`, the upload/list/start/stop endpoints |
| On-device model | `temporal_task.c` (ADR-095/096) — sparse-GQA temporal head, emits `0xC5110007` classifications |
| Sensor fusion / perception | `edge_processing.c` — vitals, presence, fall detection, feature vectors; `csi_collector.c` — capture + the `WIFI_PS_NONE` fix |
| Memory | NVS config (procedural), ring buffer (episodic), rolling stats (semantic), witness chain (audit) |
| Supervisor / watchdog | `task_wdt` + the `EDGE_BATCH_LIMIT` yield discipline (#266/#321) — "kill the agent if it starves the idle task" |
| Telemetry / trajectory | UDP packet stream (`0xC5110001``0xC5110007`), boot log, OTA status |
**Actions worth doing:**
1. **An ADR that explicitly models the firmware as a tiered autonomous agent** (Tier-0 rules → Tier-1 WASM skills → Tier-2 on-device temporal head → Tier-3 cloud), with a stated memory budget and a "supervisor" contract. This mostly *documents and unifies* ADR-040/081/086/095/096 — but the framing buys clarity and a literature.
2. **Track ESP32-P4 as a real hardware tier.** The S3's ~200 KB free heap is the binding constraint on the temporal head and the WASM tier; the P4's 32 MB PSRAM changes that calculus. Worth a feasibility note (and it's a natural place for a bigger MAE-derived head).
3. **Steal ESP-Claw's skill-registry ergonomics** for the WASM tier (ADR-040) — versioned skills, declared capabilities, a deny-by-default policy (this echoes the `brain_sdk_allow`/`deny` pattern already in the `logi-brain` MCP surface).
### A6. Retrieval & memory compression
The fast-moving sub-area: 1-bit / few-bit vector quantisation with theoretical error bounds. **RaBitQ** (arXiv 2405.12497) and **Extended-RaBitQ** are the high-recall workhorses — and RuView already runs them (ADR-084 RaBitQ similarity sensor, ADR-085 pipeline expansion, plus the ruvector RaBitQ binding). **CoDEQ** (arXiv 2512.18335, in ruvector) is the new streaming/drift-tolerant entrant: frozen kd-tree + Welford-updated leaf centroids, O(1) updates, 7.5× faster build than PQ, but ~7 % standalone recall@10 — a coarse pre-filter, never a sole index. **Graph RAG / hybrid (sparse+dense) retrieval** with MMR diversity reranking is the other settled pattern (and `ruflo-rag-memory` already implements it for the dev loop).
**RuView read (see Part C for the verdict):** RaBitQ + HNSW (current) is the right stack for high-recall CSI/pose matching. CoDEQ is *deferred* — its "no k-means retrain" edge is over PQ (which RuView doesn't use), and there's no measured bottleneck it relieves. It becomes interesting only on the ESP32 (where a tiny, streaming, retrain-free quantiser might be the *only* thing that fits) — i.e. it's a possible ingredient of the ADR-095/096 line, not of the server-side index.
### A7. Agentic verification / "show your work"
A genuine 2026 research theme: eval harnesses, trajectory judging, reproducibility gates, "the agent must produce a verifiable artifact." RuView is, unusually, *ahead* here:
- **ADR-028 deterministic-pipeline proof** — `archive/v1/data/proof/verify.py` feeds a seeded reference signal through the production pipeline and SHA-256-hashes the output; `expected_features.sha256` pins it; `verify-pipeline.yml` re-runs it on every PR (twice, for determinism). This is a "tamper-evident agent output" by construction.
- **Release witness bundle** — `scripts/generate-witness-bundle.sh` → a recipient-verifiable `VERIFY.sh` packet (witness log + proof + test results + firmware hashes + crate versions).
- **Fix-marker regression guard** (new, merged PR #526) — `scripts/fix-markers.json` + `scripts/check_fix_markers.py` + the `Fix-Marker Regression Guard` workflow: every shipped fix asserts its own continued presence; reverting one fails CI; intentional removal forces a manifest diff (= the audit trail).
- **`firmware-ci.yml` version-guard** (new) — a release tag can't ship a binary whose `version.txt` doesn't match the tag (the #505 lesson, automated).
**Action:** write this up as a reusable pattern (it generalises well beyond RuView — it's basically "ReasoningBank verdicts, but for a repo"). A `docs/` note or a public gist; possibly an ADR ("verification artifacts as a project contract"). The `ruflo-core:witness` plugin is the cross-project version of the same idea — worth a cross-reference.
---
## PART B — RF / WiFi-sensing SOTA (related research)
### B1. WiFi-sensing foundation models & self-supervised CSI — *the big one*
The dominant 2025→2026 result: **masked autoencoding (MAE) pre-training on large, heterogeneous CSI pools beats supervised baselines on cross-domain tasks, and the bottleneck is data breadth, not model size.**
- *"Scale What Counts, Mask What Matters: Evaluating Foundation Models for Zero-Shot Cross-Domain Wi-Fi Sensing"* (arXiv 2511.18792) — pre-trains/evaluates across **14 datasets, >1.3 M CSI samples, 4 device types, 2.4/5/6 GHz**; finds **log-linear** cross-domain gains with pre-training data (+2.2 % to +15.7 % over supervised), **marginal** gains from bigger models. Tasks: activity, gesture, user-ID.
- **AM-FM** (2026) — billed as the first true WiFi foundation model: **~9.2 M samples, ~20 device types**.
- *"A Tutorial-cum-Survey on Self-Supervised Learning for Wi-Fi Sensing"* (arXiv 2506.12052) and the ACM TOSN evaluation (DOI 10.1145/3715130) — MAE is the consistently strong SSL choice for CSI.
- **CIG-MAE** (arXiv 2512.04723) — dual-stream MAE reconstructing **both amplitude and phase**, with information-guided masking (mask the high-info regions). Phase reconstruction is now competitive — historically the hard part.
- **CIRCSI consistency** (arXiv 2502.11965), **WiFo-CF** (arXiv 2508.04068) — channel/CSI-feedback foundation models from the comms side; relevant as architecture priors and for the multi-link / MIMO framing.
**RuView mapping.** This *is* the MERIDIAN program (ADR-027) — and RuView already has the pieces:
- Data plumbing: `scripts/collect-training-data.py`, `scripts/collect-ground-truth.py`, overnight CSI recordings, MM-Fi + Wi-Pose ingestion (ADR-015), the `data/recordings/` corpus.
- Training graph: `wifi-densepose-train` (ADR-016, ruvector-integrated) — a place to bolt an MAE pre-text head on.
- Embedding: AETHER contrastive head (ADR-024) — natural fine-tune target after MAE pre-train.
- Compression: `CompressedCsiBuffer` (`dataset.rs`, ruvector-temporal-tensor) — already streams CSI history; a sensible substrate for masked-token pre-training.
**Concrete plan (this is the recommendation):** ADR-027 should become "**heterogeneous-CSI MAE pre-train (amplitude+phase, CIG-MAE-style) → small task head**", with the explicit thesis *data breadth > pose-net capacity*. Phase 1: pool every CSI source RuView can reach (own recordings, MM-Fi, Wi-Pose, public CSI datasets, multi-band virtual subcarriers from `multiband.rs`) and run an MAE pre-train. Phase 2: fine-tune the 17-keypoint head + AETHER embedding on top. Phase 3: ship the encoder; the per-room work becomes adaptation (B2), not retraining.
### B2. Source-free / unsupervised domain adaptation
- **MU-SHOT-Fi** (arXiv 2605.01369) — self-supervised *multi-user* Wi-Fi sensing with **source-free** unsupervised domain adaptation: adapt with target stream only, no source data, no labels.
- **Wi-SFDAGR** — WiFi cross-domain gesture recognition via source-free domain adaptation (IEEE).
- *Self-supervised WiFi-based identity recognition in multi-user smart environments* (PMC12115556) — relevant for AETHER re-ID across rooms.
**RuView mapping.** This is the *missing action* behind `ruvsense/coherence_gate.rs::Recalibrate` and the recalibration recommendations in `coherence.rs` (`RECOMMEND_RECAL` quality flag, already on the wire in `rv_feature_state.h`). Today the gate detects environment drift and degrades; SFDA lets it *fix* itself: a bounded MicroLoRA-delta adaptation pass on the temporal head, EWC++-regularised, triggered by the coherence z-score, logged via the witness chain. Multi-user SFDA (MU-SHOT-Fi) is directly relevant because RuView's whole point is multi-person (the `DynamicPersonMatcher`, the COCO-17 multi-track output).
### B3. The DensePose-from-WiFi lineage — where RuView sits
Origin: **CMU's "DensePose From WiFi"** (Geng et al., arXiv 2301.00250, building on the 2022 RI thesis CMU-RI-TR-22-59) — UV-coordinate dense pose from CSI using 3×3 MIMO commercial NICs (Intel 5300 / Atheros). The honest gap (well-documented in RuView's own issues #506/#509): that work relies on rich multi-antenna spatial resolution; ESP32 is 1×1 SISO. RuView's bet is to recover spatial diversity *across nodes* (46 ESP32, TDM, multistatic attention-weighted fusion, 168 virtual subcarriers via 3 channels × 56) rather than within one rich NIC — plus a Rust pipeline at sub-50 ms (~800× over the original Python) and SONA on-device adaptation. The foundation-model results (B1) are what make the ESP32 path plausible: a strong pre-trained CSI encoder lowers how much the noisy SISO multistatic input has to carry.
### B4. Multistatic / distributed RF sensing & multi-band fusion
Active area, and RuView's `ruvsense/` is a fairly complete implementation of it: multi-band frame fusion + cross-channel coherence (`multiband.rs`), iterative LO phase-offset estimation (`phase_align.rs`), attention-weighted multistatic fusion with geometric diversity (`multistatic.rs`), RF tomography with an ISTA L1 solver (`tomography.rs`), the persistent room-eigenstructure field model (`field_model.rs`, ADR-030), cross-viewpoint attention with geometric bias and Cramér-Rao / Fisher-information bounds (`ruvector/src/viewpoint/`). The SOTA reading is mostly: the geometry-aware-attention + information-bound framing RuView already uses is the right one; the gaps are (a) a cleaner statistical fault model (→ A4, ADR-032) and (b) tying the field model to the foundation-model encoder (a pre-trained encoder + a per-room eigenstructure prior is a strong combo).
### B5. mmWave + WiFi fusion, vital signs
ADR-063 (mmWave sensor fusion, ESP32-C6 + Seeed MR60BHA2 over UART, 60 GHz FMCW) and ADR-021 (ESP32 CSI-grade vital-sign extraction, the `wifi-densepose-vitals` 4-stage pipeline) put RuView in the multimodal-vitals SOTA. The literature trend: WiFi gives coarse presence/macro-motion + room-scale coverage; 60 GHz FMCW gives precise HR/BR but narrow FOV; fusion (Kalman / attention) beats either. RuView's `mmwave_fusion_bridge.py` + the `0xC5110004` fused-vitals packet are the implementation. Watch: the cardiac/respiration-from-CSI-alone work keeps improving (RuView already does breathing/HR from CSI in `breathing.rs` / `bvp.rs`) — a good MAE pre-train (B1) should help here too since respiration is a periodic-structure problem.
### B6. Adversarial robustness & privacy
`ruvsense/adversarial.rs` (physically-impossible-signal detection, multi-link consistency) + ADR-032 (multistatic mesh security) are RuView's stake. The 2026 framing: (a) **spoofing/jamming** as Byzantine faults in a sensor swarm (→ A4) with a stated adversary model; (b) **privacy** — WiFi sensing is "camera-free" but still biometric (gait, breathing, re-ID embeddings are PII); the project already gestures at this (privacy logs in the ADR-084 RaBitQ-sensor framing), and the broader move is toward on-device-only processing + differential-privacy on any exported embedding. The `aidefence` / PII-detection surface in the ruflo toolchain is the dev-side analogue.
### B7. Through-wall / NLOS, stand-off radar tier
ADR-091 (stand-off radar tier research) and the single-sided through-wall thread (issue #424) sit here. SOTA: through-wall pose/activity is real but resolution-limited; multistatic helps (more look-angles see around the wall differently); the foundation-model encoders (B1) are starting to include NLOS data in their pre-training pools, which is the cleanest path to robustness. Mostly a "watch" item for RuView — the multi-node multistatic architecture is already the right substrate.
---
## PART C — Synthesis: what's actionable for RuView
Prioritised. "Effort" is rough; "horizon" is short (<1 mo) / medium (13 mo) / long (>3 mo) / defer.
| Rank | Action | Impact | Effort | Horizon | New ADR? | Notes |
|---|---|---|---|---|---|---|
| **1** | **MERIDIAN ⇒ heterogeneous-CSI MAE pre-train (amplitude+phase, CIG-MAE-style) → small task head.** Pool all reachable CSI (own recordings + MM-Fi + Wi-Pose + public + multi-band virtual subcarriers), MAE pre-train in `wifi-densepose-train`, fine-tune the 17-kpt + AETHER heads on top. | Very high — this is the cross-room story | High | Long | Re-scope ADR-027; possibly fold ADR-016 | All plumbing exists. Thesis: *data breadth > pose-net capacity* (2511.18792). |
| **2** | **Source-free online adaptation as the action behind `coherence_gate.rs::Recalibrate`.** Bounded MicroLoRA-delta + EWC++ pass on the temporal head, triggered by the coherence z-score, logged via the witness chain. | High — turns a "detect" into a "fix" | Medium | Medium | New ADR (sibling to ADR-027/081/095) | MU-SHOT-Fi / Wi-SFDAGR. Multi-user variant matters (RuView is multi-person). |
| **3** | **ADR: "firmware as a tiered autonomous agent."** Document & unify ADR-040/081/086/095/096 under the ESP-Claw-style agent model (Tier-0 rules → Tier-1 WASM skills → Tier-2 on-device temporal head → Tier-3 cloud), with a memory budget and a supervisor contract. | Medium — clarity + a literature to lean on | Low | Short | Yes (mostly documentation) | Cheap; high coherence value. |
| **4** | **Track ESP32-P4 as a hardware tier.** Feasibility note: 32 MB PSRAM vs. the S3's ~200 KB free heap changes what the temporal head / WASM tier / a bigger MAE-derived head can be. | Medium — unblocks the on-device-model ceiling | Low | Short | Note → ADR if pursued | Espressif's AI-focused part; S3 vector ISA stays the floor. |
| **5** | **Write up the verification-artifact pattern** (ADR-028 proof + witness bundle + fix-marker guard + version-guard) as a reusable reference; cross-link `ruflo-core:witness`. Keep growing `fix-markers.json`. | Medium — reuse beyond RuView; protects long-horizon dev | Low | Short | Optional ADR | RuView is ahead of the field here; make it legible. |
| **6** | **ADR: firmware mesh as a BFT sensor swarm** — explicit role-auction + shared-field-model-as-CRDT + a stated adversary model. Unifies ADR-029/030/032/081/083. | Medium — coherence; sets up ADR-032 properly | Medium | Medium | Yes | Lean on the agentic-systems / Byzantine-sensor-fusion literature. |
| **7** | **Phase-aware features end-to-end.** RuView discards most phase after `phase_align.rs`/`coherence.rs`; CIG-MAE shows phase reconstruction is now SOTA-competitive. Carry an amplitude+phase representation into the embedding. | Medium | Medium | Medium | Folds into #1 | Likely just falls out of #1 if the MAE is dual-stream. |
| **8** | **CoDEQ — DEFER, with a stub.** Add a one-paragraph "alternatives considered" to ADR-017: streaming/drift-tolerant coarse quant; no current bottleneck; revisit only if (a) on-ESP32 adaptive quant becomes a goal or (b) HNSW build/rebuild shows up in a profile as the mesh scales. | Low (now) | Trivial | Defer | Stub in ADR-017 | RaBitQ+HNSW is the right stack today; CoDEQ's "no k-means" edge is over PQ, which RuView doesn't use. |
**If you do one thing:** #1. If you do two: #1 + #2 (they're complementary — pre-train for breadth, adapt for the room). #3/#4/#5 are cheap and worth slipstreaming. #6 is housekeeping that pays off when ADR-032 gets serious. #7 is probably free given #1. #8 is "don't lose the idea."
---
## PART D — Watch list / open questions
- **AM-FM and the next WiFi foundation models** — when one ships with permissive weights + a CSI tokeniser RuView can adopt, #1 gets dramatically cheaper. Watch arXiv / Hugging Face.
- **Phase-faithful CSI capture on ESP32** — how much usable phase does the S3 actually deliver under the MGMT-only promiscuous regime (#396)? Worth a measurement; gates how much of CIG-MAE's amplitude+phase advantage RuView can realise on cheap hardware.
- **On-device MAE-derived heads** — is a distilled/quantised MAE encoder small enough for the S3 (or does it need the P4)? Determines whether the foundation model lives only server-side or also on the node.
- **LOCOMO-style eval for the dev-loop memory** — does ReasoningBank recall actually surface the right ADR/pattern? Cheap to measure; would catch memory rot.
- **Byzantine fault model for the mesh** — pin down the adversary (spoofed node? jammed link? compromised firmware? replay?) before ADR-032 implementation, not after.
- **Differential privacy on exported embeddings** — AETHER re-ID embeddings are biometric; if any leave the box (multi-room hand-off, cloud tier), what's the DP budget?
- **CoDEQ revisit trigger** — only if a profile shows HNSW build/rebuild as a bottleneck, or if on-ESP32 adaptive quant becomes a stated goal.
- **ESP-Claw / on-MCU agent frameworks** — track Espressif's releases; the skill-registry / capability-policy ergonomics are directly stealable for the WASM tier (ADR-040).
---
## PART E — References
WiFi / RF sensing:
- DensePose From WiFi — Geng et al., arXiv [2301.00250](https://arxiv.org/abs/2301.00250); CMU RI thesis CMU-RI-TR-22-59.
- Scale What Counts, Mask What Matters: Evaluating Foundation Models for Zero-Shot Cross-Domain Wi-Fi Sensing — arXiv [2511.18792](https://arxiv.org/html/2511.18792).
- CIG-MAE: Cross-Modal Information-Guided Masked Autoencoder for Self-Supervised WiFi Sensing — arXiv [2512.04723](https://arxiv.org/html/2512.04723v1).
- MU-SHOT-Fi: Self-Supervised Multi-User Wi-Fi Sensing with Source-free Unsupervised Domain Adaptation — arXiv [2605.01369](https://arxiv.org/html/2605.01369).
- A Tutorial-cum-Survey on Self-Supervised Learning for Wi-Fi Sensing — arXiv [2506.12052](https://arxiv.org/html/2506.12052).
- Evaluating Self-Supervised Learning for WiFi CSI-Based Human Activity Recognition — ACM TOSN, [10.1145/3715130](https://dl.acm.org/doi/10.1145/3715130).
- A MIMO Wireless Channel Foundation Model via CIRCSI Consistency — arXiv [2502.11965](https://arxiv.org/html/2502.11965).
- WiFo-CF: Wireless Foundation Model for CSI Feedback — arXiv [2508.04068](https://arxiv.org/pdf/2508.04068).
- Wi-SFDAGR: WiFi-Based Cross-Domain Gesture Recognition via Source-Free Domain Adaptation — IEEE (DOI per IEEE Xplore listing).
- Self-Supervised WiFi-Based Identity Recognition in Multi-User Smart Environments — PMC [PMC12115556](https://pmc.ncbi.nlm.nih.gov/articles/PMC12115556/).
- (project context) RuView issues #68 (MERIDIAN/ADR-027), #506, #509, #424.
Agentic AI / memory / edge:
- Agentic Artificial Intelligence (AI): Architectures, Taxonomies, and Evaluation of LLM Agents — arXiv [2601.12560](https://arxiv.org/html/2601.12560v1).
- MemAgents: Memory for LLM-Based Agentic Systems — ICLR-2026 workshop proposal, OpenReview [U51WxL382H](https://openreview.net/pdf?id=U51WxL382H).
- Memory in the Age of AI Agents: A Survey — paper list: github.com/Shichun-Liu/Agent-Memory-Paper-List.
- 2026 agent papers collection — github.com/VoltAgent/awesome-ai-agent-papers.
- State of AI Agent Memory 2026 — mem0.ai/blog/state-of-ai-agent-memory-2026.
- LOCOMO benchmark (long-term conversational memory) — see the mem0 and agent-memory survey references.
- METR — measuring AI ability to complete long tasks (task-duration doubling).
- ESP-Claw / on-MCU AI agents — Espressif; xda-developers coverage; ESP32-P4 / ESP32-S3 vector ISA datasheets.
- (project) ReasoningBank — the trajectory verdict→distil→consolidate loop RuView/ruflo implements; `reasoningbank-*` skills.
Retrieval / quantisation:
- RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound — arXiv [2405.12497](https://arxiv.org/abs/2405.12497); Extended-RaBitQ follow-ups.
- CoDEQ: streaming vector quantisation (frozen kd-tree + Welford leaf centroids) — arXiv 2512.18335; in `ruvector`. Gist: https://gist.github.com/ruvnet/d10fe656bd0fa68b4eb873ad299c6d4e.
RuView internal (for the mapping):
- ADRs: 014, 015, 016, 017, 021, 022, 024, 027, 028, 029, 030, 031, 032, 039, 040, 045, 060, 061, 062, 063, 069, 080086, 089096 — see `docs/adr/`.
- Crates: `wifi-densepose-{core,signal,nn,train,mat,hardware,ruvector,api,db,config,wasm,cli,sensing-server,wifiscan,vitals}`, `nvsim` — see project `CLAUDE.md`.
- Modules: `ruvsense/*` (signal crate), `viewpoint/*` (ruvector crate); firmware `main/*.c`.
- Verification artifacts: `archive/v1/data/proof/verify.py`, `scripts/generate-witness-bundle.sh`, `scripts/check_fix_markers.py` + `scripts/fix-markers.json`, `.github/workflows/{verify-pipeline,firmware-ci,fix-regression-guard}.yml`.
- Related surveys in this repo: `docs/research/sota/2026-Q2-rf-sensing-and-edge-rust.md`, `docs/research/sota-surveys/{wifi-sensing-ruvector-sota-2026,ruview-multistatic-fidelity-sota-2026,sota-wifi-sensing-2025}.md`, `docs/research/rf-topological-sensing/*`.
---
*Generated by Claude Code (research pass), 2026-05-11. Treat as input to ADR discussions, not as decisions.*
+121 -27
View File
@@ -25,13 +25,20 @@
/* ADR-060: Access the global NVS config for MAC filter and channel override. */
extern nvs_config_t g_nvs_config;
/* Defensive fix (#232, #375, #385, #386, #390): capture node_id at init-time
* into a module-local static. Using the global g_nvs_config.node_id directly
* at every callback is vulnerable to any memory corruption that clobbers the
* struct (which users have reported reverting node_id to the Kconfig default
* of 1). The local copy is set once at csi_collector_init() and then used
* exclusively by csi_serialize_frame(). */
/* Defensive fix (#232, #375, #385, #386, #390): capture NVS config fields into
* module-local statics BEFORE wifi_init_sta() runs, because WiFi driver init
* can corrupt g_nvs_config (confirmed on device 80:b5:4e:c1:be:b8).
* main.c calls csi_collector_set_node_id() immediately after nvs_config_load(),
* and all runtime paths use the local copies exclusively. */
static uint8_t s_node_id = 1;
static bool s_node_id_early_set = false;
/* Defensive copy of MAC filter config — the CSI callback fires at 100-500 Hz
* and reads filter_mac_set + filter_mac on every invocation. If wifi_init_sta()
* corrupts g_nvs_config, the callback would read garbage, potentially causing
* LoadProhibited panics (observed: Core 0 panic after ~2400 callbacks). */
static uint8_t s_filter_mac[6] = {0};
static bool s_filter_mac_set = false;
/* ADR-057: Build-time guard — fail early if CSI is not enabled in sdkconfig.
* Without this, the firmware compiles but crashes at runtime with:
@@ -60,6 +67,24 @@ static uint32_t s_rate_skip = 0;
#define CSI_MIN_SEND_INTERVAL_US (20 * 1000)
static int64_t s_last_send_us = 0;
/**
* Minimum interval between processing ANY CSI callback in microseconds.
* Promiscuous MGMT+DATA can fire 100-500+ times/sec. At rates above ~50 Hz,
* the WiFi FIQ handler (wDev_ProcessFiq) races with SPI flash cache operations,
* causing Core 0 LoadProhibited panics in cache_ll_l1_resume_icache.
*
* This early gate drops excess callbacks BEFORE any processing (serialization,
* UDP, edge enqueue), keeping the effective callback rate at ~50 Hz while
* preserving the full MGMT+DATA promiscuous filter and HT-LTF/STBC CSI quality.
*
* The WiFi hardware still captures all frames and the CSI data is generated,
* but we simply discard the excess in software. This reduces the time spent
* in callback context per second, giving the WiFi ISR more headroom.
*/
#define CSI_MIN_PROCESS_INTERVAL_US (20 * 1000) /* 50 Hz */
static int64_t s_last_process_us = 0;
static uint32_t s_early_drop = 0;
/* ---- ADR-029: Channel-hop state ---- */
/** Channel hop table (populated from NVS at boot or via set_hop_table). */
@@ -165,9 +190,20 @@ static void wifi_csi_callback(void *ctx, wifi_csi_info_t *info)
{
(void)ctx;
/* ADR-060: MAC address filtering — drop frames from non-matching sources. */
if (g_nvs_config.filter_mac_set) {
if (memcmp(info->mac, g_nvs_config.filter_mac, 6) != 0) {
/* Early rate gate: drop excess callbacks to ~50 Hz to prevent
* SPI flash cache crash in WiFi ISR (wDev_ProcessFiq). */
int64_t now_us = esp_timer_get_time();
if ((now_us - s_last_process_us) < CSI_MIN_PROCESS_INTERVAL_US) {
s_early_drop++;
return;
}
s_last_process_us = now_us;
/* ADR-060: MAC address filtering — drop frames from non-matching sources.
* Uses defensively-copied s_filter_mac instead of g_nvs_config (which can
* be corrupted by wifi_init_sta — same root cause as the node_id clobber). */
if (s_filter_mac_set) {
if (memcmp(info->mac, s_filter_mac, 6) != 0) {
return; /* Source MAC doesn't match filter — skip frame. */
}
}
@@ -222,14 +258,60 @@ static void wifi_promiscuous_cb(void *buf, wifi_promiscuous_pkt_type_t type)
(void)type;
}
void csi_collector_set_node_id(uint8_t node_id)
{
s_node_id = node_id;
s_node_id_early_set = true;
ESP_LOGI(TAG, "Early capture node_id=%u (before WiFi init, #232/#390)",
(unsigned)node_id);
/* Also capture MAC filter config now — same struct, same corruption risk.
* The CSI callback reads filter_mac_set on every invocation (100-500 Hz),
* so a corrupted value could cause erratic filtering or crash. */
s_filter_mac_set = (g_nvs_config.filter_mac_set != 0);
if (s_filter_mac_set) {
memcpy(s_filter_mac, g_nvs_config.filter_mac, 6);
ESP_LOGI(TAG, "Early capture filter_mac=%02x:%02x:%02x:%02x:%02x:%02x",
s_filter_mac[0], s_filter_mac[1], s_filter_mac[2],
s_filter_mac[3], s_filter_mac[4], s_filter_mac[5]);
}
}
void csi_collector_init(void)
{
/* Capture node_id into module-local static at init time. After this point
* csi_serialize_frame() uses s_node_id exclusively, isolating the UDP
* frame node_id field from any memory corruption of g_nvs_config. */
s_node_id = g_nvs_config.node_id;
ESP_LOGI(TAG, "Captured node_id=%u at init (defensive copy for #232/#375/#385/#390)",
(unsigned)s_node_id);
if (!s_node_id_early_set) {
/* Fallback: no early capture — use current g_nvs_config (may be clobbered). */
s_node_id = g_nvs_config.node_id;
ESP_LOGW(TAG, "Late capture node_id=%u (no early set_node_id call)",
(unsigned)s_node_id);
} else if (g_nvs_config.node_id != s_node_id) {
/* Canary: early capture disagrees with current g_nvs_config — corruption
* happened between nvs_config_load() and here (likely wifi_init_sta). */
ESP_LOGW(TAG, "node_id clobber CONFIRMED: early=%u g_nvs_config=%u "
"(WiFi init likely corrupted struct, using early value)",
(unsigned)s_node_id, (unsigned)g_nvs_config.node_id);
} else {
ESP_LOGI(TAG, "node_id=%u verified (early capture matches g_nvs_config)",
(unsigned)s_node_id);
}
/* Canary for filter_mac: check if WiFi init corrupted the filter fields. */
if (s_node_id_early_set) {
bool mac_set_now = (g_nvs_config.filter_mac_set != 0);
if (mac_set_now != s_filter_mac_set) {
ESP_LOGW(TAG, "filter_mac_set clobber CONFIRMED: early=%d g_nvs_config=%d",
(int)s_filter_mac_set, (int)mac_set_now);
} else if (s_filter_mac_set &&
memcmp(s_filter_mac, g_nvs_config.filter_mac, 6) != 0) {
ESP_LOGW(TAG, "filter_mac clobber CONFIRMED: bytes differ after WiFi init");
}
} else {
/* No early capture — grab filter config now (may already be corrupted). */
s_filter_mac_set = (g_nvs_config.filter_mac_set != 0);
if (s_filter_mac_set) {
memcpy(s_filter_mac, g_nvs_config.filter_mac, 6);
}
}
/* ADR-060: Determine the CSI channel.
* Priority: 1) NVS override (--channel), 2) connected AP channel, 3) Kconfig default. */
@@ -254,18 +336,40 @@ void csi_collector_init(void)
/* Update the hop table's first channel to match. */
s_hop_channels[0] = csi_channel;
/* Disable WiFi modem sleep — reliable CSI capture needs the radio awake.
* The ESP-IDF STA default is WIFI_PS_MIN_MODEM, which lets the modem
* sleep between DTIM beacons; with the MGMT-only promiscuous filter
* (RuView#396) that starves the CSI callback and the per-second yield
* collapses toward 0 pps (RuView#521). Operators who want battery
* duty-cycling opt back in via power_mgmt_init() (provision.py
* --duty-cycle <N>), which runs after this and re-enables modem sleep. */
esp_err_t ps_err = esp_wifi_set_ps(WIFI_PS_NONE);
if (ps_err != ESP_OK) {
ESP_LOGW(TAG, "esp_wifi_set_ps(WIFI_PS_NONE) failed: %s — CSI yield may be low",
esp_err_to_name(ps_err));
} else {
ESP_LOGI(TAG, "WiFi modem sleep disabled (WIFI_PS_NONE) for CSI capture");
}
/* Enable promiscuous mode — required for reliable CSI callbacks.
* Without this, CSI only fires on frames destined to this station,
* which may be very infrequent on a quiet network. */
ESP_ERROR_CHECK(esp_wifi_set_promiscuous(true));
ESP_ERROR_CHECK(esp_wifi_set_promiscuous_rx_cb(wifi_promiscuous_cb));
/* MGMT-only promiscuous filter + active probe injection (RuView#396).
*
* DATA frames cause 100-500+ WiFi HW interrupts/sec which crashes Core 0
* in wDev_ProcessFiq (SPI flash cache race in ESP-IDF WiFi blob).
* MGMT-only gives ~10 Hz (beacons). Probe request injection at 10 Hz
* adds ~10 Hz probe responses from APs → ~20 Hz total, matching the
* edge processing designed sample rate of 20 Hz. */
wifi_promiscuous_filter_t filt = {
.filter_mask = WIFI_PROMIS_FILTER_MASK_MGMT | WIFI_PROMIS_FILTER_MASK_DATA,
.filter_mask = WIFI_PROMIS_FILTER_MASK_MGMT,
};
ESP_ERROR_CHECK(esp_wifi_set_promiscuous_filter(&filt));
ESP_LOGI(TAG, "Promiscuous mode enabled for CSI capture");
ESP_LOGI(TAG, "Promiscuous mode enabled (MGMT-only, RuView#396)");
wifi_csi_config_t csi_config = {
.lltf_en = true,
@@ -290,16 +394,6 @@ void csi_collector_init(void)
ESP_LOGI(TAG, "CSI collection initialized (node_id=%u, channel=%u)",
(unsigned)s_node_id, (unsigned)csi_channel);
/* Clobber-detection canary: if g_nvs_config.node_id no longer matches the
* value we captured, something corrupted the struct between nvs_config_load
* and here. This is the historic #232/#375 symptom. */
if (g_nvs_config.node_id != s_node_id) {
ESP_LOGW(TAG, "node_id clobber detected: captured=%u but g_nvs_config=%u "
"(frames will use captured value %u). Please report to #390.",
(unsigned)s_node_id, (unsigned)g_nvs_config.node_id,
(unsigned)s_node_id);
}
}
/* Accessor for other modules that need the authoritative runtime node_id. */
+16 -6
View File
@@ -30,14 +30,24 @@
void csi_collector_init(void);
/**
* Get the runtime node_id captured at csi_collector_init().
* Capture node_id BEFORE wifi_init_sta() or any other heavy init.
*
* This is a defensive copy of g_nvs_config.node_id taken at init time. Other
* modules (edge_processing, wasm_runtime, display_ui) should prefer this
* accessor over reading g_nvs_config.node_id directly, because the global
* struct can be clobbered by memory corruption (see #232, #375, #385, #390).
* Must be called from app_main() immediately after nvs_config_load().
* WiFi driver initialization can corrupt g_nvs_config.node_id (confirmed
* on device 80:b5:4e:c1:be:b8, NVS=3 but post-WiFi reads as 1).
* This early capture shields s_node_id from that corruption window.
*
* @return Node ID (0-255) as loaded from NVS or Kconfig default at boot.
* @param node_id Value from g_nvs_config.node_id, read right after NVS load.
*/
void csi_collector_set_node_id(uint8_t node_id);
/**
* Get the runtime node_id (early capture if available, otherwise init-time).
*
* Other modules (edge_processing, wasm_runtime, display_ui) should prefer
* this accessor over reading g_nvs_config.node_id directly.
*
* @return Node ID (0-255) as loaded from NVS at boot.
*/
uint8_t csi_collector_get_node_id(void);
@@ -714,8 +714,11 @@ static void process_frame(const edge_ring_slot_t *slot)
s_frame_count++;
s_latest_rssi = slot->rssi;
/* Assumed CSI sample rate (~20 Hz for typical ESP32 CSI). */
const float sample_rate = 20.0f;
/* CSI sample rate. MGMT-only promiscuous filter (RuView#396, csi_collector.c)
* yields ~10 Hz from beacons; keep this value aligned with csi_collector's
* effective callback rate or estimate_bpm_zero_crossing() reports the wrong
* BPM (2× rate mismatch → 2× wrong breathing/HR). */
const float sample_rate = 10.0f;
/* --- Step 1-2: Phase extraction + unwrapping per subcarrier --- */
float phases[EDGE_MAX_SUBCARRIERS];
+5
View File
@@ -140,6 +140,11 @@ void app_main(void)
/* Load runtime config (NVS overrides Kconfig defaults) */
nvs_config_load(&g_nvs_config);
/* Capture node_id IMMEDIATELY — before wifi_init_sta() can corrupt
* g_nvs_config. See #232/#375/#390: WiFi driver init clobbers the struct
* on some devices, reverting node_id to the Kconfig default of 1. */
csi_collector_set_node_id(g_nvs_config.node_id);
const esp_app_desc_t *app_desc = esp_app_get_description();
ESP_LOGI(TAG, "ESP32-S3 CSI Node (ADR-018) — v%s — Node ID: %d",
app_desc->version, g_nvs_config.node_id);
+4 -1
View File
@@ -155,7 +155,10 @@ def flash_nvs(port, baud, nvs_bin):
"--chip", "esp32s3",
"--port", port,
"--baud", str(baud),
"write-flash",
# Keep underscore form — ESP-IDF v5.4 bundles esptool 4.10.0 which only
# accepts "write_flash". pip's esptool >=5.x accepts both (hyphenated
# form preferred) but keeps underscore working. Do not "correct" this.
"write_flash",
hex(NVS_PARTITION_OFFSET), bin_path,
]
print(f"Flashing NVS partition ({len(nvs_bin)} bytes) to {port}...")
+2 -3
View File
@@ -32,6 +32,5 @@ CONFIG_LWIP_SO_RCVBUF=y
# FreeRTOS: increase task stack for CSI processing
CONFIG_ESP_MAIN_TASK_STACK_SIZE=8192
# ADR-081: adaptive_controller runs emit_feature_state + stream_sender
# network I/O inside Timer Svc callbacks, exceeding the 2 KiB default.
CONFIG_FREERTOS_TIMER_TASK_STACK_DEPTH=8192
# Extra WiFi IRAM placement (defense-in-depth for RuView#396 SPI cache race)
CONFIG_ESP_WIFI_EXTRA_IRAM_OPT=y
+1 -1
View File
@@ -1 +1 @@
0.6.2
0.6.4
+190
View File
@@ -0,0 +1,190 @@
#!/usr/bin/env python3
"""Fix-marker regression guard for RuView.
Reads ``scripts/fix-markers.json`` and asserts that every previously-shipped
fix is still present in the codebase:
* every file listed in a marker must exist;
* every ``require`` pattern must appear in at least one of the marker's files
(a missing pattern means the fix was probably reverted);
* no ``forbid`` pattern may appear in any of the marker's files
(a re-appearing anti-pattern means the bug was re-introduced).
A pattern is a literal substring by default. Wrap it in ``/.../`` to treat it
as a (multiline, case-sensitive) regular expression, e.g. ``"/fall_thresh\\s*=\\s*2\\.0/"``.
This is a stdlib-only script — no dependencies, runs anywhere Python 3.8+ does.
Usage::
python scripts/check_fix_markers.py # check everything (CI)
python scripts/check_fix_markers.py --list # list all markers
python scripts/check_fix_markers.py --json # machine-readable result
python scripts/check_fix_markers.py --only RuView#396 RuView#521
Exit codes: 0 = all markers OK, 1 = one or more regressions, 2 = bad manifest.
"""
from __future__ import annotations
import argparse
import json
import re
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
MANIFEST_PATH = REPO_ROOT / "scripts" / "fix-markers.json"
# Best-effort UTF-8 stdout (Windows consoles default to cp1252); harmless on
# Linux/CI where it's already UTF-8. We still keep all symbols ASCII below so
# the script works even if reconfigure() is unavailable.
try: # pragma: no cover - environment-dependent
sys.stdout.reconfigure(encoding="utf-8", errors="replace")
except Exception:
pass
# ANSI colours — disabled automatically when stdout isn't a TTY (CI logs are
# plain either way, but keep them readable locally).
_TTY = sys.stdout.isatty()
def _c(code: str, s: str) -> str:
return f"\033[{code}m{s}\033[0m" if _TTY else s
GREEN = lambda s: _c("32", s)
RED = lambda s: _c("31", s)
YELLOW = lambda s: _c("33", s)
DIM = lambda s: _c("2", s)
BOLD = lambda s: _c("1", s)
OK_MARK = "PASS"
BAD_MARK = "FAIL"
ARROW = "->"
class ManifestError(Exception):
pass
def load_manifest() -> dict:
if not MANIFEST_PATH.exists():
raise ManifestError(f"manifest not found: {MANIFEST_PATH}")
try:
data = json.loads(MANIFEST_PATH.read_text(encoding="utf-8"))
except json.JSONDecodeError as e:
raise ManifestError(f"manifest is not valid JSON: {e}") from e
if not isinstance(data, dict) or not isinstance(data.get("markers"), list):
raise ManifestError("manifest must be an object with a 'markers' array")
ids = [m.get("id") for m in data["markers"]]
dupes = {i for i in ids if ids.count(i) > 1}
if dupes:
raise ManifestError(f"duplicate marker ids: {sorted(dupes)}")
return data
def _pattern_found(text: str, pattern: str) -> bool:
if len(pattern) >= 2 and pattern.startswith("/") and pattern.endswith("/"):
return re.search(pattern[1:-1], text, re.MULTILINE) is not None
return pattern in text
def check_marker(marker: dict) -> tuple[bool, list[str]]:
"""Return (ok, problems) for a single marker."""
problems: list[str] = []
files = marker.get("files", [])
require = marker.get("require", [])
forbid = marker.get("forbid", [])
if not files:
problems.append("marker lists no files")
return False, problems
contents: dict[str, str] = {}
for rel in files:
p = REPO_ROOT / rel
if not p.exists():
problems.append(f"missing file: {rel}")
continue
try:
contents[rel] = p.read_text(encoding="utf-8", errors="replace")
except OSError as e:
problems.append(f"cannot read {rel}: {e}")
haystack = "\n".join(contents.values())
for pat in require:
if not _pattern_found(haystack, pat):
problems.append(f"required marker absent (fix likely reverted): {pat!r}")
for pat in forbid:
for rel, text in contents.items():
if _pattern_found(text, pat):
problems.append(f"forbidden pattern re-appeared in {rel} (bug re-introduced?): {pat!r}")
return (len(problems) == 0), problems
def cmd_list(manifest: dict) -> int:
print(BOLD(f"{len(manifest['markers'])} fix markers tracked:\n"))
for m in manifest["markers"]:
print(f" {BOLD(m['id']):<28} {m.get('title', '')}")
if m.get("ref"):
print(DIM(f" {m['ref']}"))
for f in m.get("files", []):
print(DIM(f" - {f}"))
return 0
def main(argv: list[str]) -> int:
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--list", action="store_true", help="list all markers and exit")
ap.add_argument("--json", action="store_true", help="emit a JSON result object")
ap.add_argument("--only", nargs="+", metavar="ID", help="only check the given marker ids")
args = ap.parse_args(argv)
try:
manifest = load_manifest()
except ManifestError as e:
print(RED(f"[manifest error] {e}"), file=sys.stderr)
return 2
if args.list:
return cmd_list(manifest)
markers = manifest["markers"]
if args.only:
wanted = set(args.only)
markers = [m for m in markers if m["id"] in wanted]
unknown = wanted - {m["id"] for m in markers}
if unknown:
print(RED(f"[error] unknown marker id(s): {sorted(unknown)}"), file=sys.stderr)
return 2
results = []
failed = 0
for m in markers:
ok, problems = check_marker(m)
results.append({"id": m["id"], "title": m.get("title", ""), "ok": ok, "problems": problems})
if not ok:
failed += 1
if args.json:
print(json.dumps({"ok": failed == 0, "checked": len(markers), "failed": failed, "markers": results}, indent=2))
return 0 if failed == 0 else 1
print(BOLD(f"Fix-marker regression guard - {len(markers)} marker(s)\n"))
for r in results:
if r["ok"]:
print(f" {GREEN('[' + OK_MARK + ']')} {r['id']:<28} {DIM(r['title'])}")
else:
print(f" {RED('[' + BAD_MARK + ']')} {BOLD(r['id']):<28} {r['title']}")
for p in r["problems"]:
print(f" {RED(ARROW)} {p}")
print()
if failed:
print(RED(BOLD(f"{failed}/{len(markers)} marker(s) regressed.")))
print(DIM(" A reverted fix is a regression. Restore the marker, or - if the change is"))
print(DIM(" intentional - update scripts/fix-markers.json in the same PR with a rationale."))
return 1
print(GREEN(BOLD(f"All {len(markers)} fix markers present.")))
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+115
View File
@@ -0,0 +1,115 @@
{
"_comment": "Fix-marker regression guard for RuView. Each marker asserts that a previously-shipped fix is still present. CI (.github/workflows/fix-regression-guard.yml) fails if a `require` pattern is missing from all of a marker's `files` (the fix was likely reverted) or if a `forbid` pattern reappears (the bug was re-introduced). Run locally: `python scripts/check_fix_markers.py` (or `--list`, `--json`, `--only ID`). Patterns are literal substrings unless wrapped in /.../ (regex). Add a marker whenever you ship a fix that would be expensive to silently lose.",
"schema_version": 1,
"markers": [
{
"id": "RuView#396",
"title": "ESP32-S3 CSI: MGMT-only promiscuous filter (SPI flash cache race crash fix)",
"files": ["firmware/esp32-csi-node/main/csi_collector.c"],
"require": ["WIFI_PROMIS_FILTER_MASK_MGMT", "RuView#396"],
"rationale": "Promiscuous MGMT+DATA produces 100-500 Hz HW interrupts that crash Core 0 in wDev_ProcessFiq (SPI flash cache race in the WiFi blob). Reverting to the full filter reintroduces the boot-loop / crash.",
"ref": "https://github.com/ruvnet/RuView/issues/396"
},
{
"id": "RuView#521",
"title": "ESP32-S3 CSI: disable WiFi modem sleep (WIFI_PS_NONE) so the CSI callback isn't starved",
"files": ["firmware/esp32-csi-node/main/csi_collector.c"],
"require": ["esp_wifi_set_ps(WIFI_PS_NONE)", "RuView#521"],
"rationale": "The ESP-IDF STA default WIFI_PS_MIN_MODEM lets the modem sleep between DTIM beacons; combined with the MGMT-only filter the per-second CSI yield collapses toward 0 pps. csi_collector_init() must force WIFI_PS_NONE.",
"ref": "https://github.com/ruvnet/RuView/issues/521"
},
{
"id": "RuView#517",
"title": "Aggregator classifies sibling RuView UDP packet magics instead of erroring on them",
"files": [
"v2/crates/wifi-densepose-hardware/src/esp32_parser.rs",
"v2/crates/wifi-densepose-hardware/src/error.rs",
"v2/crates/wifi-densepose-hardware/src/bin/aggregator.rs"
],
"require": ["ruview_sibling_packet_name", "NonCsiPacket", "RUVIEW_VITALS_MAGIC"],
"rationale": "The firmware multiplexes 0xC5110002..0xC5110007 (vitals, feature, fused, compressed, feature-state, temporal) onto the CSI UDP port. The parser must report these as ParseError::NonCsiPacket so the aggregator can skip them, not log 'invalid magic' parse-error noise.",
"ref": "https://github.com/ruvnet/RuView/issues/517"
},
{
"id": "RuView#505",
"title": "Firmware release: version.txt must match the release tag (firmware-ci version-guard)",
"files": [".github/workflows/firmware-ci.yml"],
"require": ["version-guard", "version.txt"],
"rationale": "v0.6.3-esp32 shipped a binary that internally identified as 0.6.2 because version.txt was never bumped. The version-guard job fails the release run when the tag's X.Y.Z doesn't match firmware/esp32-csi-node/version.txt.",
"ref": "https://github.com/ruvnet/RuView/issues/505"
},
{
"id": "RuView#354",
"title": "Firmware embeds its version from version.txt and logs it at boot",
"files": [
"firmware/esp32-csi-node/CMakeLists.txt",
"firmware/esp32-csi-node/main/main.c"
],
"require": ["PROJECT_VER", "version.txt", "esp_app_get_description"],
"rationale": "esp_app_get_description()->version must derive from version.txt (CMake file(STRINGS ...)), and the boot log line surfaces it for fleet monitoring.",
"ref": "https://github.com/ruvnet/RuView/issues/354"
},
{
"id": "RuView#263",
"title": "Fall detection: default threshold 15.0 rad/s2 + consecutive-frame debounce + cooldown",
"files": [
"firmware/esp32-csi-node/main/nvs_config.c",
"firmware/esp32-csi-node/main/edge_processing.c",
"firmware/esp32-csi-node/main/edge_processing.h"
],
"require": ["15.0f", "EDGE_FALL_CONSEC_MIN", "EDGE_FALL_COOLDOWN_MS"],
"forbid": ["/fall_thresh\\s*=\\s*2\\.0f\\b/"],
"rationale": "Default fall_thresh of 2.0 rad/s2 caused alert storms (false positives). 15.0 with a 3-consecutive-frame debounce + 5 s cooldown verified 0 false alerts in 600 frames on COM7.",
"ref": "https://github.com/ruvnet/RuView/issues/263"
},
{
"id": "RuView#266-321",
"title": "Edge DSP task: batch limit so it can't starve IDLE1 and trip the task watchdog",
"files": ["firmware/esp32-csi-node/main/edge_processing.c", "firmware/esp32-csi-node/main/edge_processing.h"],
"require": ["EDGE_BATCH_LIMIT"],
"rationale": "On busy LANs the edge DSP task processed frames back-to-back with only 1-tick yields, starving IDLE1 enough to trip the 5-second task watchdog. The batch limit forces a longer yield every N frames.",
"ref": "https://github.com/ruvnet/RuView/issues/266"
},
{
"id": "RuView#265",
"title": "4 MB flash variant: dual-OTA partition table + 4mb sdkconfig, built by firmware-ci",
"files": [
"firmware/esp32-csi-node/partitions_4mb.csv",
"firmware/esp32-csi-node/sdkconfig.defaults.4mb",
".github/workflows/firmware-ci.yml"
],
"require": ["sdkconfig.defaults.4mb"],
"rationale": "Support for ESP32-S3-N16R8 / N8R2 and other 4 MB boards. The firmware-ci build matrix must keep building the 4mb variant so it doesn't bit-rot.",
"ref": "https://github.com/ruvnet/RuView/issues/265"
},
{
"id": "RuView#232-375-385-386-390",
"title": "ESP32-S3 CSI: defensive early-capture of NVS config before wifi_init_sta() corrupts it",
"files": ["firmware/esp32-csi-node/main/csi_collector.c"],
"require": ["early capture", "s_filter_mac"],
"rationale": "wifi_init_sta() can clobber g_nvs_config (confirmed on device 80:b5:4e:c1:be:b8). Module-local statics must be captured before WiFi init and used by the CSI callback instead of g_nvs_config.",
"ref": "https://github.com/ruvnet/RuView/issues/390"
},
{
"id": "ADR-028-proof",
"title": "Deterministic pipeline proof (Trust Kill Switch): artifacts present and re-run in CI",
"files": [
"archive/v1/data/proof/verify.py",
"archive/v1/data/proof/expected_features.sha256",
"archive/v1/data/proof/sample_csi_data.json",
".github/workflows/verify-pipeline.yml"
],
"require": ["VERDICT", "expected_features.sha256", "verify.py"],
"rationale": "verify.py feeds a seeded reference signal through the production CSI pipeline and SHA-256-hashes the output; expected_features.sha256 pins it; verify-pipeline.yml re-runs it on every PR. Losing any of these removes the project's tamper-evidence guarantee (ADR-028).",
"ref": "docs/adr/ADR-028-esp32-capability-audit.md"
},
{
"id": "ADR-028-witness-bundle",
"title": "Release-time witness bundle generator + self-verification script",
"files": ["scripts/generate-witness-bundle.sh"],
"require": ["VERIFY.sh", "witness-bundle"],
"rationale": "scripts/generate-witness-bundle.sh produces the self-contained, recipient-verifiable witness bundle (witness log + proof + test results + firmware hashes + VERIFY.sh). Part of the ADR-028 attestation chain.",
"ref": "docs/WITNESS-LOG-028.md"
}
]
}
+162
View File
@@ -0,0 +1,162 @@
#!/bin/bash
# ==============================================================================
# GCloud GPU driver for the MERIDIAN CSI masked-autoencoder pre-train (ADR-027 §2.0)
# ==============================================================================
#
# Creates a GCloud VM with a GPU, builds wifi-densepose-train with the
# `tch-backend` (+ `cuda`) feature, runs the `pretrain-mae` binary, downloads
# the pre-trained variable store (`.ot`), and tears the VM down.
#
# STATUS: prototype wiring stub (ADR-027 §2.0, iteration 3). The `pretrain-mae`
# binary currently drives the *deterministic SyntheticCsiDataset* — that's the
# end-to-end smoke path. The real heterogeneous-CSI pre-train (MM-Fi + Wi-Pose +
# data/recordings/ + multi-band virtual sub-carriers) needs the ingest pipeline
# tracked in ADR-027 §2.0 "Iteration 3 plan"; the TODO markers below show where
# it plugs in. This script is intentionally a thin, reviewable shell of the real
# gcloud-train.sh (which it mirrors) — it has NOT been run.
#
# Usage:
# bash scripts/pretrain-mae-gcloud.sh [OPTIONS]
#
# Options:
# --gpu l4|a100|h100 GPU type (default: l4)
# --zone ZONE GCloud zone (default: us-central1-a)
# --hours N Max VM lifetime in hours (default: 3)
# --epochs N Pre-train epochs (default: 20)
# --samples N Synthetic samples (until the real ingest lands) (default: 4096)
# --batch N Mini-batch size (default: 64)
# --mask-ratio R Token mask ratio (default: 0.75)
# --lr R Adam learning rate (default: 1e-3)
# --out FILE Local path for the downloaded .ot (default: data/models/mae-pretrained.ot)
# --data-dir DIR (future) heterogeneous CSI corpus to upload — see TODO below
# --dry-run Build + run a tiny pre-train locally with synthetic data; no VM
# --keep-vm Do not delete the VM after the run
# --instance NAME Custom VM instance name
#
# Prerequisites (same as gcloud-train.sh):
# - gcloud CLI authenticated: gcloud auth login
# - Project set: gcloud config set project cognitum-20260110
# - GPU quota in the chosen zone
#
# Cost (same envelope as gcloud-train.sh):
# L4 ~$0.80/hr (prototyping) · A100 40GB ~$3.60/hr (full pre-train) · H100 80GB ~$11/hr
# ==============================================================================
set -euo pipefail
# ── Defaults ──────────────────────────────────────────────────────────────────
PROJECT="cognitum-20260110"
GPU_TYPE="l4"
ZONE="us-central1-a"
HOURS=3
EPOCHS=20
SAMPLES=4096
BATCH=64
MASK_RATIO=0.75
LR="1e-3"
OUT="data/models/mae-pretrained.ot"
DATA_DIR=""
DRY_RUN=0
KEEP_VM=0
INSTANCE="meridian-mae-$(date +%s)"
# ── Arg parse ─────────────────────────────────────────────────────────────────
while [[ $# -gt 0 ]]; do
case "$1" in
--gpu) GPU_TYPE="$2"; shift 2;;
--zone) ZONE="$2"; shift 2;;
--hours) HOURS="$2"; shift 2;;
--epochs) EPOCHS="$2"; shift 2;;
--samples) SAMPLES="$2"; shift 2;;
--batch) BATCH="$2"; shift 2;;
--mask-ratio) MASK_RATIO="$2"; shift 2;;
--lr) LR="$2"; shift 2;;
--out) OUT="$2"; shift 2;;
--data-dir) DATA_DIR="$2"; shift 2;;
--dry-run) DRY_RUN=1; shift;;
--keep-vm) KEEP_VM=1; shift;;
--instance) INSTANCE="$2"; shift 2;;
-h|--help) sed -n '2,46p' "$0"; exit 0;;
*) echo "unknown option: $1" >&2; exit 2;;
esac
done
case "$GPU_TYPE" in
l4) ACCEL="type=nvidia-l4,count=1"; MACHINE="g2-standard-8";;
a100) ACCEL="type=nvidia-tesla-a100,count=1"; MACHINE="a2-highgpu-1g";;
h100) ACCEL="type=nvidia-h100-80gb,count=1"; MACHINE="a3-highgpu-1g";;
*) echo "unknown --gpu: $GPU_TYPE (l4|a100|h100)" >&2; exit 2;;
esac
PRETRAIN_ARGS="--epochs $EPOCHS --samples $SAMPLES --batch $BATCH --mask-ratio $MASK_RATIO --lr $LR --save mae-pretrained.ot"
# ── Dry run: build + tiny pre-train locally (synthetic data), no VM ───────────
if [[ "$DRY_RUN" -eq 1 ]]; then
echo "[dry-run] cargo run -p wifi-densepose-train --features tch-backend --bin pretrain-mae -- --epochs 2 --samples 64 --batch 8"
echo "[dry-run] (requires LibTorch — set LIBTORCH or use a tch download-libtorch feature build)"
cd "$(dirname "$0")/../v2"
cargo run -p wifi-densepose-train --features tch-backend --bin pretrain-mae -- --epochs 2 --samples 64 --batch 8
exit 0
fi
# ── Provision VM ──────────────────────────────────────────────────────────────
echo "==> Project: $PROJECT Zone: $ZONE GPU: $GPU_TYPE Machine: $MACHINE Instance: $INSTANCE"
gcloud config set project "$PROJECT" >/dev/null
gcloud compute instances create "$INSTANCE" \
--zone="$ZONE" --machine-type="$MACHINE" \
--accelerator="$ACCEL" --maintenance-policy=TERMINATE \
--image-family=pytorch-latest-gpu --image-project=deeplearning-platform-release \
--boot-disk-size=128GB --metadata="install-nvidia-driver=True" \
--max-run-duration="${HOURS}h" --instance-termination-action=DELETE
cleanup() {
if [[ "$KEEP_VM" -eq 0 ]]; then
echo "==> Deleting VM $INSTANCE"
gcloud compute instances delete "$INSTANCE" --zone="$ZONE" --quiet || true
else
echo "==> --keep-vm set; VM $INSTANCE left running (remember to delete it)."
fi
}
trap cleanup EXIT
run_remote() { gcloud compute ssh "$INSTANCE" --zone="$ZONE" --command="$1"; }
echo "==> Waiting for SSH..."
for _ in $(seq 1 30); do run_remote "true" 2>/dev/null && break; sleep 10; done
echo "==> Provisioning toolchain on the VM"
run_remote 'set -e
curl --proto "=https" --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
source "$HOME/.cargo/env"
# The pytorch-latest-gpu image ships libtorch; point tch at it.
TORCH_DIR="$(python -c "import torch,os;print(os.path.dirname(torch.__file__))")"
echo "export LIBTORCH=$TORCH_DIR" >> "$HOME/.bashrc"
echo "export LD_LIBRARY_PATH=$TORCH_DIR/lib:\$LD_LIBRARY_PATH" >> "$HOME/.bashrc"
sudo apt-get update -qq && sudo apt-get install -y -qq git build-essential pkg-config'
echo "==> Uploading repo"
# rsync the repo (excluding build artifacts) — same approach as gcloud-train.sh.
gcloud compute scp --recurse --zone="$ZONE" \
../v2 ../scripts ../docs "$INSTANCE":~/ruview/ >/dev/null
# TODO (ADR-027 §2.0, iter 3 ingest): when --data-dir is given, upload the
# heterogeneous CSI corpus and point pretrain-mae at it instead of the synthetic
# dataset (needs a `--data-dir`/`--datasets` flag on the bin first — see the plan).
if [[ -n "$DATA_DIR" ]]; then
echo "==> Uploading CSI corpus from $DATA_DIR"
gcloud compute scp --recurse --zone="$ZONE" "$DATA_DIR" "$INSTANCE":~/ruview/csi-corpus/ >/dev/null
PRETRAIN_ARGS="$PRETRAIN_ARGS # TODO: --data-dir ~/ruview/csi-corpus"
fi
echo "==> Building + running pre-train on the VM"
run_remote "set -e; source \$HOME/.cargo/env; source \$HOME/.bashrc
cd ~/ruview/v2
cargo build --release -p wifi-densepose-train --features tch-backend,cuda
cargo run --release -p wifi-densepose-train --features tch-backend,cuda --bin pretrain-mae -- $PRETRAIN_ARGS"
echo "==> Downloading pre-trained variable store → $OUT"
mkdir -p "$(dirname "$OUT")"
gcloud compute scp --zone="$ZONE" "$INSTANCE":~/ruview/v2/mae-pretrained.ot "$OUT"
echo "==> Done. Pre-trained encoder: $OUT"
echo " Next: fine-tune the ADR-027 §2.x heads on top of it (see §2.0 'Iteration 3 plan')."
+10 -46
View File
@@ -5127,9 +5127,9 @@
"license": "BSD-3-Clause"
},
"node_modules/@xmldom/xmldom": {
"version": "0.8.11",
"resolved": "https://registry.npmjs.org/@xmldom/xmldom/-/xmldom-0.8.11.tgz",
"integrity": "sha512-cQzWCtO6C8TQiYl1ruKNn2U6Ao4o4WBBcbL61yJl84x+j5sOWWFU9X7DpND8XZG3daDppSsigMdfAIl2upQBRw==",
"version": "0.8.13",
"resolved": "https://registry.npmjs.org/@xmldom/xmldom/-/xmldom-0.8.13.tgz",
"integrity": "sha512-KRYzxepc14G/CEpEGc3Yn+JKaAeT63smlDr+vjB8jRfgTBBI9wRj/nkQEO+ucV8p8I9bfKLWp37uHgFrbntPvw==",
"license": "MIT",
"engines": {
"node": ">=10.0.0"
@@ -5310,18 +5310,6 @@
"node": ">= 8"
}
},
"node_modules/anymatch/node_modules/picomatch": {
"version": "2.3.1",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.1.tgz",
"integrity": "sha512-JU3teHTNjmE2VCGFzuY8EXzCDVwEqB2a8fsIvwaStHhAWJEeVd1o1QD80CU6+ZdEXXSLbSsuLwJjkCBWqRQUVA==",
"license": "MIT",
"engines": {
"node": ">=8.6"
},
"funding": {
"url": "https://github.com/sponsors/jonschlinkert"
}
},
"node_modules/arg": {
"version": "5.0.2",
"resolved": "https://registry.npmjs.org/arg/-/arg-5.0.2.tgz",
@@ -11935,18 +11923,6 @@
"node": ">=8"
}
},
"node_modules/jest-util/node_modules/picomatch": {
"version": "2.3.1",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.1.tgz",
"integrity": "sha512-JU3teHTNjmE2VCGFzuY8EXzCDVwEqB2a8fsIvwaStHhAWJEeVd1o1QD80CU6+ZdEXXSLbSsuLwJjkCBWqRQUVA==",
"license": "MIT",
"engines": {
"node": ">=8.6"
},
"funding": {
"url": "https://github.com/sponsors/jonschlinkert"
}
},
"node_modules/jest-validate": {
"version": "29.7.0",
"resolved": "https://registry.npmjs.org/jest-validate/-/jest-validate-29.7.0.tgz",
@@ -13389,18 +13365,6 @@
"node": ">=8.6"
}
},
"node_modules/micromatch/node_modules/picomatch": {
"version": "2.3.1",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.1.tgz",
"integrity": "sha512-JU3teHTNjmE2VCGFzuY8EXzCDVwEqB2a8fsIvwaStHhAWJEeVd1o1QD80CU6+ZdEXXSLbSsuLwJjkCBWqRQUVA==",
"license": "MIT",
"engines": {
"node": ">=8.6"
},
"funding": {
"url": "https://github.com/sponsors/jonschlinkert"
}
},
"node_modules/mime": {
"version": "1.6.0",
"resolved": "https://registry.npmjs.org/mime/-/mime-1.6.0.tgz",
@@ -13594,9 +13558,9 @@
}
},
"node_modules/node-forge": {
"version": "1.3.3",
"resolved": "https://registry.npmjs.org/node-forge/-/node-forge-1.3.3.tgz",
"integrity": "sha512-rLvcdSyRCyouf6jcOIPe/BgwG/d7hKjzMKOas33/pHEr6gbq18IK9zV7DiPvzsz0oBJPme6qr6H6kGZuI9/DZg==",
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/node-forge/-/node-forge-1.4.0.tgz",
"integrity": "sha512-LarFH0+6VfriEhqMMcLX2F7SwSXeWwnEAJEsYm5QKWchiVYVvJyV9v7UDvUv+w5HO23ZpQTXDv/GxdDdMyOuoQ==",
"license": "(BSD-3-Clause OR GPL-2.0)",
"engines": {
"node": ">= 6.13.0"
@@ -14056,12 +14020,12 @@
"license": "ISC"
},
"node_modules/picomatch": {
"version": "4.0.3",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.3.tgz",
"integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==",
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.2.tgz",
"integrity": "sha512-V7+vQEJ06Z+c5tSye8S+nHUfI51xoXIXjHQ99cQtKUkQqqO1kO/KCJUfZXuB47h/YBlDhah2H3hdUGXn8ie0oA==",
"license": "MIT",
"engines": {
"node": ">=12"
"node": ">=8.6"
},
"funding": {
"url": "https://github.com/sponsors/jonschlinkert"
+5
View File
@@ -49,5 +49,10 @@
"react-native-worklets": "^0.7.4",
"typescript": "~5.9.2"
},
"overrides": {
"@xmldom/xmldom": "0.8.13",
"node-forge": "^1.4.0",
"picomatch": "^2.3.2"
},
"private": true
}
@@ -10,7 +10,7 @@ use std::net::UdpSocket;
use std::process;
use clap::Parser;
use wifi_densepose_hardware::Esp32CsiParser;
use wifi_densepose_hardware::{Esp32CsiParser, ParseError};
/// UDP aggregator for ESP32 CSI nodes (ADR-018).
#[derive(Parser)]
@@ -65,6 +65,15 @@ fn main() {
mean_amp,
);
}
// The firmware sends several packet types on this UDP port
// (ADR-039 vitals, ADR-081 feature state, ADR-095 temporal, …)
// alongside ADR-018 CSI frames. Those are expected, not errors —
// this CSI-only aggregator just skips them. (RuView#517)
Err(ParseError::NonCsiPacket { kind, .. }) => {
if cli.verbose {
eprintln!(" [skipped {} packet — not a CSI frame]", kind);
}
}
Err(e) => {
if cli.verbose {
eprintln!(" parse error: {}", e);
@@ -19,6 +19,18 @@ pub enum ParseError {
got: u32,
},
/// A recognized RuView wire packet was received that is *not* an
/// ADR-018 raw CSI frame (e.g. ADR-039 vitals, ADR-081 feature state,
/// ADR-095 temporal classification). The firmware multiplexes several
/// packet types onto the same UDP port, so a CSI parser will see these
/// interleaved with CSI frames — that is expected, not a corruption.
/// Consumers should route the packet to the matching decoder or skip it.
#[error("Non-CSI RuView packet on CSI socket: {kind} (magic {magic:#010x})")]
NonCsiPacket {
magic: u32,
kind: &'static str,
},
/// The frame indicates more subcarriers than physically possible.
#[error("Invalid subcarrier count: {count} (max {max})")]
InvalidSubcarrierCount {
@@ -35,7 +35,43 @@ use crate::csi_frame::{AntennaConfig, Bandwidth, CsiFrame, CsiMetadata, Subcarri
use crate::error::ParseError;
/// ESP32 CSI binary frame magic number (ADR-018).
const ESP32_CSI_MAGIC: u32 = 0xC5110001;
pub const ESP32_CSI_MAGIC: u32 = 0xC5110001;
// ── Sibling RuView wire packets ──────────────────────────────────────────────
// The ESP32 firmware multiplexes several packet types onto the same UDP port
// as ADR-018 raw CSI frames. A CSI-only consumer will therefore see these
// interleaved with CSI frames. They are *not* corruption — they just need a
// different decoder (or can be skipped). See firmware `rv_feature_state.h`.
/// ADR-039 edge vitals packet (32 bytes: HR/BR/presence).
pub const RUVIEW_VITALS_MAGIC: u32 = 0xC5110002;
/// ADR-069 feature-vector packet.
pub const RUVIEW_FEATURE_MAGIC: u32 = 0xC5110003;
/// ADR-063 fused-vitals packet (multi-sensor fusion).
pub const RUVIEW_FUSED_VITALS_MAGIC: u32 = 0xC5110004;
/// ADR-039 compressed-CSI packet.
pub const RUVIEW_COMPRESSED_CSI_MAGIC: u32 = 0xC5110005;
/// ADR-081 compact feature-state packet (the default upstream payload).
pub const RUVIEW_FEATURE_STATE_MAGIC: u32 = 0xC5110006;
/// ADR-095 / #513 on-device temporal-classification packet.
pub const RUVIEW_TEMPORAL_MAGIC: u32 = 0xC5110007;
/// If `magic` is a recognized RuView wire packet other than the ADR-018 raw
/// CSI frame, return a human-readable name for it; otherwise `None`.
///
/// Used by CSI consumers to distinguish "a sibling packet I should route or
/// skip" from "genuine garbage on the wire".
pub fn ruview_sibling_packet_name(magic: u32) -> Option<&'static str> {
match magic {
RUVIEW_VITALS_MAGIC => Some("ADR-039 edge vitals"),
RUVIEW_FEATURE_MAGIC => Some("ADR-069 feature vector"),
RUVIEW_FUSED_VITALS_MAGIC => Some("ADR-063 fused vitals"),
RUVIEW_COMPRESSED_CSI_MAGIC => Some("ADR-039 compressed CSI"),
RUVIEW_FEATURE_STATE_MAGIC => Some("ADR-081 feature state"),
RUVIEW_TEMPORAL_MAGIC => Some("ADR-095 temporal classification"),
_ => None,
}
}
/// ADR-018 header size in bytes (before I/Q data).
const HEADER_SIZE: usize = 20;
@@ -55,6 +91,18 @@ impl Esp32CsiParser {
/// The buffer must contain at least the header (20 bytes) plus the I/Q data.
/// Returns the parsed frame and the number of bytes consumed.
pub fn parse_frame(data: &[u8]) -> Result<(CsiFrame, usize), ParseError> {
// A recognized sibling packet (ADR-039 vitals, ADR-081 feature state, …)
// multiplexed onto the CSI UDP port should be reported as such — not as
// "insufficient data" or "invalid magic" — so callers can route or skip
// it. These packets are all >= 4 bytes; classify before the CSI-frame
// length gate. (RuView#517)
if data.len() >= 4 {
let magic = u32::from_le_bytes([data[0], data[1], data[2], data[3]]);
if let Some(kind) = ruview_sibling_packet_name(magic) {
return Err(ParseError::NonCsiPacket { magic, kind });
}
}
if data.len() < HEADER_SIZE {
return Err(ParseError::InsufficientData {
needed: HEADER_SIZE,
@@ -310,12 +358,50 @@ mod tests {
#[test]
fn test_parse_invalid_magic() {
let mut data = build_test_frame(1, 1, &[(10, 20)]);
// Corrupt magic
data[0] = 0xFF;
// Corrupt magic to a value that isn't any known RuView packet.
data[0..4].copy_from_slice(&0xDEAD_BEEFu32.to_le_bytes());
let result = Esp32CsiParser::parse_frame(&data);
assert!(matches!(result, Err(ParseError::InvalidMagic { .. })));
}
#[test]
fn test_sibling_vitals_packet_is_not_invalid_magic() {
// RuView#517: a 32-byte ADR-039 vitals packet (magic 0xC5110002)
// arrives on the same UDP port as CSI frames. It must be reported as
// a recognized sibling packet, not a corrupt CSI frame.
let mut data = vec![0u8; 32];
data[0..4].copy_from_slice(&RUVIEW_VITALS_MAGIC.to_le_bytes());
match Esp32CsiParser::parse_frame(&data) {
Err(ParseError::NonCsiPacket { magic, kind }) => {
assert_eq!(magic, RUVIEW_VITALS_MAGIC);
assert_eq!(kind, "ADR-039 edge vitals");
}
other => panic!("expected NonCsiPacket, got {other:?}"),
}
}
#[test]
fn test_all_sibling_magics_classified() {
for m in [
RUVIEW_VITALS_MAGIC,
RUVIEW_FEATURE_MAGIC,
RUVIEW_FUSED_VITALS_MAGIC,
RUVIEW_COMPRESSED_CSI_MAGIC,
RUVIEW_FEATURE_STATE_MAGIC,
RUVIEW_TEMPORAL_MAGIC,
] {
assert!(ruview_sibling_packet_name(m).is_some(), "{m:#010x} unclassified");
let mut data = vec![0u8; 24];
data[0..4].copy_from_slice(&m.to_le_bytes());
assert!(
matches!(Esp32CsiParser::parse_frame(&data), Err(ParseError::NonCsiPacket { .. })),
"{m:#010x} should parse as NonCsiPacket"
);
}
// The CSI magic itself is not a "sibling".
assert!(ruview_sibling_packet_name(ESP32_CSI_MAGIC).is_none());
}
#[test]
fn test_amplitude_phase_from_known_iq() {
let pairs = vec![(100i8, 0i8), (0, 50), (30, 40)];
+5 -1
View File
@@ -49,7 +49,11 @@ pub mod radio_ops;
pub use csi_frame::{CsiFrame, CsiMetadata, SubcarrierData, Bandwidth, AntennaConfig};
pub use error::ParseError;
pub use esp32_parser::Esp32CsiParser;
pub use esp32_parser::{
Esp32CsiParser, ruview_sibling_packet_name, ESP32_CSI_MAGIC, RUVIEW_VITALS_MAGIC,
RUVIEW_FEATURE_MAGIC, RUVIEW_FUSED_VITALS_MAGIC, RUVIEW_COMPRESSED_CSI_MAGIC,
RUVIEW_FEATURE_STATE_MAGIC, RUVIEW_TEMPORAL_MAGIC,
};
pub use bridge::CsiData;
pub use radio_ops::{
RadioOps, RadioMode, CaptureProfile, RadioHealth, RadioError, MockRadio,
@@ -14,6 +14,7 @@ serde_json = { workspace = true }
tokio = { workspace = true }
anyhow = { workspace = true }
axum = { workspace = true }
tower-http = { workspace = true }
clap = { version = "4", features = ["derive"] }
chrono = "0.4"
dirs = "5"
@@ -8,11 +8,13 @@ use crate::fusion;
use crate::pointcloud;
use axum::{
extract::State,
http::{HeaderValue, Method},
response::Html,
routing::get,
Json, Router,
};
use std::sync::{Arc, Mutex};
use tower_http::cors::{AllowOrigin, CorsLayer};
struct AppState {
latest_cloud: Mutex<pointcloud::PointCloud>,
@@ -108,12 +110,36 @@ pub async fn serve(bind: &str, _brain: Option<&str>) -> anyhow::Result<()> {
if has_camera { eprintln!(" Camera: LIVE (/dev/video0)"); }
else { eprintln!(" Camera: DEMO"); }
// CORS — allow the hosted GitHub Pages viewer to fetch /api/splats from a
// locally-running instance of this server. Modern browsers treat
// 127.0.0.1/localhost as a "potentially trustworthy" origin so the HTTPS
// page can reach a plain-HTTP loopback backend without mixed-content
// blocking. Origins permitted:
// - https://ruvnet.github.io (the published RuView Pages demo)
// - http://localhost:* / http://127.0.0.1:* (developer running the
// viewer.html bundled with this binary)
// Anything else is denied, so this is not a "wildcard" CORS.
let cors = CorsLayer::new()
.allow_origin(AllowOrigin::predicate(|origin: &HeaderValue, _req| {
let s = match origin.to_str() {
Ok(v) => v,
Err(_) => return false,
};
s == "https://ruvnet.github.io"
|| s.starts_with("http://localhost")
|| s.starts_with("http://127.0.0.1")
|| s == "null" // file:// origins
}))
.allow_methods([Method::GET, Method::OPTIONS])
.allow_headers([axum::http::header::CONTENT_TYPE]);
let app = Router::new()
.route("/", get(index))
.route("/api/cloud", get(api_cloud))
.route("/api/splats", get(api_splats))
.route("/api/status", get(api_status))
.route("/health", get(api_health))
.layer(cors)
.with_state(state);
println!("╔══════════════════════════════════════════════╗");
@@ -2,27 +2,45 @@
<html>
<head>
<title>RuView — Camera + WiFi CSI Point Cloud</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate">
<meta name="ruview-viewer-version" content="0.2.0-face-mesh">
<!-- Inline amber-dot favicon avoids a stray /favicon.ico 404 in the console. -->
<link rel="icon" type="image/svg+xml" href="data:image/svg+xml;utf8,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><circle cx='16' cy='16' r='10' fill='%23e8a634'/></svg>">
<style>
body { margin: 0; background: #0a0a0a; color: #e8a634; font-family: monospace; }
canvas { display: block; }
#info { position: absolute; top: 10px; left: 10px; padding: 12px; background: rgba(0,0,0,0.85); border: 1px solid #e8a634; border-radius: 6px; min-width: 240px; font-size: 13px; line-height: 1.5; }
#info { position: absolute; top: 10px; left: 10px; padding: 12px; background: rgba(0,0,0,0.85); border: 1px solid #e8a634; border-radius: 6px; min-width: 240px; font-size: 13px; line-height: 1.5; z-index: 10; }
#cam-cta { position: absolute; bottom: 16px; left: 50%; transform: translateX(-50%); padding: 10px 18px; background: #e8a634; color: #0a0a0a; border: none; border-radius: 4px; font-family: monospace; font-size: 14px; font-weight: bold; cursor: pointer; z-index: 10; }
#cam-cta:hover { background: #ffc04d; }
#cam-cta.hidden { display: none; }
#esp-cta { position: absolute; bottom: 16px; right: 16px; padding: 8px 14px; background: transparent; color: #e8a634; border: 1px solid #e8a634; border-radius: 4px; font-family: monospace; font-size: 12px; cursor: pointer; z-index: 10; }
#esp-cta:hover { background: rgba(232, 166, 52, 0.12); }
#esp-cta.connected { background: #4f4; color: #0a0a0a; border-color: #4f4; }
.live { color: #4f4; } .demo { color: #f44; }
.face { color: #4cf; }
.section { margin-top: 6px; padding-top: 6px; border-top: 1px solid #333; }
.label { color: #888; }
</style>
<script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/r128/three.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/three@0.128.0/examples/js/controls/OrbitControls.js"></script>
<!-- MediaPipe Face Mesh — runs in demo mode so each visitor sees their own face as a point cloud -->
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh@0.4/face_mesh.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/camera_utils@0.3/camera_utils.js"></script>
</head>
<body>
<div id="info">
<h3 style="margin:0 0 8px 0">RuView Point Cloud</h3>
<h3 style="margin:0 0 4px 0">RuView · Seldon Vault</h3>
<div style="font-size: 11px; color: #888; margin-bottom: 8px; max-width: 240px; line-height: 1.4; font-style: italic;">"Psychohistory deals with reactions of human conglomerates to fixed social and economic stimuli." — Hari Seldon</div>
<div id="stats">Loading...</div>
</div>
<button id="cam-cta">▶ Project Subject — render your face into the Vault</button>
<button id="esp-cta" title="Stream live CSI from a local ruview-pointcloud serve instance (e.g. http://127.0.0.1:9880)">📡 Connect ESP32…</button>
<script>
var scene = new THREE.Scene();
scene.background = new THREE.Color(0x0a0a0a);
var camera = new THREE.PerspectiveCamera(75, window.innerWidth/window.innerHeight, 0.1, 100);
camera.position.set(0, 2, -4);
var camera = new THREE.PerspectiveCamera(72, window.innerWidth/window.innerHeight, 0.1, 200);
camera.position.set(0, 0.2, -3.5);
camera.lookAt(0, 0, 2);
var renderer = new THREE.WebGLRenderer({ antialias: true });
@@ -104,10 +122,436 @@
scene.add(skeletonGroup);
}
async function fetchCloud() {
// ----- Transport configuration -----
// ?backend=<url> → fetch splats from <url>/api/splats (CORS-permitting host)
// ?backend=auto → try /api/splats, fall back to synthetic demo on failure (default)
// ?backend=demo → always render synthetic demo (no network)
// ?live=1 → require live; show error instead of demo fallback
var urlParams = new URLSearchParams(window.location.search);
var backendArg = urlParams.get("backend") || "auto";
var requireLive = urlParams.get("live") === "1";
var transportMode = "demo"; // resolved at first fetch: "live" | "remote" | "demo"
var demoStartMs = Date.now();
var demoFrameNum = 0;
var latestFaceLandmarks = null; // populated by MediaPipe when camera enabled
var faceMeshState = "idle"; // "idle" | "starting" | "running" | "denied" | "unavailable"
// ----- MediaPipe Face Mesh (browser equivalent of camera-depth backprojection) -----
// Locally, ruview-pointcloud serve fuses real camera depth + WiFi CSI. In the
// browser we don't have depth from a webcam, but Face Mesh produces 468
// 3D landmarks (x,y in [0,1], z roughly in [-0.5,0.5]) at ~30 fps — enough to
// reproduce the "I can see the outline of my face in points" experience. The
// landmarks feed into the same splat render path as live /api/splats data.
async function startFaceMesh() {
if (faceMeshState !== "idle") return;
if (!window.FaceMesh || !window.Camera) {
faceMeshState = "unavailable";
return;
}
faceMeshState = "starting";
try {
var resp = await fetch("/api/splats");
var videoEl = document.createElement("video");
videoEl.style.display = "none";
videoEl.autoplay = true;
videoEl.playsInline = true;
videoEl.muted = true;
document.body.appendChild(videoEl);
var fm = new FaceMesh({
locateFile: function(file) {
return "https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh@0.4/" + file;
}
});
fm.setOptions({
maxNumFaces: 1,
refineLandmarks: true,
minDetectionConfidence: 0.5,
minTrackingConfidence: 0.5
});
fm.onResults(function(results) {
if (results.multiFaceLandmarks && results.multiFaceLandmarks[0]) {
latestFaceLandmarks = results.multiFaceLandmarks[0];
}
});
var mpCamera = new Camera(videoEl, {
onFrame: async function() { await fm.send({ image: videoEl }); },
width: 640,
height: 480
});
await mpCamera.start();
faceMeshState = "running";
var btn = document.getElementById("cam-cta");
if (btn) btn.classList.add("hidden");
} catch (err) {
faceMeshState = "denied";
console.warn("Face mesh unavailable:", err);
}
}
// ---- Foundation-inspired galactic context (Asimov / Trantor / Seldon) ----
// Shared between face-mesh and synthetic-fallback paths. The subject (face
// or procedural figure) is the foreground; this function paints the Seldon
// time-vault around it: holographic surveyor grid underfoot, slow galactic
// spiral receding into the distance, distant starfield, and a halo ring.
function pushFoundationContext(splats) {
var t = (Date.now() - demoStartMs) / 1000.0;
// 1. Holographic surveyor grid — amber lattice at y=+1.4 (renders below
// the subject because the renderer flips y to Three.js Y-up).
var gx, gz;
for (gx = -10; gx <= 10; gx++) {
for (gz = 0; gz <= 30; gz++) {
var alpha = 0.35 + 0.15 * Math.sin(t * 0.5 + gz * 0.2);
splats.push({
center: [gx * 0.5, 1.4, gz * 0.4],
color: [0.40 * alpha, 0.28 * alpha, 0.10 * alpha],
opacity: 1.0,
scale: [0.018, 0.018, 0.018]
});
}
}
for (gz = 0; gz <= 30; gz += 2) {
for (gx = -20; gx <= 20; gx++) {
splats.push({
center: [gx * 0.25, 1.4, gz * 0.4 + 0.1],
color: [0.30, 0.22, 0.08],
opacity: 1.0,
scale: [0.014, 0.014, 0.014]
});
}
}
// 2. Galactic spiral — Trantor recedes behind the subject. ~640 stars
// across two logarithmic arms, slowly rotating. Warmer at the core,
// cooler at the edges (Hertzsprung-Russell-ish).
var arm, k, theta_arm, r_arm, sx, sz, twist, arm_color;
for (arm = 0; arm < 2; arm++) {
for (k = 0; k < 320; k++) {
var prog = k / 320;
theta_arm = arm * Math.PI + prog * 6.0 + t * 0.05;
r_arm = 4.0 + prog * 14.0;
twist = Math.sin(prog * 8.0) * 0.4;
sx = Math.cos(theta_arm) * r_arm + twist;
sz = Math.sin(theta_arm) * r_arm + 12.0;
var coreFade = Math.max(0.15, 1.0 - prog);
arm_color = [
coreFade * 0.85 + 0.15 * (1 - prog),
coreFade * 0.70 + 0.20,
coreFade * 0.55 + 0.45 * prog
];
splats.push({
center: [sx, -2.5 + Math.sin(prog * 12) * 0.3, sz],
color: arm_color,
opacity: 1.0,
scale: [0.025, 0.025, 0.025]
});
}
}
// 3. Distant starfield — 800 deterministic stars on a spherical shell.
// Fixed LCG seed so visitors don't see noise flicker between frames.
var seed = 42;
function nextRand() {
seed = (seed * 1664525 + 1013904223) >>> 0;
return seed / 4294967296;
}
var s, r_s, phi, costheta, sinphi;
for (s = 0; s < 800; s++) {
phi = nextRand() * Math.PI * 2;
costheta = nextRand() * 2 - 1;
sinphi = Math.sqrt(1 - costheta * costheta);
r_s = 22 + nextRand() * 8;
var brightness = 0.4 + nextRand() * 0.6;
var hue = nextRand();
splats.push({
center: [
Math.cos(phi) * sinphi * r_s,
costheta * r_s * 0.5 - 1.0,
Math.sin(phi) * sinphi * r_s + 5.0
],
color: hue > 0.85
? [brightness, brightness * 0.85, brightness * 0.6]
: (hue > 0.3
? [brightness * 0.9, brightness * 0.95, brightness]
: [brightness * 0.5, brightness * 0.7, brightness]),
opacity: 1.0,
scale: [0.020, 0.020, 0.020]
});
}
// 4. Holographic projection halo around the subject — Seldon vault
// projections always had a faint encircling ring of particles.
var ring;
for (ring = 0; ring < 60; ring++) {
var rt = ring / 60 * Math.PI * 2 + t * 0.3;
splats.push({
center: [
Math.cos(rt) * 1.6,
Math.sin(rt) * 1.2 - 0.2,
2.0 + Math.sin(rt * 3 + t * 0.5) * 0.3
],
color: [0.95, 0.55, 0.15],
opacity: 1.0,
scale: [0.014, 0.014, 0.014]
});
}
}
// Map a single landmark to world coords. Coord conventions:
// x: 0.5 - lm.x → mirror so left-of-screen = your left side (selfie)
// y: lm.y - 0.5 → keep MediaPipe's y-DOWN convention; the renderer's
// existing -y flip in updateSplats does the single flip
// to Three.js Y-up. Pre-flipping here would double-flip
// and the face would render upside down.
// z: 2.0 + lm.z*8 → amplify lm.z (~[-0.1,+0.1]) so the nose/eye-socket
// depth is visible from an oblique camera angle.
function lmToCenter(lm) {
return [
(0.5 - lm.x) * 4.0,
(lm.y - 0.5) * 3.0,
2.0 + lm.z * 8.0
];
}
function pushFaceSplat(splats, center, alpha) {
splats.push({
center: center,
color: [0.95 * alpha, 0.65 * alpha, 0.20 * alpha],
opacity: 1.0,
scale: [0.006, 0.006, 0.006]
});
}
// FACEMESH_TESSELATION is a flat array [a0,b0, a1,b1, ...] of vertex indices
// forming edges of the triangulated face mesh. ~1300 edges × 2 = 2600 entries.
// We interpolate 6 splats per edge → ~8000 splats per face vs 478 vertices.
var FACE_EDGES = (typeof FACEMESH_TESSELATION !== "undefined") ? FACEMESH_TESSELATION : null;
// Push the user's face mesh point cloud into `splats` (no Foundation
// context — that is the demo path's responsibility). Used both as the
// demo subject AND as an overlay on top of live/remote backend data
// when the camera is enabled. Returns true if any splats were pushed.
function pushFaceSplats(splats) {
if (faceMeshState !== "running" || !latestFaceLandmarks) return false;
var lms = latestFaceLandmarks;
var i;
// 1. Original 478 vertices — bright anchor points for features.
for (i = 0; i < lms.length; i++) {
splats.push({
center: lmToCenter(lms[i]),
color: [1.0, 0.72, 0.25],
opacity: 1.0,
scale: [0.010, 0.010, 0.010]
});
}
// 2. Edge interpolation — 6 splats per FACEMESH_TESSELATION edge.
if (FACE_EDGES) {
var edgeCount = FACE_EDGES.length;
var SAMPLES = 6;
var e, a, b, t, f;
for (e = 0; e < edgeCount; e += 2) {
a = lms[FACE_EDGES[e]];
b = lms[FACE_EDGES[e + 1]];
if (!a || !b) continue;
var aPos = lmToCenter(a);
var bPos = lmToCenter(b);
var ax = aPos[0], ay = aPos[1], az = aPos[2];
var bx = bPos[0], by = bPos[1], bz = bPos[2];
for (t = 1; t <= SAMPLES; t++) {
f = t / (SAMPLES + 1);
pushFaceSplat(splats, [
ax * (1 - f) + bx * f,
ay * (1 - f) + by * f,
az * (1 - f) + bz * f
], 0.85);
}
}
}
return true;
}
function faceMeshFrame() {
if (faceMeshState !== "running" || !latestFaceLandmarks) return null;
var splats = [];
pushFaceSplats(splats);
pushFoundationContext(splats);
demoFrameNum += 1;
return {
splats: splats,
count: splats.length,
frame: demoFrameNum,
live: false,
source: "face-mesh",
pipeline: {
skeleton: null,
vitals: { breathing_rate: 14, motion_score: 0.15 }
}
};
}
function buildSplatsUrl() {
if (backendArg === "demo") return null;
if (backendArg === "auto") return "/api/splats";
// User-supplied URL — strip trailing slash and append /api/splats.
var base = backendArg.replace(/\/+$/, "");
return base + "/api/splats";
}
function syntheticFrame() {
// Used when camera permission is denied / unavailable. Renders a
// procedural standing figure inside the Seldon vault context.
// y-down convention: head at small/negative y, feet at large/positive y;
// the renderer flips y so head ends up at the top of the screen.
var t = (Date.now() - demoStartMs) / 1000.0;
var sway = Math.sin(t * 0.8) * 0.05;
var breath = Math.sin(t * 1.2) * 0.015;
var splats = [];
// Standing figure — 240 points in a vertical cylinder, denser than
// before to feel like a holographic projection.
var ring, k_ring, theta, r, py;
for (ring = 0; ring < 30; ring++) {
py = -1.0 + (ring / 30) * 2.2; // head (-1.0) → feet (+1.2) in y-down
r = 0.20 + breath * (py < 0 ? 1.5 : 0); // chest expands more on inhale
for (k_ring = 0; k_ring < 16; k_ring++) {
theta = (k_ring / 16) * Math.PI * 2;
splats.push({
center: [
sway + Math.cos(theta) * r,
py,
2.3 + Math.sin(theta) * r
],
color: [0.91, 0.65, 0.20],
opacity: 1.0,
scale: [0.018, 0.018, 0.018]
});
}
}
// 17 COCO keypoints in normalized [0,1] image coords (matches live shape)
var headY = 0.18;
var keypoints = [
[0.50 + sway * 0.05, headY, 0.95], // 0 nose
[0.52 + sway * 0.05, headY - 0.01, 0.92], // 1 leftEye
[0.48 + sway * 0.05, headY - 0.01, 0.92], // 2 rightEye
[0.54 + sway * 0.05, headY, 0.85], // 3 leftEar
[0.46 + sway * 0.05, headY, 0.85], // 4 rightEar
[0.60 + sway * 0.04, 0.32, 0.93], // 5 leftShoulder
[0.40 + sway * 0.04, 0.32, 0.93], // 6 rightShoulder
[0.65 + sway * 0.03, 0.46, 0.90], // 7 leftElbow
[0.35 + sway * 0.03, 0.46, 0.90], // 8 rightElbow
[0.68, 0.60 + Math.sin(t * 1.4) * 0.02, 0.86], // 9 leftWrist
[0.32, 0.60 - Math.sin(t * 1.4) * 0.02, 0.86], // 10 rightWrist
[0.57, 0.58, 0.94], // 11 leftHip
[0.43, 0.58, 0.94], // 12 rightHip
[0.58, 0.74, 0.90], // 13 leftKnee
[0.42, 0.74, 0.90], // 14 rightKnee
[0.59, 0.92, 0.88], // 15 leftAnkle
[0.41, 0.92, 0.88] // 16 rightAnkle
];
// Wrap the figure in the Seldon-vault context (grid, spiral, starfield, halo)
pushFoundationContext(splats);
demoFrameNum += 1;
return {
splats: splats,
count: splats.length,
frame: demoFrameNum,
live: false,
pipeline: {
skeleton: { keypoints: keypoints, confidence: 0.86 },
vitals: {
breathing_rate: 14 + Math.round(Math.sin(t * 0.05) * 2),
motion_score: 0.18 + Math.abs(sway) * 2
}
}
};
}
function pickDemoFrame() {
// Prefer real face-mesh data when the camera is running; else procedural.
return faceMeshFrame() || syntheticFrame();
}
// Once auto mode confirms there is no /api/splats backend on this origin,
// set this flag so we stop hammering the network with 404 fetches every
// tick. Console stays clean; demo renders locally.
var networkDisabled = false;
// Exponential backoff state for explicit ?backend=<url>. The user's
// local server may be down (ERR_CONNECTION_REFUSED) and we shouldn't
// hammer it 10 Hz indefinitely. After each failure we lengthen the
// delay; on success we snap back to the normal cadence.
var BASE_INTERVAL_MS = 250;
var MAX_INTERVAL_MS = 30000;
var currentIntervalMs = BASE_INTERVAL_MS;
var consecutiveFailures = 0;
var fetchTimer = null;
var lastBackendError = null;
function scheduleNextFetch(delayMs) {
if (fetchTimer) clearTimeout(fetchTimer);
fetchTimer = setTimeout(fetchCloud, delayMs);
}
async function fetchCloud() {
// Demo-only mode: never hit the network. Use the normal cadence.
if (backendArg === "demo" || networkDisabled) {
transportMode = "demo";
handleData(pickDemoFrame());
scheduleNextFetch(BASE_INTERVAL_MS);
return;
}
try {
var resp = await fetch(buildSplatsUrl(), { cache: "no-store" });
if (!resp.ok) throw new Error("HTTP " + resp.status);
var data = await resp.json();
transportMode = (backendArg === "auto") ? "live" : "remote";
consecutiveFailures = 0;
currentIntervalMs = BASE_INTERVAL_MS;
lastBackendError = null;
handleData(data);
scheduleNextFetch(BASE_INTERVAL_MS);
} catch (err) {
consecutiveFailures += 1;
lastBackendError = err && err.message ? err.message : String(err);
if (requireLive) {
document.getElementById("stats").innerHTML =
'<span class="demo">&#9679; OFFLINE</span><br>Live backend required (?live=1) but unreachable.<br><span class="label">' + lastBackendError + '</span>';
// Even strict-live: back off so we don't spam.
currentIntervalMs = Math.min(currentIntervalMs * 2, MAX_INTERVAL_MS);
scheduleNextFetch(currentIntervalMs);
return;
}
// Auto mode + first failure → assume static host (Pages), disable
// network entirely so the console stays clean.
if (backendArg === "auto") {
networkDisabled = true;
transportMode = "demo";
handleData(pickDemoFrame());
scheduleNextFetch(BASE_INTERVAL_MS);
return;
}
// Explicit backend (?backend=<url>) — keep trying with
// exponential backoff: 250 ms → 500 ms → 1 s → 2 s … up to 30 s.
// Render the demo while we wait so the scene stays alive, and
// surface the failure so the user knows the server is down.
currentIntervalMs = Math.min(Math.max(BASE_INTERVAL_MS * Math.pow(2, consecutiveFailures - 1), 1000), MAX_INTERVAL_MS);
transportMode = "demo";
var demoFrame = pickDemoFrame();
demoFrame._backendUnreachable = true;
demoFrame._backendUrl = backendArg;
demoFrame._backendError = lastBackendError;
demoFrame._retryInMs = currentIntervalMs;
handleData(demoFrame);
scheduleNextFetch(currentIntervalMs);
}
}
function handleData(data) {
try {
if (data.splats && data.frame !== lastFrame) {
// Compute CSI frame rate
var now = Date.now();
@@ -117,7 +561,23 @@
}
prevTimestamp = now;
lastFrame = data.frame;
updateSplats(data.splats);
// Overlay browser face mesh on top of backend splats when both
// are active — lets visitors see their own face *plus* the
// ESP32-driven point cloud in the same scene. Demo mode (where
// data.source === "face-mesh") already includes the face, so
// we skip this branch there to avoid double-counting.
var rendered = data.splats;
var faceOverlay = false;
if (data.source !== "face-mesh"
&& faceMeshState === "running"
&& latestFaceLandmarks) {
rendered = data.splats.slice();
pushFaceSplats(rendered);
faceOverlay = true;
}
data._faceOverlay = faceOverlay;
updateSplats(rendered);
// Draw skeleton if available
var pipe = data.pipeline;
@@ -127,14 +587,40 @@
clearSkeleton();
}
// Build info panel
var mode = data.live
? '<span class="live">&#9679; LIVE</span>'
: '<span class="demo">&#9679; DEMO</span>';
var html = mode + " Camera + CSI<br>"
+ "Splats: " + data.count + "<br>"
// Build info panel — badge reflects active transport
var mode;
if (transportMode === "live") {
mode = '<span class="live">&#9679; LIVE</span> Local Backend';
} else if (transportMode === "remote") {
mode = '<span class="live">&#9679; REMOTE</span> ' + backendArg;
} else if (data.source === "face-mesh") {
mode = '<span class="face">&#9679; DEMO</span> Your Face (MediaPipe)';
} else {
mode = '<span class="demo">&#9679; DEMO</span> Synthetic';
}
if (data._faceOverlay) {
mode += ' <span class="face">+ face overlay</span>';
}
var splatCount = rendered ? rendered.length : data.count;
var html = mode + "<br>"
+ "Splats: " + splatCount + "<br>"
+ "Frame: " + data.frame;
// Unreachable backend banner — explicit ?backend=<url> failed
// to connect. Show actionable guidance instead of leaving the
// user staring at a "demo" badge wondering why their ESP32
// feed isn't visible.
if (data._backendUnreachable) {
var nextSec = Math.round((data._retryInMs || 1000) / 1000);
html += '<div class="section">'
+ '<span class="demo">&#9679; ' + data._backendUrl + '</span> unreachable'
+ '<br><span class="label">' + (data._backendError || "connection failed") + '</span>'
+ '<br><span class="label">retry in ' + nextSec + 's</span>'
+ '<br><br><span class="label">start the server:</span>'
+ '<br><code style="color:#e8a634">cargo run -p wifi-densepose-pointcloud --release \\<br>&nbsp;&nbsp;-- serve --bind 127.0.0.1:9880</code>'
+ '</div>';
}
// CSI frame rate
html += '<div class="section">'
+ '<span class="label">CSI Rate:</span> '
@@ -187,8 +673,69 @@
}
} catch(e) {}
}
// Wire the camera CTA. The camera is now overlay-able on every
// transport mode: in demo it IS the subject; in live/remote it
// overlays the backend splats so the visitor sees their face
// alongside the ESP32-driven point cloud.
(function wireCamCta() {
var btn = document.getElementById("cam-cta");
if (!btn) return;
if (requireLive) {
// Strict-live mode shows the offline panel — no camera UI.
btn.classList.add("hidden");
return;
}
// In remote mode, label the button as an overlay action.
if (backendArg.startsWith("http")) {
btn.textContent = "▶ Add face overlay";
}
btn.addEventListener("click", function() {
btn.textContent = backendArg.startsWith("http")
? "Starting overlay…"
: "Initializing the Vault…";
startFaceMesh();
});
})();
// Wire the ESP32 backend CTA: prompts for a ruview-pointcloud serve URL,
// persists last-used value in localStorage, and reloads with the
// ?backend=<url> query so the existing remote-mode path takes over.
// Disconnect by clicking again when already connected.
(function wireEspCta() {
var btn = document.getElementById("esp-cta");
if (!btn) return;
var connected = backendArg.startsWith("http");
if (connected) {
btn.classList.add("connected");
btn.textContent = "📡 ESP32 connected · disconnect";
}
btn.addEventListener("click", function() {
if (connected) {
// Strip ?backend= from current URL and reload — return to demo.
var u = new URL(window.location.href);
u.searchParams.delete("backend");
window.location.href = u.toString();
return;
}
var stored;
try { stored = localStorage.getItem("ruview.backendUrl"); } catch (_) { stored = null; }
var def = stored || "http://127.0.0.1:9880";
var url = window.prompt(
"Enter the ruview-pointcloud serve URL (run `ruview-pointcloud serve` locally with your ESP32 streaming CSI to UDP port 3333):",
def
);
if (!url) return;
url = url.replace(/\/+$/, "");
try { localStorage.setItem("ruview.backendUrl", url); } catch (_) {}
var u2 = new URL(window.location.href);
u2.searchParams.set("backend", url);
window.location.href = u2.toString();
});
})();
// fetchCloud self-schedules via setTimeout — no setInterval to avoid
// overlapping calls on slow networks and to support exponential backoff.
fetchCloud();
setInterval(fetchCloud, 500);
function updateSplats(splats) {
if (pointsMesh) scene.remove(pointsMesh);
@@ -20,6 +20,11 @@ name = "verify-training"
path = "src/bin/verify_training.rs"
required-features = ["tch-backend"]
[[bin]]
name = "pretrain-mae"
path = "src/bin/pretrain_mae.rs"
required-features = ["tch-backend"]
[features]
default = []
tch-backend = ["tch"]
+18
View File
@@ -82,6 +82,24 @@ wifi-densepose-train/src/
trainer.rs -- (tch) Training loop orchestrator [feature-gated]
```
## MERIDIAN-MAE — masked-autoencoder pre-training (ADR-027 §2.0)
The `csi_mae` module implements a CIG-MAE-style **dual-stream (amplitude + phase)** masked
autoencoder for cross-domain CSI pre-training. The thesis (2026-Q2 SOTA survey, arXiv:2511.18792):
cross-room generalisation is a *data-breadth* problem — pre-train one CSI encoder on heterogeneous
capture, attach a small task head — not a bigger-pose-net problem.
* Pure-Rust (always built): `MaeConfig`, `MaskStrategy` (`Random` / `InfoGuided` — the latter
variance-weights token selection so high-information tokens are masked), `TokenLayout`,
`mask_csi_window`, `reassemble_tokens`. Dependency-free deterministic masking.
* `csi_mae::model` (feature `tch-backend`): `CsiMae` (encoder over visible tokens → latent →
decoder reconstructs masked amplitude+phase), `reconstruction_loss`, `MaeBatch`, `pretrain_step`.
* Driver: `cargo run -p wifi-densepose-train --features tch-backend --bin pretrain-mae -- --epochs 5`
(synthetic data). GPU run: `bash scripts/pretrain-mae-gcloud.sh` (prototype wiring stub).
See `docs/adr/ADR-027-cross-environment-domain-generalization.md` §2.0 for the full plan
(heterogeneous-CSI ingest, GPU pre-train, fine-tune handoff, cross-domain eval).
## Related Crates
| Crate | Role |
@@ -0,0 +1,108 @@
//! `pretrain-mae` — drive the MERIDIAN CSI masked-autoencoder pre-train on a
//! deterministic `SyntheticCsiDataset` (ADR-027 §2.0, prototype iteration 2).
//!
//! This is the *prototype* driver — it exercises the full pre-train loop
//! (mask → encode visible → reconstruct masked amplitude+phase → optimiser
//! step) end-to-end on synthetic CSI. Real cross-domain pre-training (iter 3+)
//! ingests heterogeneous capture — MM-Fi / Wi-Pose / `data/recordings/` /
//! multi-band virtual sub-carriers — and runs on GPU (`scripts/gcloud-train.sh`
//! / the cognitum project).
//!
//! ```text
//! cargo run -p wifi-densepose-train --features tch-backend --bin pretrain-mae -- --epochs 5
//! ```
//!
//! Only compiled with `--features tch-backend` (see Cargo.toml `required-features`).
use clap::Parser;
use tch::nn::OptimizerConfig;
use tch::{nn, Device};
use wifi_densepose_train::csi_mae::model::{pretrain_step, CsiMae, MaeBatch};
use wifi_densepose_train::csi_mae::{MaeConfig, MaskStrategy, TokenLayout};
use wifi_densepose_train::dataset::{CsiDataset, SyntheticConfig, SyntheticCsiDataset};
/// MERIDIAN CSI masked-autoencoder pre-train (prototype, synthetic data).
#[derive(Parser, Debug)]
#[command(name = "pretrain-mae", version, about)]
struct Cli {
/// Number of epochs over the synthetic dataset.
#[arg(long, default_value_t = 5)]
epochs: usize,
/// Mini-batch size (windows per optimiser step).
#[arg(long, default_value_t = 8)]
batch: usize,
/// Number of synthetic samples to generate.
#[arg(long, default_value_t = 256)]
samples: usize,
/// Adam learning rate.
#[arg(long, default_value_t = 1e-3)]
lr: f64,
/// Fraction of tokens masked per window.
#[arg(long, default_value_t = 0.75)]
mask_ratio: f64,
/// Optional path to save the pre-trained variable store (`.ot`).
#[arg(long)]
save: Option<String>,
}
fn main() -> anyhow::Result<()> {
let cli = Cli::parse();
let _ = tracing_subscriber::fmt::try_init();
let ds = SyntheticCsiDataset::new(cli.samples, SyntheticConfig::default());
if ds.len() < cli.batch {
anyhow::bail!("need at least --batch ({}) samples, have {}", cli.batch, ds.len());
}
let s0 = ds.get(0)?;
let layout = TokenLayout::from_window(s0.amplitude.view());
let n_tokens = layout.n_tokens as i64;
let mut cfg = MaeConfig::default();
cfg.token_dim = layout.token_dim;
cfg.mask_ratio = cli.mask_ratio;
cfg.validate().map_err(anyhow::Error::msg)?;
let device = Device::cuda_if_available();
let vs = nn::VarStore::new(device);
let model = CsiMae::new(&vs.root(), &cfg, n_tokens);
let mut opt = nn::Adam::default().build(&vs, cli.lr)?;
println!(
"pretrain-mae: device={device:?} n_tokens={n_tokens} token_dim={} V={} M={} samples={} batch={} epochs={} lr={} mask_ratio={}",
cfg.token_dim, model.n_visible, model.n_masked, cli.samples, cli.batch, cli.epochs, cli.lr, cli.mask_ratio
);
let mut step: u64 = 0;
for epoch in 0..cli.epochs {
let mut epoch_loss = 0.0_f64;
let mut nb = 0_usize;
let mut i = 0_usize;
while i + cli.batch <= ds.len() {
let mut windows = Vec::with_capacity(cli.batch);
for j in i..i + cli.batch {
let s = ds.get(j)?;
windows.push((s.amplitude, s.phase));
}
let seed = step.wrapping_mul(0x9E37_79B9_7F4A_7C15) ^ 0xC511_0027;
let batch = MaeBatch::from_windows(&windows, &cfg, seed, MaskStrategy::InfoGuided, device)
.map_err(anyhow::Error::msg)?;
let loss = pretrain_step(&model, &mut opt, &batch);
if !loss.is_finite() {
anyhow::bail!("non-finite loss at epoch {epoch} step {step}");
}
epoch_loss += loss;
nb += 1;
step += 1;
i += cli.batch;
}
let avg = if nb > 0 { epoch_loss / nb as f64 } else { f64::NAN };
println!("epoch {epoch}: avg reconstruction loss = {avg:.6} ({nb} batches)");
}
if let Some(path) = cli.save {
vs.save(&path)?;
println!("saved pre-trained variable store → {path}");
}
Ok(())
}
File diff suppressed because it is too large Load Diff
@@ -44,6 +44,7 @@
#![warn(missing_docs)]
pub mod config;
pub mod csi_mae;
pub mod dataset;
pub mod domain;
pub mod error;
@@ -79,6 +80,7 @@ pub use error::TrainResult as TrainResultAlias;
pub use subcarrier::{compute_interp_weights, interpolate_subcarriers, select_subcarriers_by_variance};
// MERIDIAN (ADR-027) re-exports.
pub use csi_mae::{mask_csi_window, reassemble_tokens, MaeConfig, MaskStrategy, MaskedCsi, TokenLayout};
pub use domain::{
AdversarialSchedule, DomainClassifier, DomainFactorizer, GradientReversalLayer,
};