mirror of
https://github.com/ruvnet/RuView
synced 2026-06-09 10:13:17 +00:00
Compare commits
3 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| f751740d3d | |||
| db6df747b9 | |||
| 4bbb004f2d |
@@ -14,14 +14,10 @@
|
||||
> **Beta Software** — Under active development. APIs and firmware may change. Known limitations:
|
||||
> - ESP32-C3 and original ESP32 are not supported (single-core, insufficient for CSI DSP)
|
||||
> - Single ESP32 deployments have limited spatial resolution — use 2+ nodes or add a [Cognitum Seed](https://cognitum.one) for best results
|
||||
> - Camera-free pose accuracy is limited (PCK@20 ≈ 2.5% with proxy labels) — [camera ground-truth training](docs/adr/ADR-079-camera-ground-truth-training.md) targets **35%+ PCK@20**; the pipeline is implemented, but the data-collection and evaluation phases (ADR-079 P7–P9) are still pending, so no measured camera-supervised PCK@20 has been published yet
|
||||
> - Camera-free pose accuracy is limited (PCK@20 ≈ 2.5% with proxy labels) — [camera ground-truth training](docs/adr/ADR-079-camera-ground-truth-training.md) targets **35%+ PCK@20**; the pipeline is implemented, but the data-collection and evaluation phases (ADR-079 P7–P9) are still pending.
|
||||
>
|
||||
> Contributions and bug reports welcome at [Issues](https://github.com/ruvnet/RuView/issues).
|
||||
|
||||
> **What's new (2026-05-23):**
|
||||
> - **ESP32-C6 firmware substrate closed** ([ADR-110](docs/adr/ADR-110-esp32-c6-firmware-extension.md), [v0.7.0-esp32](https://github.com/ruvnet/RuView/releases/tag/v0.7.0-esp32)) — Wi-Fi 6 + 802.15.4 + TWT + LP-core dual-target firmware with a **measured 99.56 % cross-board ESP-NOW mesh RX rate**, **104.1 µs smoothed sync stdev**, **3.95× EMA suppression** — the ADR-110 §2.4 ≤100 µs multistatic alignment target is empirically met. 32-byte sync packet, host decoders in Python + Rust with a cross-language hex pin, REST `/api/v1/mesh` + `/mesh/metrics` (Prometheus), WebSocket `sensing_update.sync` field. [PR #764](https://github.com/ruvnet/RuView/pull/764).
|
||||
> - **Home Assistant + Matter integration** ([ADR-115](docs/adr/ADR-115-home-assistant-integration.md)) — drop into any HA install with `--mqtt`, pair into Apple Home / Google Home / Alexa / SmartThings as a Matter Bridge, 21 entity kinds per node (11 raw + 10 inferred semantic primitives), 8 starter HA Blueprints, 3 Lovelace dashboards, privacy mode that strips biometrics at the wire while semantic states keep working. [PR #778](https://github.com/ruvnet/RuView/pull/778).
|
||||
|
||||
## **See through walls with WiFi** ##
|
||||
|
||||
**Turn ordinary WiFi into a spatial intelligence / sensing system.** Detect people, measure breathing and heart rate, track movement, and monitor rooms — through walls, in the dark, with no cameras or wearables. Just physics.
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
# ADR-116: Home Assistant + Matter as a Cognitum Seed cog (`cog-ha-matter`)
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Status** | Proposed (research in progress — `docs/research/ADR-116-ha-matter-cog-research.md`) |
|
||||
| **Date** | 2026-05-23 |
|
||||
| **Deciders** | ruv |
|
||||
| **Codename** | **HA-COG** — HA + Matter, packaged for the Seed |
|
||||
| **Relates to** | [ADR-110](ADR-110-esp32-c6-firmware-extension.md) (C6 firmware substrate), [ADR-115](ADR-115-home-assistant-integration.md) (HA-DISCO + HA-MIND + HA-FABRIC), [ADR-102](ADR-102-edge-module-registry.md) (cog catalog), [ADR-101](ADR-101-pose-estimation-cog.md) (cog packaging precedent) |
|
||||
| **Tracking issue** | TBD — file under RuView issue tracker once research dossier lands |
|
||||
|
||||
---
|
||||
|
||||
## 1. Context
|
||||
|
||||
ADR-115 shipped the Home Assistant + Matter integration as a **`--mqtt` flag on `wifi-densepose-sensing-server`** — a Rust binary that runs on a Pi / Linux box, consumes UDP frames from the ESP32 fleet, and publishes MQTT for any Home Assistant install to discover. That works, but it makes HA+Matter a *configuration of the aggregator*, not an *installable artifact* a Cognitum Seed user can drop into their existing fleet.
|
||||
|
||||
The Cognitum Seed already has a [105-cog catalog](https://seed.cognitum.one/store) — packaged Seed apps (`cog-pose-estimation`, `cog-quantum-vitals`, `cog-person-matching`, etc.) that anyone can install from `app-registry.json`. **There is no `cog-ha-matter` yet.** That's the gap this ADR closes.
|
||||
|
||||
The cog packaging precedent is ADR-101 (`cog-pose-estimation`) which ships signed aarch64 + x86_64 binaries on GCS with a `pose_v1.safetensors` weight blob — same shape we'd want for the HA cog.
|
||||
|
||||
### 1.1 Why a cog, not just the existing flag?
|
||||
|
||||
| Path | Distribution | Discovery | Update | Witness | Local AI |
|
||||
|---|---|---|---|---|---|
|
||||
| `--mqtt` on `sensing-server` | manual install of the Rust binary | none | manual | none | external |
|
||||
| **`cog-ha-matter` Seed cog** | `app-registry.json` listing, one-click install | mDNS / cog browser | OTA via cog runtime | Ed25519 witness chain | local ruvllm + RuVector |
|
||||
|
||||
The cog ships HA+Matter as a first-class Seed feature — same UX as installing a pose estimator or person matcher.
|
||||
|
||||
### 1.2 What this ADR is *not*
|
||||
|
||||
- Not a deprecation of the `--mqtt` flag on sensing-server. The flag stays for Pi / Linux deployments without a Seed; the cog is the Seed-native option.
|
||||
- Not a port of HA-MIND / HA-DISCO logic to a different language. The Rust crate already exists; the cog *wraps* it as a Seed-installable artifact + adds Seed-specific surfaces (witness, RuVector, ruvllm-driven thresholds).
|
||||
- Not a Matter SDK ship. ADR-115 §9.10 deferred the matter-rs SDK wiring to v0.7.1; this ADR continues that deferral and focuses on the *cog packaging* + *first-class Seed integration*, with Matter Bridge mode shipping in v0.8 once the SDK is ready.
|
||||
|
||||
## 2. Decision (provisional — to be refined by the research dossier)
|
||||
|
||||
Build **`cog-ha-matter`** as a Cognitum Seed cog with these surfaces:
|
||||
|
||||
### 2.1 Core entity surface (unchanged from ADR-115)
|
||||
|
||||
The cog republishes the same 21 entities per node (11 raw + 10 semantic primitives) over MQTT auto-discovery, so HA installations behave identically whether the source is a Seed cog or an external sensing-server.
|
||||
|
||||
### 2.2 Seed-native enhancements
|
||||
|
||||
- **Self-contained MQTT broker (optional)** — if the user doesn't already run mosquitto, the cog can host an embedded broker on `cognitum-seed.local:1883` and act as the HA endpoint directly.
|
||||
- **mDNS service advertisement** — `_ruview-ha._tcp` so HA's discovery integration finds the Seed without manual config.
|
||||
- **RuVector-backed semantic-primitive thresholds** — instead of static `semantic-thresholds.yaml`, the cog learns per-home thresholds via a SONA-adapted RuVector model (matches the Seed's local-first AI story).
|
||||
- **Ed25519 witness chain** — every state transition logged with a Seed signature so care-home / regulated deployments can audit decisions.
|
||||
- **OTA firmware coordination** — the cog manages C6 firmware updates for ESP32-C6 nodes in the mesh (ADR-110 substrate).
|
||||
|
||||
### 2.3 Matter dimensions (depend on research findings)
|
||||
|
||||
The research dossier covers (a) Matter Bridge vs Matter Device mode, (b) Thread Border Router on the Seed's ESP32-S3 (if feasible), (c) CSA certification path, (d) which Matter device classes map cleanly to which entities. **Decision deferred** until the dossier lands; this ADR will be updated in §3 with the specific Matter feature set.
|
||||
|
||||
### 2.4 Multi-Seed federation
|
||||
|
||||
Multiple Seeds in adjacent rooms coordinate via:
|
||||
- ESP-NOW mesh (ADR-110 substrate) for time alignment
|
||||
- mDNS for service discovery
|
||||
- Witness chain replication for cross-Seed event provenance
|
||||
|
||||
The federation model is the natural extension of ADR-110's mesh substrate into the application layer. Specifically: ADR-110 gives us ≤100 µs cross-board sync; this ADR uses that to deduplicate cross-Seed events (one fall, one alert) and reconstruct multi-room transitions (one occupant, room A → hallway → room B).
|
||||
|
||||
## 3. Open questions (research dossier will answer)
|
||||
|
||||
1. **Matter Bridge vs Matter Root** — can the Seed act as a commissioner, or is Bridge mode the only realistic 2026 option?
|
||||
2. **Thread Border Router** — does the ESP32-S3 in the Seed (or a paired ESP32-C6 node) host a Thread Border Router cleanly, and does that buy us anything HA users care about?
|
||||
3. **HACS integration value-add** — beyond MQTT auto-discovery, what does a custom HA integration unlock? (config flow / repairs / device-class custom entities / service catalog).
|
||||
4. **CSA certification cost / timeline** — what's the minimum CSA-compliant subset and what does it cost to ship a CSA-certified Matter device today?
|
||||
5. **Cog binary size + Seed RAM budget** — the Seed has 8 MB PSRAM + 320 KB SRAM. The cog must fit.
|
||||
6. **ruvllm + RuVector latency for semantic primitives** — can the 10 inferred states run at 5 Hz on the Seed without external compute?
|
||||
7. **HIPAA / FDA classification** — when does fall-detection cross into regulated medical-device territory, and does the `--privacy-mode` strip + Matter Health device class give us a defensible "wellness device" position?
|
||||
|
||||
## 4. Implementation phases
|
||||
|
||||
| Phase | Scope | Status |
|
||||
|---|---|---|
|
||||
| **P1** | Research dossier (`docs/research/ADR-116-ha-matter-cog-research.md`) | _in progress_ (deep-researcher agent) |
|
||||
| **P2** | Cog manifest + binary scaffold (`cogs/ha-matter/`) | pending |
|
||||
| **P3** | Wrap existing ADR-115 MQTT publisher as cog entry point | pending |
|
||||
| **P4** | Seed-native enhancements (embedded broker, mDNS, witness) | pending |
|
||||
| **P5** | RuVector-backed threshold learning (SONA adaptation) | pending |
|
||||
| **P6** | Multi-Seed federation (cross-Seed dedup + witness) | pending |
|
||||
| **P7** | Matter Bridge mode (depends on matter-rs / esp-matter readiness) | pending |
|
||||
| **P8** | Cog signing + `app-registry.json` listing + Seed Store entry | pending |
|
||||
| **P9** | HACS integration repo (`hass-wifi-densepose`) for HA-side install path | pending |
|
||||
| **P10** | Witness bundle + CSA-style spec compliance check | pending |
|
||||
|
||||
## 5. References
|
||||
|
||||
- ADR-101 — `cog-pose-estimation` packaging precedent (signed binaries on GCS, .cog manifest)
|
||||
- ADR-102 — edge module registry (`app-registry.json` surfaces all cogs)
|
||||
- ADR-110 — ESP32-C6 firmware substrate (mesh time alignment that multi-Seed federation depends on)
|
||||
- ADR-115 — HA-DISCO + HA-MIND + HA-FABRIC (the Rust crate this cog wraps)
|
||||
- `docs/research/ADR-116-ha-matter-cog-research.md` — companion research dossier (deep-researcher agent in progress)
|
||||
- Cognitum Seed store: https://seed.cognitum.one/store
|
||||
- Matter spec: https://csa-iot.org/all-solutions/matter/
|
||||
- HACS integration target: https://github.com/ruvnet/hass-wifi-densepose (planned)
|
||||
@@ -389,6 +389,120 @@ Per [ADR-115 §3.9](../adr/ADR-115-home-assistant-integration.md#39-tls--auth),
|
||||
|
||||
---
|
||||
|
||||
## Applications — what people actually do with this
|
||||
|
||||
The 21 entities per node — 11 raw signals (presence, person count, breathing, heart rate, motion, RSSI, etc.) and 10 inferred semantic states (someone-sleeping, possible-distress, room-active, elderly-inactivity-anomaly, meeting-in-progress, bathroom-occupied, fall-risk-elevated, bed-exit, no-movement, multi-room-transition) — slot into Home Assistant like any other sensor. The list below groups real-world uses so you can pick the ones that match your space.
|
||||
|
||||
### Personal & home
|
||||
|
||||
| Use case | Which entities | What HA does with it |
|
||||
|---|---|---|
|
||||
| **"Goodnight" routine** | `someone_sleeping` | Dim hallway lights to 5%, lock doors, drop thermostat 2 °C, mute notifications. Blueprint `02-dim-hallway-when-sleeping.yaml`. |
|
||||
| **"Wake up" routine** | `bed_exit` | When you get out of bed in the morning, turn on the bathroom heater, raise blinds, start the coffee. Blueprint `03-wake-routine-on-bed-exit.yaml`. |
|
||||
| **Meeting / focus mode** | `meeting_in_progress` | Multi-person presence in the office for >5 min → set a "Do Not Disturb" status, dim overhead lights, pause vacuum schedule. Blueprint `05-meeting-lights-presence-mode.yaml`. |
|
||||
| **Bathroom fan automation** | `bathroom_occupied` | Turn the exhaust fan on while a bathroom is occupied; turn it off 5 min after you leave. Blueprint `06-bathroom-fan-while-occupied.yaml`. |
|
||||
| **Forgotten kitchen / iron** | `presence` per room | "Stove on, kitchen empty for 10 min" → push notification + optional smart-plug cut-off. |
|
||||
| **Pet-only at home** | `n_persons == 0` for hours but `motion > 0` | Distinguish dog moving around from human presence — don't trigger empty-home automations during the day. |
|
||||
| **Sleep quality tracking** | `breathing_rate_bpm`, `heart_rate_bpm` (privacy off) | Push nightly averages to HA Statistics, graph in Grafana. No watch, no app. |
|
||||
| **Toddler bed safety** | `no_movement` in a child's room overnight | Alert parents if breathing-rate signal drops out unexpectedly. |
|
||||
| **Pre-arrival lighting** | `multi_room_transition` | When you walk from the entry hall toward the living room, anticipate the route and pre-warm those lights. |
|
||||
|
||||
### Healthcare & assisted living (AAL)
|
||||
|
||||
| Use case | Which entities | Why this works |
|
||||
|---|---|---|
|
||||
| **Fall detection + escalation** | `fall_detected` | Phase-acceleration spike + 3-frame debounce. Trigger a Lovelace alert, then escalate to a phone call if the person stays still for >2 min. Blueprint `07-fall-risk-escalation.yaml`. |
|
||||
| **Elderly inactivity anomaly** | `elderly_inactivity_anomaly` | Learns a person's normal day-pattern and flags deviations (e.g. usually up by 9 am, hasn't moved by 11 am). Blueprint `04-alert-elderly-inactivity-anomaly.yaml`. |
|
||||
| **Privacy-mode care monitoring** | `possible_distress` + `no_movement` + `someone_sleeping` | Run with `--privacy-mode` — heart rate and breathing values are stripped at the wire, but the *inferred states* keep working. Care staff sees "Distress detected" without ever seeing the underlying biometric numbers. The architectural win that makes RuView legally deployable in care homes. |
|
||||
| **Sleep apnea screening** | `breathing_rate_bpm` + `breathing_confidence` | Track per-night BPM histograms; flag dips that correlate with apnea events. |
|
||||
| **Post-surgery recovery monitoring** | `no_movement` + `bed_exit` + `breathing_rate_bpm` | Hospital-discharge patient at home; rule: "no bed exits in 12 h" triggers a check-in call. |
|
||||
| **Dementia wandering detection** | `multi_room_transition` + nighttime gate | Multi-room transitions between 23:00 and 06:00 alert a caregiver — without GPS tags or wearables the person may refuse to wear. |
|
||||
| **Bathroom occupancy timeout** | `bathroom_occupied` for >30 min | Possible fall or medical incident; push to caregiver. |
|
||||
|
||||
### Security & safety
|
||||
|
||||
| Use case | Which entities | What HA does with it |
|
||||
|---|---|---|
|
||||
| **Auto-arm when no one's home** | `presence` across all nodes for >10 min | Switch HA alarm panel to "armed_away" — replaces door-sensor + key-fob combos. Blueprint `08-auto-arm-security-when-not-active.yaml`. |
|
||||
| **Intrusion detection (presence without entry)** | `presence` true while no door/window sensor opened | Real signal of someone inside who shouldn't be. RF-based, can't be defeated by covering a camera. |
|
||||
| **Through-wall presence verification** | `presence` per room, even with doors closed | Confirms HA "someone is home" estimate without requiring per-room PIR sensors. |
|
||||
| **Hostage / silent-distress mode** | `possible_distress` (motion + elevated HR) | If you've published HR + privacy is off, abnormal motion-plus-physiology can trigger a silent alarm. |
|
||||
| **Garage / shed monitoring** | `presence` in outbuildings | Wi-Fi reaches places PIR doesn't (metal shed walls block IR but pass through Wi-Fi). |
|
||||
| **Camera-free child safety zone** | `presence` near pool / stairs / fireplace | Push alert if a known child-room sensor sees presence in restricted zone — no cameras, no privacy concerns. |
|
||||
|
||||
### Commercial buildings & retail
|
||||
|
||||
| Use case | Which entities | What it enables |
|
||||
|---|---|---|
|
||||
| **Real-time office occupancy** | `n_persons`, `presence`, `room_active` | Live dashboard of how full each meeting room is — no cameras, no badges. Better than door-counters because people are detected mid-meeting, not just on entry. |
|
||||
| **HVAC demand-controlled ventilation** | `n_persons` | Adjust ventilation per room based on people present — saves 20-30% on cooling/heating in shared offices. |
|
||||
| **Meeting room booking truth** | `meeting_in_progress` vs calendar | "Meeting booked, but no one's there" → auto-release the room. |
|
||||
| **Retail dwell time + heat-mapping** | `presence` + `motion` over time | Where do customers linger? Which aisles are empty? Anonymous (no faces), through-clothing, works in low light. |
|
||||
| **Queue length estimation** | `n_persons` near a checkout | Trigger "open another register" automation. |
|
||||
| **Cleaning verification** | `no_movement` in a room for >X min after hours | Confirms cleaning crew has finished the room without requiring badges. |
|
||||
| **Lone-worker safety (warehouses, labs)** | `no_movement` + `possible_distress` | OSHA-compatible solo-worker monitoring without wearables. |
|
||||
|
||||
### Industrial & infrastructure
|
||||
|
||||
| Use case | Which entities | What it enables |
|
||||
|---|---|---|
|
||||
| **Manned-station occupancy** | `presence` | Control rooms / lab benches — confirm operator presence without log-in friction. |
|
||||
| **Restricted-zone intrusion** | `presence` + `multi_room_transition` | Server room / clean room / pharmaceutical lab — RF passes through doors better than IR. |
|
||||
| **Equipment-room ventilation** | `presence` in a UPS / battery room | Turn on exhaust fans when a technician enters. |
|
||||
| **Hazardous-area worker tracking** | `presence` + `no_movement` | Confirm workers in an electrical or chemical area are still moving (not collapsed). |
|
||||
| **Construction-site after-hours** | `presence` + scheduled gate | Detect anyone on-site after 18:00 → site supervisor alert. |
|
||||
| **Maritime / offshore quarters** | `breathing_rate` overnight | Confirm bunk occupants are alive without wearables that often get removed during sleep. |
|
||||
|
||||
### Education & public spaces
|
||||
|
||||
| Use case | Which entities | What it enables |
|
||||
|---|---|---|
|
||||
| **Classroom occupancy** | `n_persons`, `room_active` | HVAC and lighting per actual headcount — saves energy in classrooms used 40% of the day. |
|
||||
| **Library / study room availability** | `presence` + `n_persons` | Live "rooms available" page without webcams. |
|
||||
| **Lecture hall attendance** | `n_persons` time-series | No card-swipe required — RF presence is robust to phones-in-pockets. |
|
||||
| **Restroom occupancy signage** | `bathroom_occupied` per stall | Privacy-friendly "in use / available" indicators. |
|
||||
| **Gym / pool capacity** | `n_persons` | Live capacity counter for compliance with limits — no turnstiles needed. |
|
||||
| **Public-transport waiting areas** | `n_persons` + `room_active` | Real-time platform crowd density for transit operator dashboards. |
|
||||
|
||||
### Energy & sustainability
|
||||
|
||||
| Use case | Which entities | What it enables |
|
||||
|---|---|---|
|
||||
| **Per-room lighting auto-off** | `presence` per node | The room-level version of motion-PIR — works through walls, no false-off when sitting still reading. |
|
||||
| **Smart-thermostat zoning** | `room_active`, `n_persons` | Only heat / cool occupied rooms — substantial savings in homes >150 m². |
|
||||
| **Vampire-load cut-off** | `presence` for whole house | When no one is home, smart plugs cut TV / chargers / standby loads. |
|
||||
| **Solar / battery dispatch tuning** | `n_persons`, `motion_energy` | Predict next-hour load based on activity, dispatch battery accordingly. |
|
||||
| **Cold-chain refrigeration alerts** | `presence` + `bathroom_occupied` confusion | Trigger door-checks when an unexpected person spends >10 min near a walk-in freezer. |
|
||||
|
||||
### Research, prototyping & developer use
|
||||
|
||||
| Use case | Which entities | What it enables |
|
||||
|---|---|---|
|
||||
| **Behavioral studies** | Full snapshot stream | Anonymous behavioral data — count, motion, vitals — without IRB-blocking cameras. |
|
||||
| **HCI experiments** | `multi_room_transition` + `presence` | Path-following studies in living labs. |
|
||||
| **Healthcare datasets** | `breathing_rate_bpm` time-series | Generate breathing-rate corpora for ML training without consent forms for facial data. |
|
||||
| **Custom RuView Cogs** | Raw CSI feed + the WebSocket sync field | Bring your own model, consume the firmware-side mesh-aligned timestamps for multistatic fusion. |
|
||||
|
||||
### Combining entities — recipe patterns
|
||||
|
||||
A few patterns appear over and over; if you understand these you can build most of the above yourself:
|
||||
|
||||
1. **"Negative + duration" trip wires** — `no_movement` for N minutes AND time-of-day window → most healthcare and pet/child safety automations.
|
||||
2. **"Two states agree" guards** — `presence == false` AND security panel disarmed AND no door sensor open → strong "house is empty" signal.
|
||||
3. **"Threshold + cooldown"** — `presence_score > 0.7` for 30 s before triggering (smooths over flicker), then a 5 min cooldown before re-arming (prevents oscillation).
|
||||
4. **"Calendar vs reality"** — pair an HA calendar event with `n_persons` → meeting-room auto-release, classroom unused-period detection.
|
||||
5. **"Privacy-mode + semantic-only"** — run `--privacy-mode`, expose only the semantic primitives to HA, keep biometrics on-device. The right default for any deployment with non-tenant occupants.
|
||||
|
||||
### What about regulated environments?
|
||||
|
||||
Run RuView with `--privacy-mode` and only the 10 inferred semantic states reach Home Assistant — heart rate, breathing rate, and pose values are stripped at the MQTT wire. Per ADR-115 §6, this passes:
|
||||
|
||||
- **HIPAA-style minimum-necessary** (no biometric numbers leave the device)
|
||||
- **GDPR purpose-limitation** (the inferred states are the smallest dataset that supports the automation)
|
||||
- **CCPA "sensitive personal information"** (no health data crosses the wire)
|
||||
|
||||
The fall-risk-elevated / possible-distress / someone-sleeping flags still work — they're computed *inside* the sensor pipeline and only the boolean outputs are published. That's the architectural win that makes RuView deployable in care homes, hospitals, schools, and shared-housing scenarios where raw biometrics would be a non-starter.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR-115](../adr/ADR-115-home-assistant-integration.md) — full design rationale
|
||||
|
||||
Reference in New Issue
Block a user