Compare commits

..

6 Commits

Author SHA1 Message Date
rUv 40f19622af fix(firmware,server): watchdog crash + no detection from edge vitals (#321, #323)
* fix(firmware,server): watchdog crash on busy LANs + no detection from edge vitals (#321, #323)

**Firmware (#321):** edge_dsp task now batch-limits frame processing to 4
frames before a 10ms yield. On corporate LANs with high CSI frame rates,
the previous 1-tick-per-frame yield wasn't enough to prevent IDLE1
starvation and task watchdog triggers.

**Sensing server (#323):** When ESP32 runs the edge DSP pipeline (Tier 2+),
it sends vitals packets (magic 0xC5110002) instead of raw CSI frames.
Previously, the server broadcast these as raw edge_vitals but never
generated a sensing_update, so the UI showed "connected" but "0 persons".
Now synthesizes a full sensing_update from vitals data including
classification, person count, and pose generation.

Closes #321
Closes #323

Co-Authored-By: claude-flow <ruv@ruv.net>

* fix(firmware): address review findings — idle busy-spin and observability

- Fix pdMS_TO_TICKS(5)==0 at 100Hz causing busy-spin in idle path (use
  vTaskDelay(1) instead)
- Post-batch yield now 2 ticks (20ms) for genuinely longer pause
- Add s_ring_drops counter to ring_push for diagnosing frame drops
- Expose drop count in periodic vitals log line

Co-Authored-By: claude-flow <ruv@ruv.net>

* fix(server): set breathing_band_power for skeleton animation from vitals

When presence is detected via edge vitals, set breathing_band_power to
0.5 so the UI's torso breathing animation works. Previously hardcoded
to 0.0 which made the skeleton appear static even when breathing rate
was being reported.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-03-27 17:31:06 -04:00
rUv 022499b2f5 fix: add wifi_densepose package for correct module import (#314)
The README Quick Start tells users to `pip install wifi-densepose` and then
`from wifi_densepose import WiFiDensePose`, but no `wifi_densepose` Python
package existed — only `v1/src`. This adds a top-level `wifi_densepose/`
package with a WiFiDensePose facade class matching the documented API, and
updates pyproject.toml to include it in the distribution.

Closes #314
2026-03-27 17:31:03 -04:00
rUv e6068c5efe Enhance README with Cognitum.One reference
Updated project description to include Cognitum.One.
2026-03-25 21:21:58 -04:00
rUv 7a13877fa3 fix(sensing-server): detect ESP32 offline after 5s frame timeout (#300)
The source field was set to "esp32" on the first UDP frame but never
reverted when frames stopped arriving. This caused the UI to show
"Real hardware connected" indefinitely after powering off all nodes.

Changes:
- Add last_esp32_frame timestamp to AppStateInner
- Add effective_source() method with 5-second timeout
- Source becomes "esp32:offline" when no frames received within 5s
- Health endpoint shows "degraded" instead of "healthy" when offline
- All 6 status/health/info API endpoints use effective_source()

Fixes #297

Co-authored-by: Reuven <cohen@ruv-mac-mini.local>
2026-03-24 08:00:18 -04:00
Reuven 6c98c98920 docs(adr): ADR-067 RuVector v2.0.5 upgrade + new crate adoption plan
4-phase plan to upgrade core ruvector dependencies and adopt new crates:
- Phase 1: Bump 5 core crates 2.0.4→2.0.5 (10-30% mincut perf, security fixes)
- Phase 2: Add ruvector-coherence for spectral multi-node CSI coherence
- Phase 3: Add SONA adaptive learning to replace manual logistic regression
- Phase 4: Evaluate ruvector-core ONNX embeddings for CSI pattern matching

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-03-23 21:51:43 -04:00
rUv 5f3c90bf1c fix(sensing-server): add real hysteresis to person count estimation (#295)
The person-count heuristic was causing widespread flickering (#237, #249,
#280, #292) because:

1. Threshold 0.50 for 2-persons was too low — multipath reflections in
   small rooms easily exceeded it
2. No actual hysteresis despite the comment claiming asymmetric thresholds
3. EMA smoothing (α=0.15) was too responsive to transient spikes

Changes:
- Raise up-thresholds: 1→2 persons at 0.65 (was 0.50), 2→3 at 0.85 (was 0.80)
- Add true hysteresis with asymmetric down-thresholds: 2→1 at 0.45, 3→2 at 0.70
- Track prev_person_count in SensingState for state-aware transitions
- Increase EMA smoothing to α=0.10 (~2s time constant at 20 Hz)
- Update all 4 call sites (ESP32, Windows WiFi, multi-BSSID, simulated)

Fixes #292, #280, #237

Co-authored-by: Reuven <cohen@ruv-mac-mini.local>
2026-03-23 21:37:52 -04:00
7 changed files with 503 additions and 46 deletions
+1 -1
View File
@@ -14,7 +14,7 @@
Instead of relying on cameras or cloud models, it observes whatever signals exist in a space such as WiFi, radio waves across the spectrum, motion patterns, vibration, sound, or other sensory inputs and builds an understanding of what is happening locally.
Built on top of [RuVector](https://github.com/ruvnet/ruvector/), the project became widely known for its implementation of WiFi DensePose — a sensing technique first explored in academic research such as Carnegie Mellon University's *DensePose From WiFi* work. That research demonstrated that WiFi signals can be used to reconstruct human pose.
Built on top of [RuVector](https://github.com/ruvnet/ruvector/) Self Learning Vector Memory system and [Cognitum.One](https://Cognitum.One) , the project became widely known for its implementation of WiFi DensePose — a sensing technique first explored in academic research such as Carnegie Mellon University's *DensePose From WiFi* work. That research demonstrated that WiFi signals can be used to reconstruct human pose.
RuView extends that concept into a practical edge system. By analyzing Channel State Information (CSI) disturbances caused by human movement, RuView reconstructs body position, breathing rate, heart rate, and presence in real time using physics-based signal processing and machine learning.
+151
View File
@@ -0,0 +1,151 @@
# ADR-067: RuVector v2.0.4 to v2.0.5 Upgrade + New Crate Adoption
**Status:** Proposed
**Date:** 2026-03-23
**Deciders:** @ruvnet
**Related:** ADR-016 (RuVector training pipeline integration), ADR-017 (RuVector signal + MAT integration), ADR-029 (RuvSense multistatic sensing)
## Context
RuView currently pins all five core RuVector crates at **v2.0.4** (from crates.io) plus a vendored `ruvector-crv` v0.1.1 and optional `ruvector-gnn` v2.0.5. The upstream RuVector workspace has moved to **v2.0.5** with meaningful improvements to the crates we depend on, and has introduced new crates that could benefit RuView's detection pipeline.
### Current Integration Map
| RuView Module | RuVector Crate | Current Version | Purpose |
|---------------|----------------|-----------------|---------|
| `signal/subcarrier.rs` | ruvector-mincut | 2.0.4 | Graph min-cut subcarrier partitioning |
| `signal/spectrogram.rs` | ruvector-attn-mincut | 2.0.4 | Attention-gated spectrogram denoising |
| `signal/bvp.rs` | ruvector-attention | 2.0.4 | Attention-weighted BVP aggregation |
| `signal/fresnel.rs` | ruvector-solver | 2.0.4 | Fresnel geometry estimation |
| `mat/triangulation.rs` | ruvector-solver | 2.0.4 | TDoA survivor localization |
| `mat/breathing.rs` | ruvector-temporal-tensor | 2.0.4 | Tiered compressed breathing buffer |
| `mat/heartbeat.rs` | ruvector-temporal-tensor | 2.0.4 | Tiered compressed heartbeat spectrogram |
| `viewpoint/*` (4 files) | ruvector-attention | 2.0.4 | Cross-viewpoint fusion with geometric bias |
| `crv/` (optional) | ruvector-crv | 0.1.1 (vendored) | CRV protocol integration |
| `crv/` (optional) | ruvector-gnn | 2.0.5 | GNN graph topology |
### What Changed Upstream (v2.0.4 → v2.0.5 → HEAD)
**ruvector-mincut:**
- Flat capacity matrix + allocation reuse — **10-30% faster** for all min-cut operations
- Tier 2-3 Dynamic MinCut (ADR-124): Gomory-Hu tree construction for fast global min-cut, incremental edge insert/delete without full recomputation
- Source-anchored canonical min-cut with SHA-256 witness hashing
- Fixed: unsafe indexing removed, WASM Node.js panic from `std::time`
**ruvector-attention / ruvector-attn-mincut:**
- Migrated to workspace versioning (no API changes)
- Documentation improvements
**ruvector-temporal-tensor:**
- Formatting fixes only (no API changes)
**ruvector-gnn:**
- Panic replaced with `Result` in `MultiHeadAttention` and `RuvectorLayer` constructors (breaking improvement — safer)
- Bumped to v2.0.5
**sona (new — Self-Optimizing Neural Architecture):**
- v0.1.6 → v0.1.8: state persistence (`loadState`/`saveState`), trajectory counter fix
- Micro-LoRA and Base-LoRA for instant and background learning
- EWC++ (Elastic Weight Consolidation) to prevent catastrophic forgetting
- ReasoningBank pattern extraction and similarity search
- WASM support for edge devices
**ruvector-coherence (new):**
- Spectral coherence scoring for graph index health
- Fiedler eigenvalue estimation, effective resistance sampling
- HNSW health monitoring with alerts
- Batch evaluation of attention mechanism quality
**ruvector-core (new):**
- ONNX embedding support for real semantic embeddings
- HNSW index with SIMD-accelerated distance metrics
- Quantization (4-32x memory reduction)
- Arena allocator for cache-optimized operations
## Decision
### Phase 1: Version Bump (Low Risk)
Bump the 5 core crates from v2.0.4 to v2.0.5 in the workspace `Cargo.toml`:
```toml
ruvector-mincut = "2.0.5" # was 2.0.4 — 10-30% faster, safer
ruvector-attn-mincut = "2.0.5" # was 2.0.4 — workspace versioning
ruvector-temporal-tensor = "2.0.5" # was 2.0.4 — fmt only
ruvector-solver = "2.0.5" # was 2.0.4 — workspace versioning
ruvector-attention = "2.0.5" # was 2.0.4 — workspace versioning
```
**Expected impact:** The mincut performance improvement directly benefits `signal/subcarrier.rs` which runs subcarrier graph partitioning every tick. 10-30% faster partitioning reduces per-frame CPU cost.
### Phase 2: Add ruvector-coherence (Medium Value)
Add `ruvector-coherence` with `spectral` feature to `wifi-densepose-ruvector`:
**Use case:** Replace or augment the custom phase coherence logic in `viewpoint/coherence.rs` with spectral graph coherence scoring. The current implementation uses phasor magnitude for phase coherence — spectral Fiedler estimation would provide a more robust measure of multi-node CSI consistency, especially for detecting when a node's signal quality degrades.
**Integration point:** `viewpoint/coherence.rs` — add `SpectralCoherenceScore` as a secondary coherence metric alongside existing phase phasor coherence. Use spectral gap estimation to detect structural changes in the multi-node CSI graph (e.g., a node dropping out or a new reflector appearing).
### Phase 3: Add SONA for Adaptive Learning (High Value)
Replace the logistic regression adaptive classifier in the sensing server with a SONA-backed learning engine:
**Current state:** The sensing server's adaptive training (`POST /api/v1/adaptive/train`) uses a hand-rolled logistic regression on 15 CSI features. It requires explicit labeled recordings and provides no cross-session persistence.
**Proposed improvement:** Use `sona::SonaEngine` to:
1. **Learn from implicit feedback** — trajectory tracking on person-count decisions (was the count stable? did the user correct it?)
2. **Persist across sessions**`saveState()`/`loadState()` replaces the current `adaptive_model.json`
3. **Pattern matching**`find_patterns()` enables "this CSI signature looks like room X where we learned Y"
4. **Prevent forgetting** — EWC++ ensures learning in a new room doesn't overwrite patterns from previous rooms
**Integration point:** New `adaptive_sona.rs` module in `wifi-densepose-sensing-server`, behind a `sona` feature flag. The existing logistic regression remains the default.
### Phase 4: Evaluate ruvector-core for CSI Embeddings (Exploratory)
**Current state:** The person detection pipeline uses hand-crafted features (variance, change_points, motion_band_power, spectral_power) with fixed normalization ranges.
**Potential:** Use `ruvector-core`'s ONNX embedding support to generate learned CSI embeddings that capture room geometry, person count, and activity patterns in a single vector. This would enable:
- Similarity search: "is this CSI frame similar to known 2-person patterns?"
- Transfer learning: embeddings learned in one room partially transfer to similar rooms
- Quantized storage: 4-32x memory reduction for pattern databases
**Status:** Exploratory — requires training data collection and embedding model design. Not a near-term target.
## Consequences
### Positive
- **Phase 1:** Free 10-30% performance gain in subcarrier partitioning. Security fixes (unsafe indexing, WASM panic). Zero API changes required.
- **Phase 2:** More robust multi-node coherence detection. Helps with the "flickering persons" issue (#292) by providing a second opinion on signal quality.
- **Phase 3:** Fundamentally improves the adaptive learning pipeline. Users no longer need to manually record labeled data — the system learns from ongoing use.
- **Phase 4:** Path toward real ML-based detection instead of heuristic thresholds.
### Negative
- **Phase 1:** Minimal risk — semver minor bump, no API breaks.
- **Phase 2:** Adds a dependency. Spectral computation has O(n) cost per tick for Fiedler estimation (n = number of subcarriers, typically 56-128). Acceptable.
- **Phase 3:** SONA adds ~200KB to the binary. The learning loop needs careful tuning to avoid adapting to noise.
- **Phase 4:** Requires significant research and training data. Not guaranteed to outperform tuned heuristics for WiFi CSI.
### Risks
- `ruvector-gnn` v2.0.5 changed constructors from panic to `Result` — any existing `crv` feature users need to handle the `Result`. Our vendored `ruvector-crv` may need updates.
- SONA's WASM support is experimental — keep it behind a feature flag until validated.
## Implementation Plan
| Phase | Scope | Effort | Priority |
|-------|-------|--------|----------|
| 1 | Bump 5 crates to v2.0.5 | 1 hour | High — free perf + security |
| 2 | Add ruvector-coherence | 1 day | Medium — improves multi-node stability |
| 3 | SONA adaptive learning | 3 days | Medium — replaces manual training workflow |
| 4 | CSI embeddings via ruvector-core | 1-2 weeks | Low — exploratory research |
## Vendor Submodule
The `vendor/ruvector` git submodule has been updated from commit `f8f2c60` (v2.0.4 era) to `51a3557` (latest `origin/main`). This provides local reference for the full upstream source when developing Phases 2-4.
## References
- Upstream repo: https://github.com/ruvnet/ruvector
- ADR-124 (Dynamic MinCut): `vendor/ruvector/docs/adr/ADR-124*.md`
- SONA docs: `vendor/ruvector/crates/sona/src/lib.rs`
- ruvector-coherence spectral: `vendor/ruvector/crates/ruvector-coherence/src/spectral.rs`
- ruvector-core embeddings: `vendor/ruvector/crates/ruvector-core/src/embeddings.rs`
+27 -10
View File
@@ -41,12 +41,14 @@ static const char *TAG = "edge_proc";
* ====================================================================== */
static edge_ring_buf_t s_ring;
static uint32_t s_ring_drops; /* Frames dropped due to full ring buffer. */
static inline bool ring_push(const uint8_t *iq, uint16_t len,
int8_t rssi, uint8_t channel)
{
uint32_t next = (s_ring.head + 1) % EDGE_RING_SLOTS;
if (next == s_ring.tail) {
s_ring_drops++;
return false; /* Full — drop frame. */
}
@@ -788,12 +790,13 @@ static void process_frame(const edge_ring_slot_t *slot)
if ((s_frame_count % 200) == 0) {
ESP_LOGI(TAG, "Vitals: br=%.1f hr=%.1f motion=%.4f pres=%s "
"fall=%s persons=%u frames=%lu",
"fall=%s persons=%u frames=%lu drops=%lu",
s_breathing_bpm, s_heartrate_bpm, s_motion_energy,
s_presence_detected ? "YES" : "no",
s_fall_detected ? "YES" : "no",
(unsigned)s_latest_pkt.n_persons,
(unsigned long)s_frame_count);
(unsigned long)s_frame_count,
(unsigned long)s_ring_drops);
}
}
@@ -831,18 +834,32 @@ static void edge_task(void *arg)
edge_ring_slot_t slot;
/* Maximum frames to process before a longer yield. On busy LANs
* (corporate networks, many APs), the ring buffer fills continuously.
* Without a batch limit the task processes frames back-to-back with
* only 1-tick yields, which on high frame rates can still starve
* IDLE1 enough to trip the 5-second task watchdog. See #266, #321. */
const uint8_t BATCH_LIMIT = 4;
while (1) {
if (ring_pop(&slot)) {
uint8_t processed = 0;
while (processed < BATCH_LIMIT && ring_pop(&slot)) {
process_frame(&slot);
/* Yield after every frame to feed the Core 1 watchdog.
* process_frame() is CPU-intensive (biquad filters, Welford stats,
* BPM estimation, multi-person vitals) and can take several ms.
* Without this yield, edge_dsp at priority 5 starves IDLE1 at
* priority 0, triggering the task watchdog. See issue #266. */
processed++;
/* 1-tick yield between frames within a batch. */
vTaskDelay(1);
}
if (processed > 0) {
/* Post-batch yield: 2 ticks (~20 ms at 100 Hz) so IDLE1 can
* run and feed the Core 1 watchdog even under sustained load.
* This is intentionally longer than the 1-tick inter-frame yield. */
vTaskDelay(2);
} else {
/* No frames available — yield briefly. */
vTaskDelay(pdMS_TO_TICKS(1));
/* No frames available — sleep one full tick.
* NOTE: pdMS_TO_TICKS(5) == 0 at 100 Hz, which would busy-spin. */
vTaskDelay(1);
}
}
}
+1 -1
View File
@@ -185,7 +185,7 @@ package-dir = {"" = "."}
[tool.setuptools.packages.find]
where = ["."]
include = ["src*"]
include = ["wifi_densepose*", "src*"]
exclude = ["tests*", "docs*", "scripts*"]
[tool.setuptools.package-data]
@@ -285,6 +285,8 @@ struct AppStateInner {
frame_history: VecDeque<Vec<f64>>,
tick: u64,
source: String,
/// Instant of the last ESP32 UDP frame received (for offline detection).
last_esp32_frame: Option<std::time::Instant>,
tx: broadcast::Sender<String>,
total_detections: u64,
start_time: std::time::Instant,
@@ -304,6 +306,8 @@ struct AppStateInner {
model_loaded: bool,
/// Smoothed person count (EMA) for hysteresis — prevents frame-to-frame jumping.
smoothed_person_score: f64,
/// Previous person count for hysteresis (asymmetric up/down thresholds).
prev_person_count: usize,
// ── Motion smoothing & adaptive baseline (ADR-047 tuning) ────────────
/// EMA-smoothed motion score (alpha ~0.15 for ~10 FPS → ~1s time constant).
smoothed_motion: f64,
@@ -362,6 +366,25 @@ struct AppStateInner {
adaptive_model: Option<adaptive_classifier::AdaptiveModel>,
}
/// If no ESP32 frame arrives within this duration, source reverts to offline.
const ESP32_OFFLINE_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(5);
impl AppStateInner {
/// Return the effective data source, accounting for ESP32 frame timeout.
/// If the source is "esp32" but no frame has arrived in 5 seconds, returns
/// "esp32:offline" so the UI can distinguish active vs stale connections.
fn effective_source(&self) -> String {
if self.source == "esp32" {
if let Some(last) = self.last_esp32_frame {
if last.elapsed() > ESP32_OFFLINE_TIMEOUT {
return "esp32:offline".to_string();
}
}
}
self.source.clone()
}
}
/// Number of frames retained in `frame_history` for temporal analysis.
/// At 500 ms ticks this covers ~50 seconds; at 100 ms ticks ~10 seconds.
const FRAME_HISTORY_CAPACITY: usize = 100;
@@ -1247,12 +1270,15 @@ async fn windows_wifi_task(state: SharedState, tick_ms: u64) {
let feat_variance = features.variance;
// Multi-person estimation with temporal smoothing (EMA α=0.15).
// Multi-person estimation with temporal smoothing (EMA α=0.10).
let raw_score = compute_person_score(&features);
s.smoothed_person_score = s.smoothed_person_score * 0.85 + raw_score * 0.15;
s.smoothed_person_score = s.smoothed_person_score * 0.90 + raw_score * 0.10;
let est_persons = if classification.presence {
score_to_person_count(s.smoothed_person_score)
let count = score_to_person_count(s.smoothed_person_score, s.prev_person_count);
s.prev_person_count = count;
count
} else {
s.prev_person_count = 0;
0
};
@@ -1377,12 +1403,15 @@ async fn windows_wifi_fallback_tick(state: &SharedState, seq: u32) {
let feat_variance = features.variance;
// Multi-person estimation with temporal smoothing.
// Multi-person estimation with temporal smoothing (EMA α=0.10).
let raw_score = compute_person_score(&features);
s.smoothed_person_score = s.smoothed_person_score * 0.85 + raw_score * 0.15;
s.smoothed_person_score = s.smoothed_person_score * 0.90 + raw_score * 0.10;
let est_persons = if classification.presence {
score_to_person_count(s.smoothed_person_score)
let count = score_to_person_count(s.smoothed_person_score, s.prev_person_count);
s.prev_person_count = count;
count
} else {
s.prev_person_count = 0;
0
};
@@ -1661,7 +1690,7 @@ async fn health(State(state): State<SharedState>) -> Json<serde_json::Value> {
let s = state.read().await;
Json(serde_json::json!({
"status": "ok",
"source": s.source,
"source": s.effective_source(),
"tick": s.tick,
"clients": s.tx.receiver_count(),
}))
@@ -1724,18 +1753,45 @@ fn compute_person_score(feat: &FeatureInfo) -> f64 {
/// Convert smoothed person score to discrete count with hysteresis.
///
/// Uses asymmetric thresholds: higher threshold to add a person, lower to remove.
/// This prevents flickering at the boundary.
fn score_to_person_count(smoothed_score: f64) -> usize {
// Thresholds chosen conservatively for single-ESP32 link:
// score > 0.50 → 2 persons (needs sustained high variance + change points)
// score > 0.80 → 3 persons (very high activity, rare with single link)
if smoothed_score > 0.80 {
3
} else if smoothed_score > 0.50 {
2
} else {
1
/// Uses asymmetric thresholds: higher threshold to *add* a person, lower to
/// *drop* one. This prevents flickering when the score hovers near a boundary
/// (the #1 user-reported issue — see #237, #249, #280, #292).
fn score_to_person_count(smoothed_score: f64, prev_count: usize) -> usize {
// Up-thresholds (must exceed to increase count):
// 1→2: 0.65 (raised from 0.50 — multipath in small rooms hit 0.50 easily)
// 2→3: 0.85 (raised from 0.80 — 3 persons needs strong sustained signal)
// Down-thresholds (must drop below to decrease count):
// 2→1: 0.45 (hysteresis gap of 0.20)
// 3→2: 0.70 (hysteresis gap of 0.15)
match prev_count {
0 | 1 => {
if smoothed_score > 0.85 {
3
} else if smoothed_score > 0.65 {
2
} else {
1
}
}
2 => {
if smoothed_score > 0.85 {
3
} else if smoothed_score < 0.45 {
1
} else {
2 // hold — within hysteresis band
}
}
_ => {
// prev_count >= 3
if smoothed_score < 0.45 {
1
} else if smoothed_score < 0.70 {
2
} else {
3 // hold
}
}
}
}
@@ -1942,7 +1998,7 @@ async fn health_ready(State(state): State<SharedState>) -> Json<serde_json::Valu
let s = state.read().await;
Json(serde_json::json!({
"status": "ready",
"source": s.source,
"source": s.effective_source(),
}))
}
@@ -1953,7 +2009,10 @@ async fn health_system(State(state): State<SharedState>) -> Json<serde_json::Val
"status": "healthy",
"components": {
"api": { "status": "healthy", "message": "Rust Axum server" },
"hardware": { "status": "healthy", "message": format!("Source: {}", s.source) },
"hardware": {
"status": if s.effective_source().ends_with(":offline") { "degraded" } else { "healthy" },
"message": format!("Source: {}", s.effective_source())
},
"pose": { "status": "healthy", "message": "WiFi-derived pose estimation" },
"stream": { "status": if s.tx.receiver_count() > 0 { "healthy" } else { "idle" },
"message": format!("{} client(s)", s.tx.receiver_count()) },
@@ -1993,7 +2052,7 @@ async fn api_info(State(state): State<SharedState>) -> Json<serde_json::Value> {
"version": env!("CARGO_PKG_VERSION"),
"environment": "production",
"backend": "rust",
"source": s.source,
"source": s.effective_source(),
"features": {
"wifi_sensing": true,
"pose_estimation": true,
@@ -2014,7 +2073,7 @@ async fn pose_current(State(state): State<SharedState>) -> Json<serde_json::Valu
"timestamp": chrono::Utc::now().timestamp_millis() as f64 / 1000.0,
"persons": persons,
"total_persons": persons.len(),
"source": s.source,
"source": s.effective_source(),
}))
}
@@ -2024,7 +2083,7 @@ async fn pose_stats(State(state): State<SharedState>) -> Json<serde_json::Value>
"total_detections": s.total_detections,
"average_confidence": 0.87,
"frames_processed": s.tick,
"source": s.source,
"source": s.effective_source(),
}))
}
@@ -2048,7 +2107,7 @@ async fn stream_status(State(state): State<SharedState>) -> Json<serde_json::Val
"active": true,
"clients": s.tx.receiver_count(),
"fps": if s.tick > 1 { 10u64 } else { 0u64 },
"source": s.source,
"source": s.effective_source(),
}))
}
@@ -2584,7 +2643,7 @@ async fn vital_signs_endpoint(State(state): State<SharedState>) -> Json<serde_js
"heartbeat_samples": hb_len,
"heartbeat_capacity": hb_cap,
},
"source": s.source,
"source": s.effective_source(),
"tick": s.tick,
}))
}
@@ -2761,6 +2820,90 @@ async fn udp_receiver_task(state: SharedState, udp_port: u16) {
})) {
let _ = s.tx.send(json);
}
// Issue #323: Also emit a sensing_update so the UI renders
// detections for ESP32 nodes running the edge DSP pipeline
// (Tier 2+). Without this, vitals arrive but the UI shows
// "no detection" because it only renders sensing_update msgs.
s.source = "esp32".to_string();
s.last_esp32_frame = Some(std::time::Instant::now());
s.tick += 1;
let tick = s.tick;
let motion_level = if vitals.motion { "present_moving" }
else if vitals.presence { "present_still" }
else { "absent" };
let motion_score = if vitals.motion { 0.8 }
else if vitals.presence { 0.3 }
else { 0.05 };
let est_persons = if vitals.presence {
(vitals.n_persons as usize).max(1)
} else {
0
};
let features = FeatureInfo {
mean_rssi: vitals.rssi as f64,
variance: vitals.motion_energy as f64,
motion_band_power: vitals.motion_energy as f64,
breathing_band_power: if vitals.presence { 0.5 } else { 0.0 },
dominant_freq_hz: vitals.breathing_rate_bpm / 60.0,
change_points: 0,
spectral_power: vitals.motion_energy as f64,
};
let classification = ClassificationInfo {
motion_level: motion_level.to_string(),
presence: vitals.presence,
confidence: vitals.presence_score as f64,
};
let signal_field = generate_signal_field(
vitals.rssi as f64, motion_score, vitals.breathing_rate_bpm / 60.0,
(vitals.presence_score as f64).min(1.0), &[],
);
let mut update = SensingUpdate {
msg_type: "sensing_update".to_string(),
timestamp: chrono::Utc::now().timestamp_millis() as f64 / 1000.0,
source: "esp32".to_string(),
tick,
nodes: vec![NodeInfo {
node_id: vitals.node_id,
rssi_dbm: vitals.rssi as f64,
position: [2.0, 0.0, 1.5],
amplitude: vec![],
subcarrier_count: 0,
}],
features: features.clone(),
classification,
signal_field,
vital_signs: Some(VitalSigns {
breathing_rate_bpm: if vitals.breathing_rate_bpm > 0.0 { Some(vitals.breathing_rate_bpm) } else { None },
heart_rate_bpm: if vitals.heartrate_bpm > 0.0 { Some(vitals.heartrate_bpm) } else { None },
breathing_confidence: if vitals.presence { 0.7 } else { 0.0 },
heartbeat_confidence: if vitals.presence { 0.7 } else { 0.0 },
signal_quality: vitals.presence_score as f64,
}),
enhanced_motion: None,
enhanced_breathing: None,
posture: None,
signal_quality_score: None,
quality_verdict: None,
bssid_count: None,
pose_keypoints: None,
model_status: None,
persons: None,
estimated_persons: if est_persons > 0 { Some(est_persons) } else { None },
};
let persons = derive_pose_from_sensing(&update);
if !persons.is_empty() {
update.persons = Some(persons);
}
if let Ok(json) = serde_json::to_string(&update) {
let _ = s.tx.send(json);
}
s.latest_update = Some(update);
s.edge_vitals = Some(vitals);
continue;
}
@@ -2790,6 +2933,7 @@ async fn udp_receiver_task(state: SharedState, udp_port: u16) {
let mut s = state.write().await;
s.source = "esp32".to_string();
s.last_esp32_frame = Some(std::time::Instant::now());
// Append current amplitudes to history before extracting features so
// that temporal analysis includes the most recent frame.
@@ -2824,12 +2968,15 @@ async fn udp_receiver_task(state: SharedState, udp_port: u16) {
let vitals = smooth_vitals(&mut s, &raw_vitals);
s.latest_vitals = vitals.clone();
// Multi-person estimation with temporal smoothing.
// Multi-person estimation with temporal smoothing (EMA α=0.10).
let raw_score = compute_person_score(&features);
s.smoothed_person_score = s.smoothed_person_score * 0.85 + raw_score * 0.15;
s.smoothed_person_score = s.smoothed_person_score * 0.90 + raw_score * 0.10;
let est_persons = if classification.presence {
score_to_person_count(s.smoothed_person_score)
let count = score_to_person_count(s.smoothed_person_score, s.prev_person_count);
s.prev_person_count = count;
count
} else {
s.prev_person_count = 0;
0
};
@@ -2929,12 +3076,15 @@ async fn simulated_data_task(state: SharedState, tick_ms: u64) {
let frame_amplitudes = frame.amplitudes.clone();
let frame_n_sub = frame.n_subcarriers;
// Multi-person estimation with temporal smoothing.
// Multi-person estimation with temporal smoothing (EMA α=0.10).
let raw_score = compute_person_score(&features);
s.smoothed_person_score = s.smoothed_person_score * 0.85 + raw_score * 0.15;
s.smoothed_person_score = s.smoothed_person_score * 0.90 + raw_score * 0.10;
let est_persons = if classification.presence {
score_to_person_count(s.smoothed_person_score)
let count = score_to_person_count(s.smoothed_person_score, s.prev_person_count);
s.prev_person_count = count;
count
} else {
s.prev_person_count = 0;
0
};
@@ -3566,6 +3716,7 @@ async fn main() {
frame_history: VecDeque::new(),
tick: 0,
source: source.into(),
last_esp32_frame: None,
tx,
total_detections: 0,
start_time: std::time::Instant::now(),
@@ -3577,6 +3728,7 @@ async fn main() {
active_sona_profile: None,
model_loaded,
smoothed_person_score: 0.0,
prev_person_count: 0,
smoothed_motion: 0.0,
current_motion_level: "absent".to_string(),
debounce_counter: 0,
@@ -3739,7 +3891,7 @@ async fn main() {
"WiFi DensePose sensing model state",
);
builder.add_metadata(&serde_json::json!({
"source": s.source,
"source": s.effective_source(),
"total_ticks": s.tick,
"total_detections": s.total_detections,
"uptime_secs": s.start_time.elapsed().as_secs(),
+137
View File
@@ -0,0 +1,137 @@
"""
WiFi-DensePose WiFi-based human pose estimation using CSI data.
Usage:
from wifi_densepose import WiFiDensePose
system = WiFiDensePose()
system.start()
poses = system.get_latest_poses()
system.stop()
"""
__version__ = "1.2.0"
import sys
import os
import logging
logger = logging.getLogger(__name__)
# Allow importing the v1 src package when installed from the repo
_v1_src = os.path.join(os.path.dirname(os.path.dirname(__file__)), "v1")
if os.path.isdir(_v1_src) and _v1_src not in sys.path:
sys.path.insert(0, _v1_src)
class WiFiDensePose:
"""High-level facade for the WiFi-DensePose sensing system.
This is the primary entry point documented in the README Quick Start.
It wraps the underlying ServiceOrchestrator and exposes a simple
start / get_latest_poses / stop interface.
"""
def __init__(self, host: str = "0.0.0.0", port: int = 3000, **kwargs):
self.host = host
self.port = port
self._config = kwargs
self._orchestrator = None
self._server_task = None
self._poses = []
self._running = False
# ------------------------------------------------------------------
# Public API (matches README Quick Start)
# ------------------------------------------------------------------
def start(self):
"""Start the sensing system (blocking until ready)."""
import asyncio
loop = _get_or_create_event_loop()
loop.run_until_complete(self._async_start())
async def _async_start(self):
try:
from src.config.settings import get_settings
from src.services.orchestrator import ServiceOrchestrator
settings = get_settings()
self._orchestrator = ServiceOrchestrator(settings)
await self._orchestrator.initialize()
await self._orchestrator.start()
self._running = True
logger.info("WiFiDensePose system started on %s:%s", self.host, self.port)
except ImportError:
raise ImportError(
"Core dependencies not found. Make sure you installed "
"from the repository root:\n"
" cd wifi-densepose && pip install -e .\n"
"Or install the v1 package:\n"
" cd wifi-densepose/v1 && pip install -e ."
)
def stop(self):
"""Stop the sensing system."""
import asyncio
if self._orchestrator is not None:
loop = _get_or_create_event_loop()
loop.run_until_complete(self._orchestrator.shutdown())
self._running = False
logger.info("WiFiDensePose system stopped")
def get_latest_poses(self):
"""Return the most recent list of detected pose dicts."""
if self._orchestrator is None:
return []
try:
import asyncio
loop = _get_or_create_event_loop()
return loop.run_until_complete(self._fetch_poses())
except Exception:
return []
async def _fetch_poses(self):
try:
pose_svc = self._orchestrator.pose_service
if pose_svc and hasattr(pose_svc, "get_latest"):
return await pose_svc.get_latest()
except Exception:
pass
return []
# ------------------------------------------------------------------
# Context-manager support
# ------------------------------------------------------------------
def __enter__(self):
self.start()
return self
def __exit__(self, *exc):
self.stop()
# ------------------------------------------------------------------
# Convenience re-exports
# ------------------------------------------------------------------
@staticmethod
def version():
return __version__
def _get_or_create_event_loop():
import asyncio
try:
return asyncio.get_event_loop()
except RuntimeError:
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
return loop
__all__ = ["WiFiDensePose", "__version__"]