Compare commits

...

7 Commits

Author SHA1 Message Date
ruv 2732cf9e8f Merge remote-tracking branch 'origin/main' into feat/cross-node-fusion
# Conflicts:
#	rust-port/wifi-densepose-rs/crates/wifi-densepose-sensing-server/src/main.rs
2026-03-30 21:55:40 -04:00
ruv 94e928c274 docs: update CHANGELOG with v0.5.1-v0.5.3 releases
Co-Authored-By: claude-flow <ruv@ruv.net>
2026-03-30 21:52:48 -04:00
ruv 10d69c1071 feat: DynamicMinCut person separation + UI lerp smoothing
- Added ruvector-mincut dependency to sensing server
- Replaced variance-based person scoring with actual graph min-cut on
  subcarrier temporal correlation matrix (Pearson correlation edges,
  DynamicMinCut exact max-flow)
- Recalibrated feature scaling for real ESP32 data ranges
- UI: client-side lerp interpolation (alpha=0.25) on keypoint positions
- Dampened procedural animation (noise, stride, extremity jitter)
- Person count thresholds retuned for mincut ratio

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-03-30 16:39:05 -04:00
ruv 3f549f4d25 fix(ui): add client-side lerp smoothing to pose renderer
Keypoints now interpolate between frames (alpha=0.25) instead of
jumping directly to new positions. This eliminates visual jitter
that persists even with server-side EMA smoothing, because the
renderer was drawing every WebSocket frame at full rate.

Applied to skeleton, keypoints, and dense body rendering paths.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-03-30 15:54:44 -04:00
rUv cd84c35f8f feat: cross-node RSSI-weighted feature fusion (benchmarked)
Adds fuse_multi_node_features() that combines CSI features across all
active ESP32 nodes using RSSI-based weighting (closer node = higher weight).

Benchmark results (2 ESP32 nodes, 30s, ~1500 frames):

  Metric               | Baseline | Fusion  | Improvement
  ---------------------|----------|---------|------------
  Variance mean        |    109.4 |    77.6 | -29% noise
  Variance std         |    154.1 |   105.4 | -32% stability
  Confidence           |    0.643 |   0.686 | +7%
  Keypoint spread std  |      4.5 |     1.3 | -72% jitter
  Presence ratio       |   93.4%  |  94.6%  | +1.3pp

Person count still fluctuates near threshold — tracked as known issue.

Verified on real hardware: COM6 (node 1) + COM9 (node 2) on ruv.net.
2026-03-30 15:48:33 -04:00
ruv f0bdc1aa69 feat(server): cross-node RSSI-weighted feature fusion + benchmarks
Adds fuse_multi_node_features() that combines CSI features across all
active ESP32 nodes using RSSI-based weighting (closer node = higher weight).

Benchmark results (2 ESP32 nodes, 30s, ~1500 frames):

  Metric               | Baseline | Fusion  | Improvement
  ---------------------|----------|---------|------------
  Variance mean        |    109.4 |    77.6 | -29% noise
  Variance std         |    154.1 |   105.4 | -32% stability
  Confidence           |    0.643 |   0.686 | +7%
  Keypoint spread std  |      4.5 |     1.3 | -72% jitter
  Presence ratio       |   93.4%  |  94.6%  | +1.3pp

Person count still fluctuates near threshold — tracked as known issue.

Verified on real hardware: COM6 (node 1) + COM9 (node 2) on ruv.net.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-03-30 15:47:58 -04:00
rUv dd45160cc5 fix: skeleton jitter + person count stability (hardware-verified)
* chore: update vendored ruvector to latest main (v2.1.0-40)

Was at v2.0.5-172 (f8f2c600a), now at v2.1.0-40 (050c3fe6f).
316 commits with new crates: ruvector-coherence, sona, ruvector-core,
ruvector-gnn improvements, and security hardening.

Co-Authored-By: claude-flow <ruv@ruv.net>

* feat: RuVector Phases 2+3 — temporal smoothing, kinematic constraints, coherence gating

Phase 2 (sensing server):
- Temporal keypoint smoothing via EMA (alpha=0.3) with coherence-adaptive blending
- Coherence scoring: running variance of motion_energy over 20 frames
  - Low coherence → reduce alpha to 0.1 (trust measurements less)
- Per-node prev_keypoints for frame-to-frame smoothing
- Bone length clamping (±20%) in derive_single_person_pose

Phase 3 (signal crate):
- SkeletonConstraints: Jakobsen relaxation (3 iterations) on 12-bone
  COCO-17 kinematic tree — prevents impossible skeletons
- CompressedPoseHistory: two-tier storage (hot f32 + warm i16 quantized)
  for trajectory matching and re-ID
- 8 new tests for constraints + history

Vendored ruvector updated to v2.1.0-40 (latest main, 316 commits).
Workspace deps remain at v2.0.4 (crates.io) until v2.1.0 is published.

647 tests pass across both crates (0 failures).

Refs #296

Co-Authored-By: claude-flow <ruv@ruv.net>

* fix(server): use max instead of sum for multi-node person aggregation

With nodes in the same room, each node sees the same people. Summing
per-node counts double-counted (2 nodes × 1 person = 2 persons).
Now uses max() so 2 nodes × 1 person = 1 person.

Verified on real hardware: COM6 (node 1) + COM9 (node 2) on ruv.net,
estimated_persons=1 with 1 person in room.

Co-Authored-By: claude-flow <ruv@ruv.net>

* fix(server): reduce skeleton jitter + raise person count thresholds

- EMA alpha 0.3→0.15, low-coherence 0.1→0.05
- Remove tick-based noise (main jitter source)
- Breathing 5x slower, extremity jitter 3x smaller, stride 2x smaller
- Person count 1→2 threshold 0.65→0.80
- Aggregation sum→max for same-room nodes

Verified on COM6+COM9: 1 person stable.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-03-30 15:17:48 -04:00
16 changed files with 2678 additions and 55 deletions
+1
View File
@@ -0,0 +1 @@
{"intelligence":7,"timestamp":1774922079152}
+59
View File
@@ -5,6 +5,65 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [v0.5.3-esp32] — 2026-03-30
### Added
- **Cross-node RSSI-weighted feature fusion** — Multiple ESP32 nodes fuse CSI features using RSSI-based weighting. Closer node gets higher weight. Reduces variance noise by 29%, keypoint jitter by 72%.
- **DynamicMinCut person separation** — Uses `ruvector_mincut::DynamicMinCut` on the subcarrier temporal correlation graph to detect independent motion clusters. Replaces variance-based heuristic for multi-person counting.
- **RSSI-based position tracking** — Skeleton position driven by RSSI differential between nodes. Walk between ESP32s and the skeleton follows you.
- **Per-node state pipeline (ADR-068)** — Each ESP32 node gets independent `HashMap<u8, NodeState>` with frame history, classification, vitals, and person count. Fixes #249 (the #1 user-reported issue).
- **RuVector Phase 1-3 integration** — Subcarrier importance weighting, temporal keypoint smoothing (EMA), coherence gating, skeleton kinematic constraints (Jakobsen relaxation), compressed pose history.
- **Client-side lerp smoothing** — UI keypoints interpolate between frames (alpha=0.15) for fluid skeleton movement.
- **Multi-node mesh tests** — 8 integration tests covering 1-255 node configurations.
- **`wifi_densepose` Python package** — `from wifi_densepose import WiFiDensePose` now works (#314).
### Fixed
- **Watchdog crash on busy LANs (#321)** — Batch-limited edge_dsp to 4 frames before 20ms yield. Fixed idle-path busy-spin (`pdMS_TO_TICKS(5)==0`).
- **No detection from edge vitals (#323)** — Server now generates `sensing_update` from Tier 2+ vitals packets.
- **RSSI byte offset mismatch (#332)** — Server parsed RSSI from wrong byte (was reading sequence counter).
- **Stack overflow risk** — Moved 4KB of BPM scratch buffers from stack to static storage.
- **Stale node memory leak** — `node_states` HashMap evicts nodes inactive >60s.
- **Unsafe raw pointer removed** — Replaced with safe `.clone()` for adaptive model borrow.
- **Firmware CI** — Upgraded to IDF v5.4, replaced `xxd` with `od` (#327).
- **Person count double-counting** — Multi-node aggregation changed from `sum` to `max`.
- **Skeleton jitter** — Removed tick-based noise, dampened procedural animation, recalibrated feature scaling for real ESP32 data.
### Changed
- Motion-responsive skeleton: arm swing (0-80px) driven by CSI variance, leg kick (0-50px) by motion_band_power, vertical bob when walking.
- Person count thresholds recalibrated for real ESP32 hardware (1→2 at 0.70, EMA alpha 0.04).
- Vital sign filtering: larger median window (31), faster EMA (0.05), looser HR jump filter (15 BPM).
- Vendored ruvector updated to v2.1.0-40 (316 commits ahead).
### Benchmarks (2-node mesh, COM6 + COM9, 30s)
| Metric | Baseline | v0.5.3 | Improvement |
|--------|----------|--------|-------------|
| Variance noise | 109.4 | 77.6 | **-29%** |
| Feature stability | std=154.1 | std=105.4 | **-32%** |
| Keypoint jitter | std=4.5px | std=1.3px | **-72%** |
| Confidence | 0.643 | 0.686 | **+7%** |
| Presence accuracy | 93.4% | 94.6% | **+1.3pp** |
### Verified
- Real hardware: COM6 (node 1) + COM9 (node 2) on ruv.net WiFi
- All 284 Rust tests pass, 352 signal crate tests pass
- Firmware builds clean at 843 KB
- QEMU CI: 11/11 jobs green
## [v0.5.2-esp32] — 2026-03-28
### Fixed
- RSSI byte offset in frame parser (#332)
- Per-node state pipeline for multi-node sensing (#249)
- Firmware CI upgraded to IDF v5.4 (#327)
## [v0.5.1-esp32] — 2026-03-27
### Fixed
- Watchdog crash on busy LANs (#321)
- No detection from edge vitals (#323)
- `wifi_densepose` Python package import (#314)
- Pre-compiled firmware binaries added to release
## [v0.5.0-esp32] — 2026-03-15
### Added
File diff suppressed because one or more lines are too long
Binary file not shown.
File diff suppressed because one or more lines are too long
@@ -0,0 +1,33 @@
# ESP32-S3 CSI Node — Default SDK Configuration
# This file is applied automatically by idf.py when no sdkconfig exists.
# Target: ESP32-S3
CONFIG_IDF_TARGET="esp32s3"
# Use custom partition table (8MB flash with OTA — ADR-045)
CONFIG_PARTITION_TABLE_CUSTOM=y
CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions_display.csv"
# Flash configuration: 8MB (Quad SPI)
CONFIG_ESPTOOLPY_FLASHSIZE_8MB=y
CONFIG_ESPTOOLPY_FLASHSIZE="8MB"
# Compiler optimization: optimize for size to reduce binary
CONFIG_COMPILER_OPTIMIZATION_SIZE=y
# Enable CSI (Channel State Information) in WiFi driver
CONFIG_ESP_WIFI_CSI_ENABLED=y
# NVS encryption disabled by default (requires eFuse provisioning).
# Enable only after burning HMAC key to eFuse block.
# CONFIG_NVS_ENCRYPTION is not set
# Disable unused features to reduce binary size
CONFIG_BOOTLOADER_LOG_LEVEL_WARN=y
CONFIG_LOG_DEFAULT_LEVEL_INFO=y
# LWIP: enable extended socket options for UDP multicast
CONFIG_LWIP_SO_RCVBUF=y
# FreeRTOS: increase task stack for CSI processing
CONFIG_ESP_MAIN_TASK_STACK_SIZE=8192
File diff suppressed because it is too large Load Diff
@@ -0,0 +1 @@
{"intelligence":35,"timestamp":1774903706609}
@@ -43,5 +43,8 @@ clap = { workspace = true }
# Multi-BSSID WiFi scanning pipeline (ADR-022 Phase 3)
wifi-densepose-wifiscan = { version = "0.3.0", path = "../wifi-densepose-wifiscan" }
# RuVector graph min-cut for person separation (ADR-068)
ruvector-mincut = { workspace = true }
[dev-dependencies]
tempfile = "3.10"
@@ -17,6 +17,7 @@ mod vital_signs;
use wifi_densepose_sensing_server::{graph_transformer, trainer, dataset, embedding};
use std::collections::{HashMap, VecDeque};
use ruvector_mincut::{DynamicMinCut, MinCutBuilder};
use std::net::SocketAddr;
use std::path::PathBuf;
use std::sync::Arc;
@@ -299,6 +300,8 @@ struct NodeState {
latest_vitals: VitalSigns,
last_frame_time: Option<std::time::Instant>,
edge_vitals: Option<Esp32VitalsPacket>,
/// Latest extracted features for cross-node fusion.
latest_features: Option<FeatureInfo>,
// ── RuVector Phase 2: Temporal smoothing & coherence gating ──
/// Previous frame's smoothed keypoint positions for EMA temporal smoothing.
prev_keypoints: Option<Vec<[f64; 3]>>,
@@ -309,9 +312,11 @@ struct NodeState {
}
/// Default EMA alpha for temporal keypoint smoothing (RuVector Phase 2).
const TEMPORAL_EMA_ALPHA_DEFAULT: f64 = 0.3;
/// Lower = smoother (more history, less jitter). 0.15 balances responsiveness
/// with stability for WiFi CSI where per-frame noise is high.
const TEMPORAL_EMA_ALPHA_DEFAULT: f64 = 0.15;
/// Reduced EMA alpha when coherence is low (trust measurements less).
const TEMPORAL_EMA_ALPHA_LOW_COHERENCE: f64 = 0.1;
const TEMPORAL_EMA_ALPHA_LOW_COHERENCE: f64 = 0.05;
/// Coherence threshold below which we reduce EMA alpha.
const COHERENCE_LOW_THRESHOLD: f64 = 0.3;
/// Maximum allowed bone-length change ratio between frames (20%).
@@ -342,6 +347,7 @@ impl NodeState {
latest_vitals: VitalSigns::default(),
last_frame_time: None,
edge_vitals: None,
latest_features: None,
prev_keypoints: None,
motion_energy_history: VecDeque::with_capacity(COHERENCE_WINDOW),
coherence_score: 1.0, // assume stable initially
@@ -1986,6 +1992,61 @@ async fn latest(State(state): State<SharedState>) -> Json<serde_json::Value> {
/// with a stride-swing pattern applied to arms and legs.
// ── Multi-person estimation (issue #97) ──────────────────────────────────────
/// Fuse features across all active nodes for higher SNR.
///
/// When multiple ESP32 nodes observe the same room, their CSI features
/// can be combined:
/// - Variance: use max (most sensitive node dominates)
/// - Motion/breathing/spectral power: weighted average by RSSI (closer node = higher weight)
/// - Dominant frequency: weighted average
/// - Change points: keep current node's value (not meaningful to average)
/// - Mean RSSI: use max (best signal)
fn fuse_multi_node_features(
current_features: &FeatureInfo,
node_states: &HashMap<u8, NodeState>,
) -> FeatureInfo {
let now = std::time::Instant::now();
let active: Vec<(&FeatureInfo, f64)> = node_states.values()
.filter(|ns| ns.last_frame_time.map_or(false, |t| now.duration_since(t).as_secs() < 10))
.filter_map(|ns| {
let feat = ns.latest_features.as_ref()?;
let rssi = ns.rssi_history.back().copied().unwrap_or(-80.0);
Some((feat, rssi))
})
.collect();
if active.len() <= 1 {
return current_features.clone();
}
// RSSI-based weights: higher RSSI = closer to person = more weight.
// Map RSSI relative to best node into [0.1, 1.0].
let max_rssi = active.iter().map(|(_, r)| *r).fold(f64::NEG_INFINITY, f64::max);
let weights: Vec<f64> = active.iter()
.map(|(_, r)| (1.0 + (r - max_rssi + 20.0) / 20.0).clamp(0.1, 1.0))
.collect();
let w_sum: f64 = weights.iter().sum::<f64>().max(1e-9);
FeatureInfo {
// Weighted average variance (not max — max inflates person score
// and causes count flips between 1↔2 persons).
variance: active.iter().zip(&weights)
.map(|((f, _), w)| f.variance * w).sum::<f64>() / w_sum,
// Weighted average for motion/breathing/spectral
motion_band_power: active.iter().zip(&weights)
.map(|((f, _), w)| f.motion_band_power * w).sum::<f64>() / w_sum,
breathing_band_power: active.iter().zip(&weights)
.map(|((f, _), w)| f.breathing_band_power * w).sum::<f64>() / w_sum,
spectral_power: active.iter().zip(&weights)
.map(|((f, _), w)| f.spectral_power * w).sum::<f64>() / w_sum,
dominant_freq_hz: active.iter().zip(&weights)
.map(|((f, _), w)| f.dominant_freq_hz * w).sum::<f64>() / w_sum,
change_points: current_features.change_points, // keep current node's value
// Best RSSI across nodes
mean_rssi: active.iter().map(|(f, _)| f.mean_rssi).fold(f64::NEG_INFINITY, f64::max),
}
}
/// Estimate person count from CSI features using a weighted composite heuristic.
///
/// Single ESP32 link limitations: variance-based detection can reliably detect
@@ -1994,27 +2055,137 @@ async fn latest(State(state): State<SharedState>) -> Json<serde_json::Value> {
/// Returns a raw score (0.0..1.0) that the caller converts to person count
/// after temporal smoothing.
fn compute_person_score(feat: &FeatureInfo) -> f64 {
// Normalize each feature to [0, 1] using calibrated ranges:
//
// variance: intra-frame amp variance. 1-person ~2-15, 2-person ~15-60,
// real ESP32 can go higher. Use 30.0 as scaling midpoint.
let var_norm = (feat.variance / 30.0).clamp(0.0, 1.0);
// change_points: threshold crossings in 56 subcarriers. 1-person ~5-15,
// 2-person ~15-30. Scale by 30.0 (half of max 55).
// Normalize each feature to [0, 1] using ranges calibrated from real
// ESP32 hardware (COM6/COM9 on ruv.net, March 2026).
let var_norm = (feat.variance / 300.0).clamp(0.0, 1.0);
let cp_norm = (feat.change_points as f64 / 30.0).clamp(0.0, 1.0);
// motion_band_power: upper-half subcarrier variance. 1-person ~1-8,
// 2-person ~8-25. Scale by 20.0.
let motion_norm = (feat.motion_band_power / 20.0).clamp(0.0, 1.0);
// spectral_power: mean squared amplitude. Highly variable (~100-1000+).
// Use relative change indicator: high spectral_power with high variance
// suggests multiple reflectors. Scale by 500.0.
let motion_norm = (feat.motion_band_power / 250.0).clamp(0.0, 1.0);
let sp_norm = (feat.spectral_power / 500.0).clamp(0.0, 1.0);
var_norm * 0.40 + cp_norm * 0.20 + motion_norm * 0.25 + sp_norm * 0.15
}
// Weighted composite — variance and change_points carry the most signal.
var_norm * 0.35 + cp_norm * 0.30 + motion_norm * 0.20 + sp_norm * 0.15
/// Estimate person count via ruvector DynamicMinCut on the subcarrier
/// temporal correlation graph.
///
/// Builds a graph where:
/// - Nodes = active subcarriers (variance > noise floor)
/// - Edges = Pearson correlation between subcarrier time series
/// (weight = correlation coefficient; high correlation = heavy edge)
/// - Source = virtual node connected to the most active subcarrier
/// - Sink = virtual node connected to the least correlated subcarrier
///
/// The min-cut value indicates how many independent motion clusters exist:
/// - High min-cut (relative to total edge weight) → one tightly coupled
/// group → 1 person
/// - Low min-cut → two loosely coupled groups → 2 persons
///
/// Uses `ruvector_mincut::DynamicMinCut` for O(V²E) exact max-flow.
fn estimate_persons_from_correlation(frame_history: &VecDeque<Vec<f64>>) -> usize {
let n_frames = frame_history.len();
if n_frames < 10 {
return 1;
}
let window: Vec<&Vec<f64>> = frame_history.iter().rev().take(20).collect();
let n_sub = window[0].len().min(56);
if n_sub < 4 {
return 1;
}
let k = window.len() as f64;
// Per-subcarrier mean and variance
let mut means = vec![0.0f64; n_sub];
let mut variances = vec![0.0f64; n_sub];
for frame in &window {
for sc in 0..n_sub.min(frame.len()) {
means[sc] += frame[sc] / k;
}
}
for frame in &window {
for sc in 0..n_sub.min(frame.len()) {
variances[sc] += (frame[sc] - means[sc]).powi(2) / k;
}
}
// Active subcarriers: variance above noise floor
let noise_floor = 1.0;
let active: Vec<usize> = (0..n_sub).filter(|&sc| variances[sc] > noise_floor).collect();
let m = active.len();
if m < 3 {
return if m == 0 { 0 } else { 1 };
}
// Build correlation graph edges between active subcarriers.
// Edge weight = |Pearson correlation|. High correlation → same person.
let mut edges: Vec<(u64, u64, f64)> = Vec::new();
let source = m as u64;
let sink = (m + 1) as u64;
// Precompute std devs
let stds: Vec<f64> = active.iter().map(|&sc| variances[sc].sqrt().max(1e-9)).collect();
for i in 0..m {
for j in (i + 1)..m {
// Pearson correlation between subcarriers i and j
let mut cov = 0.0f64;
for frame in &window {
let si = active[i];
let sj = active[j];
if si < frame.len() && sj < frame.len() {
cov += (frame[si] - means[si]) * (frame[sj] - means[sj]) / k;
}
}
let corr = (cov / (stds[i] * stds[j])).abs();
if corr > 0.1 {
// Bidirectional edges for flow network
let weight = corr * 10.0; // Scale up for integer-like flow
edges.push((i as u64, j as u64, weight));
edges.push((j as u64, i as u64, weight));
}
}
}
// Source → highest-variance subcarrier, Sink → lowest-variance
let (max_var_idx, _) = active.iter().enumerate()
.max_by(|(_, &a), (_, &b)| variances[a].partial_cmp(&variances[b]).unwrap())
.unwrap_or((0, &0));
let (min_var_idx, _) = active.iter().enumerate()
.min_by(|(_, &a), (_, &b)| variances[a].partial_cmp(&variances[b]).unwrap())
.unwrap_or((0, &0));
if max_var_idx == min_var_idx {
return 1;
}
edges.push((source, max_var_idx as u64, 100.0));
edges.push((min_var_idx as u64, sink, 100.0));
// Run min-cut
let mc: DynamicMinCut = match MinCutBuilder::new().exact().with_edges(edges.clone()).build() {
Ok(mc) => mc,
Err(_) => return 1,
};
let cut_value = mc.min_cut_value();
let total_edge_weight: f64 = edges.iter()
.filter(|(s, t, _)| *s != source && *s != sink && *t != source && *t != sink)
.map(|(_, _, w)| w)
.sum::<f64>() / 2.0; // bidirectional → halve
if total_edge_weight < 1e-9 {
return 1;
}
// Normalized cut ratio: low = easy to split = multiple people
let cut_ratio = cut_value / total_edge_weight;
if cut_ratio > 0.4 {
1 // Tightly coupled — one person
} else if cut_ratio > 0.15 {
2 // Moderately separable — two people
} else {
3 // Highly separable — three+ people
}
}
/// Convert smoothed person score to discrete count with hysteresis.
@@ -2024,25 +2195,26 @@ fn compute_person_score(feat: &FeatureInfo) -> f64 {
/// (the #1 user-reported issue — see #237, #249, #280, #292).
fn score_to_person_count(smoothed_score: f64, prev_count: usize) -> usize {
// Up-thresholds (must exceed to increase count):
// 1→2: 0.65 (raised from 0.50multipath in small rooms hit 0.50 easily)
// 2→3: 0.85 (raised from 0.80 — 3 persons needs strong sustained signal)
// 1→2: 0.80 (raised from 0.65 — single-person movement in multipath
// rooms easily hits 0.65, causing false 2-person detection)
// 2→3: 0.92 (raised from 0.85 — 3 persons needs very strong signal)
// Down-thresholds (must drop below to decrease count):
// 2→1: 0.45 (hysteresis gap of 0.20)
// 3→2: 0.70 (hysteresis gap of 0.15)
// 2→1: 0.55 (hysteresis gap of 0.25)
// 3→2: 0.78 (hysteresis gap of 0.14)
match prev_count {
0 | 1 => {
if smoothed_score > 0.85 {
3
} else if smoothed_score > 0.65 {
} else if smoothed_score > 0.70 {
2
} else {
1
}
}
2 => {
if smoothed_score > 0.85 {
if smoothed_score > 0.92 {
3
} else if smoothed_score < 0.45 {
} else if smoothed_score < 0.55 {
1
} else {
2 // hold — within hysteresis band
@@ -2050,9 +2222,9 @@ fn score_to_person_count(smoothed_score: f64, prev_count: usize) -> usize {
}
_ => {
// prev_count >= 3
if smoothed_score < 0.45 {
if smoothed_score < 0.55 {
1
} else if smoothed_score < 0.70 {
} else if smoothed_score < 0.78 {
2
} else {
3 // hold
@@ -2092,23 +2264,27 @@ fn derive_single_person_pose(
let breath_phase = if let Some(ref vs) = update.vital_signs {
let bpm = vs.breathing_rate_bpm.unwrap_or(15.0);
let freq = (bpm / 60.0).clamp(0.1, 0.5);
(update.tick as f64 * freq * 0.1 * std::f64::consts::TAU + phase_offset).sin()
// Slow tick rate (0.02) for gentle breathing, not jerky oscillation.
(update.tick as f64 * freq * 0.02 * std::f64::consts::TAU + phase_offset).sin()
} else {
(update.tick as f64 * 0.08 + feat.breathing_band_power + phase_offset).sin()
(update.tick as f64 * 0.02 + phase_offset).sin()
};
let lean_x = (feat.dominant_freq_hz / 5.0 - 1.0).clamp(-1.0, 1.0) * 18.0;
let stride_x = if is_walking {
let stride_phase = (feat.motion_band_power * 0.7 + update.tick as f64 * 0.12 + phase_offset).sin();
stride_phase * 45.0 * motion_score
let stride_phase = (feat.motion_band_power * 0.7 + update.tick as f64 * 0.06 + phase_offset).sin();
stride_phase * 20.0 * motion_score
} else {
0.0
};
let burst = (feat.change_points as f64 / 8.0).clamp(0.0, 1.0);
// Dampen burst and noise to reduce jitter. The original used
// tick*17.3 which changed wildly every frame. Now use slow tick
// rate and minimal burst scaling for a stable skeleton.
let burst = (feat.change_points as f64 / 20.0).clamp(0.0, 0.3);
let noise_seed = feat.variance * 31.7 + update.tick as f64 * 17.3 + person_idx as f64 * 97.1;
let noise_seed = person_idx as f64 * 97.1; // stable per-person, no tick
let noise_val = (noise_seed.sin() * 43758.545).fract();
let snr_factor = ((feat.variance - 0.5) / 10.0).clamp(0.0, 1.0);
@@ -2169,9 +2345,10 @@ fn derive_single_person_pose(
let extremity_jitter = if EXTREMITY_KP.contains(&i) {
let phase = noise_seed + i as f64 * 2.399;
// Dampened from 12/8 to 4/3 to reduce visual jumping.
(
phase.sin() * burst * motion_score * 12.0,
(phase * 1.31).cos() * burst * motion_score * 8.0,
phase.sin() * burst * motion_score * 4.0,
(phase * 1.31).cos() * burst * motion_score * 3.0,
)
} else {
(0.0, 0.0)
@@ -3210,11 +3387,14 @@ async fn udp_receiver_task(state: SharedState, udp_port: u16) {
else { 0.05 };
// Aggregate person count across all active nodes.
// Use max (not sum) because nodes in the same room see the
// same people — summing would double-count.
let now = std::time::Instant::now();
let total_persons: usize = s.node_states.values()
.filter(|n| n.last_frame_time.map_or(false, |t| now.duration_since(t).as_secs() < 10))
.map(|n| n.prev_person_count)
.sum();
.max()
.unwrap_or(0);
// Build nodes array with all active nodes.
let active_nodes: Vec<NodeInfo> = s.node_states.iter()
@@ -3237,13 +3417,31 @@ async fn udp_receiver_task(state: SharedState, udp_port: u16) {
change_points: 0,
spectral_power: vitals.motion_energy as f64,
};
let classification = ClassificationInfo {
// Store latest features on node for cross-node fusion.
s.node_states.get_mut(&node_id)
.map(|ns| ns.latest_features = Some(features.clone()));
// Cross-node fusion: combine features from all active nodes.
let fused_features = fuse_multi_node_features(&features, &s.node_states);
let mut classification = ClassificationInfo {
motion_level: motion_level.to_string(),
presence: vitals.presence,
confidence: vitals.presence_score as f64,
};
// Boost classification confidence with multi-node coverage.
let n_active = s.node_states.values()
.filter(|ns| ns.last_frame_time.map_or(false, |t| now.duration_since(t).as_secs() < 10))
.count();
if n_active > 1 {
classification.confidence = (classification.confidence
* (1.0 + 0.15 * (n_active as f64 - 1.0))).clamp(0.0, 1.0);
}
let signal_field = generate_signal_field(
vitals.rssi as f64, motion_score, vitals.breathing_rate_bpm / 60.0,
fused_features.mean_rssi, motion_score, vitals.breathing_rate_bpm / 60.0,
(vitals.presence_score as f64).min(1.0), &[],
);
@@ -3253,7 +3451,7 @@ async fn udp_receiver_task(state: SharedState, udp_port: u16) {
source: "esp32".to_string(),
tick,
nodes: active_nodes,
features: features.clone(),
features: fused_features.clone(),
classification,
signal_field,
vital_signs: Some(VitalSigns {
@@ -3386,8 +3584,10 @@ async fn udp_receiver_task(state: SharedState, udp_port: u16) {
let vitals = smooth_vitals_node(ns, &raw_vitals);
ns.latest_vitals = vitals.clone();
let raw_score = compute_person_score(&features);
ns.smoothed_person_score = ns.smoothed_person_score * 0.90 + raw_score * 0.10;
// DynamicMinCut person estimation from subcarrier correlation.
let corr_persons = estimate_persons_from_correlation(&ns.frame_history);
let raw_score = corr_persons as f64 / 3.0;
ns.smoothed_person_score = ns.smoothed_person_score * 0.92 + raw_score * 0.08;
if classification.presence {
let count = score_to_person_count(ns.smoothed_person_score, ns.prev_person_count);
ns.prev_person_count = count;
@@ -3395,6 +3595,9 @@ async fn udp_receiver_task(state: SharedState, udp_port: u16) {
ns.prev_person_count = 0;
}
// Store latest features on node for cross-node fusion.
ns.latest_features = Some(features.clone());
// Done with per-node mutable borrow; now read aggregated
// state from all nodes (the borrow of `ns` ends here).
// (We re-borrow node_states immutably via `s` below.)
@@ -3405,6 +3608,9 @@ async fn udp_receiver_task(state: SharedState, udp_port: u16) {
}
s.latest_vitals = vitals.clone();
// Cross-node fusion: combine features from all active nodes.
let fused_features = fuse_multi_node_features(&features, &s.node_states);
s.tick += 1;
let tick = s.tick;
@@ -3413,11 +3619,23 @@ async fn udp_receiver_task(state: SharedState, udp_port: u16) {
else { 0.05 };
// Aggregate person count across all active nodes.
// Use max (not sum) because nodes in the same room see the
// same people — summing would double-count.
let now = std::time::Instant::now();
let total_persons: usize = s.node_states.values()
.filter(|n| n.last_frame_time.map_or(false, |t| now.duration_since(t).as_secs() < 10))
.map(|n| n.prev_person_count)
.sum();
.max()
.unwrap_or(0);
// Boost classification confidence with multi-node coverage.
let n_active = s.node_states.values()
.filter(|ns| ns.last_frame_time.map_or(false, |t| now.duration_since(t).as_secs() < 10))
.count();
if n_active > 1 {
classification.confidence = (classification.confidence
* (1.0 + 0.15 * (n_active as f64 - 1.0))).clamp(0.0, 1.0);
}
// Build nodes array with all active nodes.
let active_nodes: Vec<NodeInfo> = s.node_states.iter()
@@ -3439,11 +3657,11 @@ async fn udp_receiver_task(state: SharedState, udp_port: u16) {
source: "esp32".to_string(),
tick,
nodes: active_nodes,
features: features.clone(),
features: fused_features.clone(),
classification,
signal_field: generate_signal_field(
features.mean_rssi, motion_score, breathing_rate_hz,
features.variance.min(1.0), &sub_variances,
fused_features.mean_rssi, motion_score, breathing_rate_hz,
fused_features.variance.min(1.0), &sub_variances,
),
vital_signs: Some(vitals),
enhanced_motion: None,
@@ -0,0 +1 @@
{"intelligence":60,"timestamp":1774039923051}
+44 -8
View File
@@ -56,10 +56,47 @@ export class PoseRenderer {
[11, 13], [12, 14], [13, 15], [14, 16] // Legs
];
// Client-side keypoint smoothing: lerp between frames to reduce jitter.
// Maps person index → array of {x, y} for each keypoint.
this._smoothedKeypoints = new Map();
this._lerpAlpha = 0.25; // 0 = frozen, 1 = instant (no smoothing)
// Initialize rendering context
this.initializeContext();
}
// Lerp a single value toward target
_lerp(current, target, alpha) {
return current + (target - current) * alpha;
}
// Get smoothed keypoint positions for a person
_getSmoothedKeypoints(personIdx, keypoints) {
if (!this.config.enableSmoothing || !keypoints || keypoints.length === 0) {
return keypoints;
}
let prev = this._smoothedKeypoints.get(personIdx);
if (!prev || prev.length !== keypoints.length) {
// First frame or keypoint count changed — initialize
prev = keypoints.map(kp => ({ x: kp.x, y: kp.y, z: kp.z || 0, confidence: kp.confidence, name: kp.name }));
this._smoothedKeypoints.set(personIdx, prev);
return keypoints;
}
const alpha = this._lerpAlpha;
const smoothed = keypoints.map((kp, i) => ({
...kp,
x: this._lerp(prev[i].x, kp.x, alpha),
y: this._lerp(prev[i].y, kp.y, alpha),
}));
// Update stored positions
this._smoothedKeypoints.set(personIdx, smoothed.map(kp => ({ x: kp.x, y: kp.y, z: kp.z || 0, confidence: kp.confidence, name: kp.name })));
return smoothed;
}
createLogger() {
return {
debug: (...args) => console.debug('[RENDERER-DEBUG]', new Date().toISOString(), ...args),
@@ -150,18 +187,17 @@ export class PoseRenderer {
return; // Skip low confidence detections
}
console.log(`✅ [RENDERER] Rendering person ${index} with confidence: ${person.confidence}`);
// Apply client-side lerp smoothing to reduce visual jitter
const smoothedKps = this._getSmoothedKeypoints(index, person.keypoints);
// Render skeleton connections
if (this.config.showSkeleton && person.keypoints) {
console.log(`🦴 [RENDERER] Rendering skeleton for person ${index}`);
this.renderSkeleton(person.keypoints, person.confidence);
if (this.config.showSkeleton && smoothedKps) {
this.renderSkeleton(smoothedKps, person.confidence);
}
// Render keypoints
if (this.config.showKeypoints && person.keypoints) {
console.log(`🔴 [RENDERER] Rendering keypoints for person ${index}`);
this.renderKeypoints(person.keypoints, person.confidence);
if (this.config.showKeypoints && smoothedKps) {
this.renderKeypoints(smoothedKps, person.confidence);
}
// Render bounding box
@@ -265,7 +301,7 @@ export class PoseRenderer {
persons.forEach((person, personIdx) => {
if (person.confidence < this.config.confidenceThreshold || !person.keypoints) return;
const kps = person.keypoints;
const kps = this._getSmoothedKeypoints(personIdx, person.keypoints);
bodyParts.forEach((part) => {
// Collect valid keypoints for this body part