fix: WebSocket race condition, data source indicators, auto-start pose detection (#96)

* feat: RVF training pipeline & UI integration (ADR-036)

Implement full model training, management, and inference pipeline:

Backend (Rust):
- recording.rs: CSI recording API (start/stop/list/download/delete)
- model_manager.rs: RVF model loading, LoRA profile switching, model library
- training_api.rs: Training API with WebSocket progress streaming, simulated
  training mode with realistic loss curves, auto-RVF export on completion
- main.rs: Wire new modules, recording hooks in all CSI paths, data dirs

UI (new components):
- ModelPanel.js: Dark-mode model library with load/unload, LoRA dropdown
- TrainingPanel.js: Recording controls, training config, live Canvas charts
- model.service.js: Model REST API client with events
- training.service.js: Training + recording API client with WebSocket progress

UI (enhancements):
- LiveDemoTab: Model selector, LoRA profile switcher, A/B split view toggle,
  training quick-panel with 60s recording shortcut
- SettingsPanel: Full dark mode conversion (issue #92), model configuration
  (device, threads, auto-load), training configuration (epochs, LR, patience)
- PoseDetectionCanvas: 10-frame pose trail with ghost keypoints and motion
  trajectory lines, cyan trail toggle button
- pose.service.js: Model-inference confidence thresholds

UI (plumbing):
- index.html: Training tab (8th tab)
- app.js: Panel initialization and tab routing
- style.css: ~250 lines of training/model panel dark-mode styles

191 Rust tests pass, 0 failures. Closes #92.

Refs: ADR-036, #93

Co-Authored-By: claude-flow <ruv@ruv.net>

* fix: real RuVector training pipeline + UI service fixes

Training pipeline (training_api.rs):
- Replace simulated training with real signal-based training loop
- Load actual CSI data from .csi.jsonl recordings or live frame history
- Extract 180 features per frame: subcarrier amplitudes, temporal variance,
  Goertzel frequency analysis (9 bands), motion gradients, global stats
- Train calibrated linear CSI-to-pose mapping via mini-batch gradient descent
  with L2 regularization (ridge regression), Xavier init, cosine LR decay
- Self-supervised: teacher targets from derive_pose_from_sensing() heuristics
- Real validation metrics: MSE and PCK@0.2 on 80/20 train/val split
- Export trained .rvf with real weights, feature normalization stats, witness
- Add infer_pose_from_model() for live inference from trained model
- 16 new tests covering features, training, inference, serialization

UI fixes:
- Fix double-URL bug in model.service.js and training.service.js
  (buildApiUrl was called twice — once in service, once in apiService)
- Fix route paths to match Rust backend (/api/v1/train/*, /api/v1/recording/*)
- Fix request body formats (session_name, nested config object)
- Fix top-level await in LiveDemoTab.js blocking module graph
- Dynamic imports for ModelPanel/TrainingPanel in app.js
- Center nav tabs with flex-wrap for 8-tab layout

Co-Authored-By: claude-flow <ruv@ruv.net>

* fix: WebSocket onOpen race condition, data source indicators, auto-start pose detection

- Fix WebSocket onOpen race condition in websocket.service.js where
  setupEventHandlers replaced onopen after socket was already open,
  preventing pose service from receiving connection signal
- Add 4-state data source indicator (LIVE/SIMULATED/RECONNECTING/OFFLINE)
  across Dashboard, Sensing, and Live Demo tabs via sensing.service.js
- Add hot-plug ESP32 auto-detection in sensing server (auto mode runs
  both UDP listener and simulation, switches on ESP32_TIMEOUT)
- Auto-start pose detection when backend is reachable
- Hide duplicate PoseDetectionCanvas controls when enableControls=false
- Add standalone Demo button in LiveDemoTab for offline animated demo
- Add data source banner and status styling

Co-Authored-By: claude-flow <ruv@ruv.net>
This commit is contained in:
rUv
2026-03-02 13:47:49 -05:00
committed by GitHub
parent c193cd4299
commit 113011e704
20 changed files with 6124 additions and 83 deletions
+152
View File
@@ -0,0 +1,152 @@
// Model Service for WiFi-DensePose UI
// Manages model loading, listing, LoRA profiles, and lifecycle events.
import { apiService } from './api.service.js';
export class ModelService {
constructor() {
this.activeModel = null;
this.listeners = {};
this.logger = this.createLogger();
}
createLogger() {
return {
debug: (...args) => console.debug('[MODEL-DEBUG]', new Date().toISOString(), ...args),
info: (...args) => console.info('[MODEL-INFO]', new Date().toISOString(), ...args),
warn: (...args) => console.warn('[MODEL-WARN]', new Date().toISOString(), ...args),
error: (...args) => console.error('[MODEL-ERROR]', new Date().toISOString(), ...args)
};
}
// --- Event emitter helpers ---
on(event, callback) {
if (!this.listeners[event]) {
this.listeners[event] = [];
}
this.listeners[event].push(callback);
return () => this.off(event, callback);
}
off(event, callback) {
if (!this.listeners[event]) return;
this.listeners[event] = this.listeners[event].filter(cb => cb !== callback);
}
emit(event, data) {
if (!this.listeners[event]) return;
this.listeners[event].forEach(cb => {
try { cb(data); } catch (err) { this.logger.error('Listener error', { event, err }); }
});
}
// --- API methods ---
async listModels() {
try {
const data = await apiService.get('/api/v1/models');
this.logger.info('Listed models', { count: data?.models?.length ?? 0 });
return data;
} catch (error) {
this.logger.error('Failed to list models', { error: error.message });
throw error;
}
}
async getModel(id) {
try {
const data = await apiService.get(`/api/v1/models/${encodeURIComponent(id)}`);
return data;
} catch (error) {
this.logger.error('Failed to get model', { id, error: error.message });
throw error;
}
}
async loadModel(modelId) {
try {
this.logger.info('Loading model', { modelId });
const data = await apiService.post('/api/v1/models/load', { model_id: modelId });
this.activeModel = { model_id: modelId };
this.emit('model-loaded', { model_id: modelId });
return data;
} catch (error) {
this.logger.error('Failed to load model', { modelId, error: error.message });
throw error;
}
}
async unloadModel() {
try {
this.logger.info('Unloading model');
const data = await apiService.post('/api/v1/models/unload', {});
this.activeModel = null;
this.emit('model-unloaded', {});
return data;
} catch (error) {
this.logger.error('Failed to unload model', { error: error.message });
throw error;
}
}
async getActiveModel() {
try {
const data = await apiService.get('/api/v1/models/active');
this.activeModel = data || null;
return this.activeModel;
} catch (error) {
if (error.status === 404) {
this.activeModel = null;
return null;
}
this.logger.error('Failed to get active model', { error: error.message });
throw error;
}
}
async activateLoraProfile(modelId, profileName) {
try {
this.logger.info('Activating LoRA profile', { modelId, profileName });
const data = await apiService.post(
'/api/v1/models/lora/activate',
{ model_id: modelId, profile_name: profileName }
);
this.emit('lora-activated', { model_id: modelId, profile: profileName });
return data;
} catch (error) {
this.logger.error('Failed to activate LoRA', { modelId, profileName, error: error.message });
throw error;
}
}
async getLoraProfiles() {
try {
const data = await apiService.get('/api/v1/models/lora/profiles');
return data?.profiles ?? [];
} catch (error) {
this.logger.error('Failed to get LoRA profiles', { error: error.message });
throw error;
}
}
async deleteModel(id) {
try {
this.logger.info('Deleting model', { id });
const data = await apiService.delete(`/api/v1/models/${encodeURIComponent(id)}`);
return data;
} catch (error) {
this.logger.error('Failed to delete model', { id, error: error.message });
throw error;
}
}
dispose() {
this.listeners = {};
this.activeModel = null;
this.logger.info('ModelService disposed');
}
}
// Create singleton instance
export const modelService = new ModelService();
+32 -6
View File
@@ -21,13 +21,17 @@ export class PoseService {
};
this.validationErrors = [];
this.logger = this.createLogger();
// Model inference mode tracking
this.modelActive = false;
// Configuration
this.config = {
enableValidation: true,
enablePerformanceTracking: true,
maxValidationErrors: 10,
confidenceThreshold: 0.3,
confidenceThresholdModelInference: 0.15,
maxPersons: 10,
timeoutMs: 5000
};
@@ -127,9 +131,14 @@ export class PoseService {
throw new Error(`Invalid stream options: ${validationResult.errors.join(', ')}`);
}
// Use a lower confidence threshold when model inference is active
const defaultThreshold = this.modelActive
? this.config.confidenceThresholdModelInference
: this.config.confidenceThreshold;
const params = {
zone_ids: options.zoneIds?.join(','),
min_confidence: options.minConfidence || this.config.confidenceThreshold,
min_confidence: options.minConfidence || defaultThreshold,
max_fps: options.maxFps || 30,
token: options.token || apiService.authToken
};
@@ -494,9 +503,18 @@ export class PoseService {
};
}
// Extract persons from zone data
const persons = zoneData.pose.persons || [];
console.log('👥 Extracted persons:', persons);
// Determine the pose source for this message
const poseSource = originalMessage.pose_source || zoneData.pose_source || null;
// Choose confidence threshold based on pose source
const threshold = (poseSource === 'model_inference' || this.modelActive)
? this.config.confidenceThresholdModelInference
: this.config.confidenceThreshold;
// Extract persons from zone data, applying source-aware filtering
const rawPersons = zoneData.pose.persons || [];
const persons = rawPersons.filter(p => p.confidence === undefined || p.confidence >= threshold);
console.log('Extracted persons:', persons.length, '/', rawPersons.length, '(threshold:', threshold, ')');
// Create zone summary
const zoneSummary = {};
@@ -511,7 +529,7 @@ export class PoseService {
persons: persons,
zone_summary: zoneSummary,
processing_time_ms: zoneData.metadata?.processing_time_ms || 0,
pose_source: originalMessage.pose_source || zoneData.pose_source || null,
pose_source: poseSource,
metadata: {
mock_data: false,
source: 'websocket',
@@ -653,6 +671,14 @@ export class PoseService {
this.logger.info('Configuration updated', { config: this.config });
}
// Enable or disable model inference mode.
// When active, confidence thresholds are lowered because model inference
// produces more reliable detections than raw signal-derived heuristics.
setModelMode(active) {
this.modelActive = !!active;
this.logger.info('Model mode updated', { modelActive: this.modelActive });
}
// Health check
async healthCheck() {
try {
+61 -3
View File
@@ -32,8 +32,14 @@ class SensingService {
this._simTimer = null;
// Connection state: disconnected | connecting | connected | reconnecting | simulated
this._state = 'disconnected';
// Data-source label exposed to the UI: "live" | "reconnecting" | "simulated"
// Data-source label exposed to the UI:
// "live" — real ESP32 hardware connected
// "server-simulated" — server is running but using synthetic data (no hardware)
// "reconnecting" — WebSocket disconnected, retrying
// "simulated" — client-side fallback simulation (server unreachable)
this._dataSource = 'reconnecting';
// The raw source string from the server (e.g. "esp32", "simulated", "simulate")
this._serverSource = null;
this._lastMessage = null;
// Ring buffer of recent RSSI values for sparkline
@@ -113,7 +119,9 @@ class SensingService {
this._reconnectAttempt = 0;
this._stopSimulation();
this._setState('connected');
this._setDataSource('live');
// Don't assume "live" yet — wait for first frame's source field.
// Fetch server status to determine actual data source immediately.
this._detectServerSource();
};
this._ws.onmessage = (evt) => {
@@ -256,11 +264,61 @@ class SensingService {
};
}
// ---- Server source detection -------------------------------------------
/**
* Fetch `/api/v1/status` to find out if the server is using real
* hardware or simulation. Called once on WebSocket open.
*/
async _detectServerSource() {
try {
const resp = await fetch('/api/v1/status');
if (resp.ok) {
const json = await resp.json();
this._applyServerSource(json.source);
} else {
// Can't reach status endpoint — assume live until first frame tells us
this._setDataSource('live');
}
} catch {
this._setDataSource('live');
}
}
/**
* Map a raw server source string to the UI data-source label.
*/
_applyServerSource(rawSource) {
this._serverSource = rawSource;
if (rawSource === 'esp32' || rawSource === 'wifi' || rawSource === 'live') {
this._setDataSource('live');
} else if (rawSource === 'simulated' || rawSource === 'simulate') {
this._setDataSource('server-simulated');
} else {
// Unknown source — show as server-simulated to be safe
this._setDataSource('server-simulated');
}
}
/** @return {string|null} Raw server source (e.g. "esp32", "simulated") */
get serverSource() {
return this._serverSource;
}
// ---- Data handling -----------------------------------------------------
_handleData(data) {
this._lastMessage = data;
// Track the server's source field from each frame so the UI
// can react if the server switches between esp32 ↔ simulated at runtime.
if (data.source && this._state === 'connected') {
const raw = data.source;
if (raw !== this._serverSource) {
this._applyServerSource(raw);
}
}
// Update RSSI history for sparkline
if (data.features && data.features.mean_rssi != null) {
this._rssiHistory.push(data.features.mean_rssi);
@@ -292,7 +350,7 @@ class SensingService {
/**
* Update the dataSource label and notify state listeners so the UI can
* react without needing a separate subscription.
* @param {'live'|'reconnecting'|'simulated'} source
* @param {'live'|'server-simulated'|'reconnecting'|'simulated'} source
*/
_setDataSource(source) {
if (source === this._dataSource) return;
+211
View File
@@ -0,0 +1,211 @@
// Training Service for WiFi-DensePose UI
// Manages training lifecycle, progress streaming, and CSI recordings.
import { buildWsUrl } from '../config/api.config.js';
import { apiService } from './api.service.js';
export class TrainingService {
constructor() {
this.progressSocket = null;
this.listeners = {};
this.logger = this.createLogger();
}
createLogger() {
return {
debug: (...args) => console.debug('[TRAIN-DEBUG]', new Date().toISOString(), ...args),
info: (...args) => console.info('[TRAIN-INFO]', new Date().toISOString(), ...args),
warn: (...args) => console.warn('[TRAIN-WARN]', new Date().toISOString(), ...args),
error: (...args) => console.error('[TRAIN-ERROR]', new Date().toISOString(), ...args)
};
}
// --- Event emitter helpers ---
on(event, callback) {
if (!this.listeners[event]) {
this.listeners[event] = [];
}
this.listeners[event].push(callback);
return () => this.off(event, callback);
}
off(event, callback) {
if (!this.listeners[event]) return;
this.listeners[event] = this.listeners[event].filter(cb => cb !== callback);
}
emit(event, data) {
if (!this.listeners[event]) return;
this.listeners[event].forEach(cb => {
try { cb(data); } catch (err) { this.logger.error('Listener error', { event, err }); }
});
}
// --- Training API methods ---
async startTraining(config) {
try {
this.logger.info('Starting training', { config });
const data = await apiService.post('/api/v1/train/start', config);
this.emit('training-started', data);
return data;
} catch (error) {
this.logger.error('Failed to start training', { error: error.message });
throw error;
}
}
async stopTraining() {
try {
this.logger.info('Stopping training');
const data = await apiService.post('/api/v1/train/stop', {});
this.emit('training-stopped', data);
return data;
} catch (error) {
this.logger.error('Failed to stop training', { error: error.message });
throw error;
}
}
async getTrainingStatus() {
try {
const data = await apiService.get('/api/v1/train/status');
return data;
} catch (error) {
this.logger.error('Failed to get training status', { error: error.message });
throw error;
}
}
async startPretraining(config) {
try {
this.logger.info('Starting pretraining', { config });
const data = await apiService.post('/api/v1/train/pretrain', config);
this.emit('training-started', data);
return data;
} catch (error) {
this.logger.error('Failed to start pretraining', { error: error.message });
throw error;
}
}
async startLoraTraining(config) {
try {
this.logger.info('Starting LoRA training', { config });
const data = await apiService.post('/api/v1/train/lora', config);
this.emit('training-started', data);
return data;
} catch (error) {
this.logger.error('Failed to start LoRA training', { error: error.message });
throw error;
}
}
// --- Recording API methods ---
async listRecordings() {
try {
const data = await apiService.get('/api/v1/recording/list');
return data?.recordings ?? [];
} catch (error) {
this.logger.error('Failed to list recordings', { error: error.message });
throw error;
}
}
async startRecording(config) {
try {
this.logger.info('Starting recording', { config });
const data = await apiService.post('/api/v1/recording/start', config);
this.emit('recording-started', data);
return data;
} catch (error) {
this.logger.error('Failed to start recording', { error: error.message });
throw error;
}
}
async stopRecording() {
try {
this.logger.info('Stopping recording');
const data = await apiService.post('/api/v1/recording/stop', {});
this.emit('recording-stopped', data);
return data;
} catch (error) {
this.logger.error('Failed to stop recording', { error: error.message });
throw error;
}
}
async deleteRecording(id) {
try {
this.logger.info('Deleting recording', { id });
const data = await apiService.delete(
`/api/v1/recording/${encodeURIComponent(id)}`
);
return data;
} catch (error) {
this.logger.error('Failed to delete recording', { id, error: error.message });
throw error;
}
}
// --- WebSocket progress stream ---
connectProgressStream() {
if (this.progressSocket) {
this.logger.warn('Progress stream already connected');
return this.progressSocket;
}
const url = buildWsUrl('/ws/train/progress');
this.logger.info('Connecting progress stream', { url });
const ws = new WebSocket(url);
ws.onopen = () => {
this.logger.info('Progress stream connected');
this.emit('progress-connected', {});
};
ws.onmessage = (event) => {
try {
const data = JSON.parse(event.data);
this.emit('progress', data);
} catch (err) {
this.logger.warn('Failed to parse progress message', { error: err.message });
}
};
ws.onerror = (error) => {
this.logger.error('Progress stream error', { error });
this.emit('progress-error', { error });
};
ws.onclose = () => {
this.logger.info('Progress stream disconnected');
this.progressSocket = null;
this.emit('progress-disconnected', {});
};
this.progressSocket = ws;
return ws;
}
disconnectProgressStream() {
if (this.progressSocket) {
this.progressSocket.close();
this.progressSocket = null;
}
}
dispose() {
this.disconnectProgressStream();
this.listeners = {};
this.logger.info('TrainingService disposed');
}
}
// Create singleton instance
export const trainingService = new TrainingService();
+16 -1
View File
@@ -83,9 +83,24 @@ export class WebSocketService {
const ws = await this.createWebSocketWithTimeout(url);
connectionData.ws = ws;
// Set up event handlers
// Set up event handlers (replaces onopen/onmessage/etc.)
this.setupEventHandlers(url, ws, handlers);
// The WebSocket is already open at this point (createWebSocketWithTimeout
// resolved on the original onopen). setupEventHandlers replaced onopen, so
// the new handler never fires. Manually trigger the connected path now.
if (ws.readyState === WebSocket.OPEN) {
connectionData.status = 'connected';
connectionData.lastActivity = Date.now();
this.reconnectAttempts.set(url, 0);
this.notifyConnectionState(url, 'connected');
if (handlers.onOpen) {
try { handlers.onOpen(new Event('open')); } catch (e) {
this.logger.error('Error in onOpen handler', { url, error: e.message });
}
}
}
// Start heartbeat
this.startHeartbeat(url);