mirror of
https://github.com/ruvnet/RuView
synced 2026-07-23 17:33:20 +00:00
Compare commits
2 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 ruv
|
6c230ed928 | ||
|
rUv
|
bf30844835 |
@@ -1,35 +0,0 @@
|
||||
# Line-ending policy.
|
||||
#
|
||||
# `* text=auto` lets git normalise text files to LF in the repository and convert
|
||||
# to the platform's native line endings on checkout. That default is fine for
|
||||
# .md / .rs / .toml / .py — broken for shell scripts and Dockerfiles, where
|
||||
# CRLF on the shebang line causes Linux exec to look for an interpreter named
|
||||
# `/bin/sh\r` (or similar) and fail with "no such file or directory".
|
||||
#
|
||||
# Force LF for anything that ends up executed inside a Linux container or a
|
||||
# POSIX shell. This is what prevented the v0.8.0 image from booting at first
|
||||
# build until the entrypoint was renormalised.
|
||||
* text=auto
|
||||
*.sh text eol=lf
|
||||
*.bash text eol=lf
|
||||
verify text eol=lf
|
||||
Dockerfile* text eol=lf
|
||||
docker/* text eol=lf
|
||||
scripts/* text eol=lf
|
||||
|
||||
# Binary blobs that should never be touched by text-normalisation.
|
||||
*.bin binary
|
||||
*.png binary
|
||||
*.jpg binary
|
||||
*.jpeg binary
|
||||
*.gif binary
|
||||
*.ico binary
|
||||
*.zip binary
|
||||
*.tar binary
|
||||
*.tgz binary
|
||||
*.gz binary
|
||||
*.wasm binary
|
||||
*.rvf binary
|
||||
*.task binary
|
||||
*.csi.jsonl binary
|
||||
*.pcap binary
|
||||
@@ -57,13 +57,7 @@ jobs:
|
||||
"
|
||||
|
||||
- name: Run pipeline verification
|
||||
working-directory: archive/v1
|
||||
env:
|
||||
# verify.py transitively imports src.app -> src.config.settings, which
|
||||
# uses pydantic-settings with a required `secret_key` field. The proof
|
||||
# only needs the import chain to resolve; the value is never used for
|
||||
# any auth path in the proof pipeline.
|
||||
SECRET_KEY: ci-proof-replay-only-not-a-real-secret
|
||||
working-directory: v1
|
||||
run: |
|
||||
echo "=== Running pipeline verification ==="
|
||||
python data/proof/verify.py
|
||||
@@ -71,9 +65,7 @@ jobs:
|
||||
echo "Pipeline verification PASSED."
|
||||
|
||||
- name: Run verification twice to confirm determinism
|
||||
working-directory: archive/v1
|
||||
env:
|
||||
SECRET_KEY: ci-proof-replay-only-not-a-real-secret
|
||||
working-directory: v1
|
||||
run: |
|
||||
echo "=== Second run for determinism confirmation ==="
|
||||
python data/proof/verify.py
|
||||
|
||||
+1
-2
@@ -14,8 +14,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
regime classification) and `temporal-compare` (DTW pattern matching) as a
|
||||
**parallel tap** alongside RuView's existing event pipeline — no replacement,
|
||||
no behaviour change to the existing `/ws/sensing` fan-out or `wifi-densepose-signal`
|
||||
DSP. Two new endpoints (always mounted — the tap is cheap enough at 0.041 ms p99
|
||||
per-frame `update()` to ship hot by default):
|
||||
DSP. Two new endpoints (off by default, enabled via `--introspection`):
|
||||
- `GET /ws/introspection` — newline-delimited JSON snapshots streamed at the CSI
|
||||
frame rate. Each snapshot carries `frame_count`, `regime` (Idle / Periodic /
|
||||
Transient / Chaotic / Unknown), `lyapunov_exponent`, `attractor_dim`,
|
||||
|
||||
@@ -15,7 +15,7 @@
|
||||
|
||||
## **See through walls with WiFi** ##
|
||||
|
||||
**Turn ordinary WiFi into a spacial intelligence / sensing system.** Detect people, measure breathing and heart rate, track movement, and monitor rooms — through walls, in the dark, with no cameras or wearables. Just physics.
|
||||
**Turn ordinary WiFi into a spatial intelligence / sensing system.** Detect people, measure breathing and heart rate, track movement, and monitor rooms — through walls, in the dark, with no cameras or wearables. Just physics.
|
||||
|
||||
### π RuView is a WiFi sensing platform that turns radio signals into spatial intelligence.
|
||||
|
||||
|
||||
@@ -1,52 +0,0 @@
|
||||
# Reference platforms for `expected_features.sha256`
|
||||
|
||||
The hash in `expected_features.sha256` was generated on a specific BLAS / FFT
|
||||
backend. Numpy + scipy delegate FFT/linear-algebra to platform-native
|
||||
libraries, and those libraries produce **bit-different output on identical
|
||||
IEEE 754 inputs** depending on the backend. This is not a bug in the proof
|
||||
pipeline — it is a property of the underlying numerical libraries. (See
|
||||
issue #560.)
|
||||
|
||||
## Platforms where the hash is expected to MATCH
|
||||
|
||||
| Platform | BLAS backend | Status |
|
||||
|---|---|---|
|
||||
| `linux-x86_64-gnu` (Python 3.11.x, numpy 1.26.4 from PyPI wheels, scipy 1.14.1) | OpenBLAS | ✅ Reference |
|
||||
| `windows-x86_64-msvc` (Python 3.11.x / 3.13.x, numpy 1.26.4 from PyPI wheels, scipy 1.14.1) | OpenBLAS | ✅ Reference |
|
||||
|
||||
## Platforms where the hash is **expected to MISMATCH**
|
||||
|
||||
| Platform | BLAS backend | Why |
|
||||
|---|---|---|
|
||||
| `darwin-arm64` (macOS arm64, Apple Silicon) | Accelerate.framework | FFT + matrix kernels differ in last-bit positions; the SHA-256 will differ even with pinned `numpy 1.26.4` / `scipy 1.14.1`. |
|
||||
| Any environment with MKL installed | Intel MKL | Same root cause as Accelerate: different vectorized FFT path. |
|
||||
|
||||
## What to do if you get MISMATCH on a non-reference platform
|
||||
|
||||
The pipeline is still correct on your platform — the *output* is bit-different
|
||||
because the *backend* is bit-different, not because the proof code has a bug.
|
||||
Three workable responses:
|
||||
|
||||
1. **Run the proof on a reference platform** (Linux x86_64 or Windows x86_64
|
||||
with the PyPI OpenBLAS wheels). This is what CI does.
|
||||
|
||||
2. **Generate a new local-reference hash** for your platform and check it
|
||||
against the same hash on a teammate's machine with the *same* backend:
|
||||
|
||||
```bash
|
||||
# Regenerate from your platform
|
||||
python archive/v1/data/proof/verify.py --generate-hash
|
||||
|
||||
# Commit the new hash to a side file (do NOT overwrite expected_features.sha256
|
||||
# unless you are publishing a new cross-platform reference)
|
||||
```
|
||||
|
||||
3. **Compare numerical output, not the hash.** A relaxed-tolerance comparison
|
||||
on the feature vectors (e.g. `np.allclose(features, reference, atol=1e-10)`)
|
||||
will pass across backends. This is on the roadmap (see issue #560).
|
||||
|
||||
## The `verify.py` runtime environment block
|
||||
|
||||
Every run of `verify.py` now prints a `RUNTIME ENVIRONMENT` block before the
|
||||
pipeline runs. Include that block in any issue report — it identifies the
|
||||
platform + numpy version + BLAS backend in one place.
|
||||
@@ -116,48 +116,6 @@ def print_source_provenance():
|
||||
print()
|
||||
|
||||
|
||||
def print_runtime_environment():
|
||||
"""Print the platform + numpy/scipy BLAS backend.
|
||||
|
||||
The proof pipeline's SHA-256 is sensitive to the BLAS / FFT backend
|
||||
behind numpy + scipy.fft. Different platforms ship different backends
|
||||
(OpenBLAS on Linux/Windows wheels, Accelerate.framework on macOS arm64,
|
||||
MKL when installed) and they produce bit-different output on identical
|
||||
IEEE 754 inputs. Surfacing the backend up front turns an unexplained
|
||||
MISMATCH into a one-line diagnosis -- see issue #560.
|
||||
"""
|
||||
import platform
|
||||
print(" RUNTIME ENVIRONMENT:")
|
||||
print(f" Platform : {platform.platform()}")
|
||||
print(f" Machine : {platform.machine()}")
|
||||
print(f" Python : {platform.python_version()} ({platform.python_implementation()})")
|
||||
|
||||
# numpy BLAS / LAPACK backend.
|
||||
try:
|
||||
blas_info = np.__config__.blas_ilp64_opt_info # type: ignore[attr-defined]
|
||||
backend = getattr(blas_info, "get", lambda *_: None)("libraries", None) or "unknown"
|
||||
except Exception:
|
||||
# Newer numpy (>= 1.26) reports via show_config(); fall back to a stringified dump.
|
||||
try:
|
||||
import io
|
||||
buf = io.StringIO()
|
||||
np.show_config(mode="dicts") if hasattr(np, "show_config") else None
|
||||
# `show_config(mode='dicts')` returns a dict in numpy >= 1.26.
|
||||
cfg = np.show_config(mode="dicts") if hasattr(np, "show_config") else {}
|
||||
if isinstance(cfg, dict):
|
||||
blas = cfg.get("Build Dependencies", {}).get("blas", {})
|
||||
backend = blas.get("name", "unknown")
|
||||
else:
|
||||
backend = "unknown"
|
||||
except Exception:
|
||||
backend = "unknown"
|
||||
print(f" numpy BLAS : {backend}")
|
||||
print(" (FFT/BLAS backend affects the hash -- see #560 if MISMATCH on")
|
||||
print(" macOS arm64 / Accelerate. Reference platforms: linux-x86_64,")
|
||||
print(" windows-x86_64 with OpenBLAS; see expected_features.sha256.)")
|
||||
print()
|
||||
|
||||
|
||||
def load_reference_signal(data_path):
|
||||
"""Load the reference CSI signal from JSON.
|
||||
|
||||
@@ -459,7 +417,6 @@ def main():
|
||||
# ---------------------------------------------------------------
|
||||
print("[0/4] SOURCE PROVENANCE")
|
||||
print_source_provenance()
|
||||
print_runtime_environment()
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# Step 1: Load and describe reference signal
|
||||
@@ -561,23 +518,13 @@ def main():
|
||||
print()
|
||||
print(" The pipeline output does NOT match the expected hash.")
|
||||
print()
|
||||
print(" Likely causes, in order of probability:")
|
||||
print(" 1. Platform BLAS/FFT backend differs from the reference.")
|
||||
print(" The expected hash was generated on linux-x86_64 +")
|
||||
print(" windows-x86_64 with OpenBLAS. macOS arm64 ships with")
|
||||
print(" Accelerate.framework, which produces bit-different FFT")
|
||||
print(" output on identical inputs (issue #560). Inspect the")
|
||||
print(" RUNTIME ENVIRONMENT block printed at the top of this run.")
|
||||
print(" 2. Numpy/scipy version mismatch.")
|
||||
print(" Install pinned versions: pip install -r archive/v1/requirements-lock.txt")
|
||||
print(" 3. Real code change in the CSI processor that alters output.")
|
||||
print(" Investigate the diff against the reference commit.")
|
||||
print(" Possible causes:")
|
||||
print(" - Numpy/scipy version mismatch (check requirements)")
|
||||
print(" - Code change in CSI processor that alters numerical output")
|
||||
print(" - Platform floating-point differences (unlikely for IEEE 754)")
|
||||
print()
|
||||
print(" To regenerate the expected hash on a NEW reference platform:")
|
||||
print(" To update the expected hash after intentional changes:")
|
||||
print(" python verify.py --generate-hash")
|
||||
print(" (Only do this if you intend to publish a new reference; the")
|
||||
print(" single-platform contract of expected_features.sha256 is")
|
||||
print(" documented at the top of that file.)")
|
||||
print("=" * 72)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
@@ -3,11 +3,7 @@
|
||||
# Multi-stage build for minimal final image
|
||||
|
||||
# Stage 1: Build
|
||||
# Rust 1.87+ is required: `hnsw_rs 0.3.4` (transitive via wifi-densepose-ruvector ->
|
||||
# ruvector-attn-mincut) uses `u*::is_multiple_of`, stabilised in 1.87. Pinning to a
|
||||
# recent stable (1.90) for reproducibility — bump cautiously since reproducible
|
||||
# builds rely on this.
|
||||
FROM rust:1.90-bookworm AS builder
|
||||
FROM rust:1.85-bookworm AS builder
|
||||
|
||||
WORKDIR /build
|
||||
|
||||
|
||||
@@ -40,21 +40,15 @@ MSYS_NO_PATHCONV=1 docker run --rm \
|
||||
```bash
|
||||
python -m esptool --chip esp32s3 --port COM7 --baud 460800 \
|
||||
write_flash --flash_mode dio --flash_size 8MB \
|
||||
0x0 firmware/esp32-csi-node/build/bootloader/bootloader.bin \
|
||||
0x8000 firmware/esp32-csi-node/build/partition_table/partition-table.bin \
|
||||
0xf000 firmware/esp32-csi-node/build/ota_data_initial.bin \
|
||||
0x20000 firmware/esp32-csi-node/build/esp32-csi-node.bin
|
||||
0x0 firmware/esp32-csi-node/build/bootloader/bootloader.bin \
|
||||
0x8000 firmware/esp32-csi-node/build/partition_table/partition-table.bin \
|
||||
0x10000 firmware/esp32-csi-node/build/esp32-csi-node.bin
|
||||
```
|
||||
|
||||
> The app slot (`ota_0`) starts at `0x20000` per `partitions_display.csv` /
|
||||
> `partitions_4mb.csv`. `ota_data_initial.bin` at `0xf000` initialises the OTA
|
||||
> slot pointer; without it the bootloader can refuse to boot the app after a
|
||||
> factory wipe.
|
||||
|
||||
### 3. Provision WiFi credentials (no reflash needed)
|
||||
|
||||
```bash
|
||||
python firmware/esp32-csi-node/provision.py --port COM7 \
|
||||
python scripts/provision.py --port COM7 \
|
||||
--ssid "YourSSID" --password "YourPass" --target-ip 192.168.1.20
|
||||
```
|
||||
|
||||
@@ -260,10 +254,9 @@ Find your serial port: `COM7` on Windows, `/dev/ttyUSB0` on Linux, `/dev/cu.SLAB
|
||||
```bash
|
||||
python -m esptool --chip esp32s3 --port COM7 --baud 460800 \
|
||||
write_flash --flash_mode dio --flash_size 8MB \
|
||||
0x0 firmware/esp32-csi-node/build/bootloader/bootloader.bin \
|
||||
0x8000 firmware/esp32-csi-node/build/partition_table/partition-table.bin \
|
||||
0xf000 firmware/esp32-csi-node/build/ota_data_initial.bin \
|
||||
0x20000 firmware/esp32-csi-node/build/esp32-csi-node.bin
|
||||
0x0 firmware/esp32-csi-node/build/bootloader/bootloader.bin \
|
||||
0x8000 firmware/esp32-csi-node/build/partition_table/partition-table.bin \
|
||||
0x10000 firmware/esp32-csi-node/build/esp32-csi-node.bin
|
||||
```
|
||||
|
||||
### Serial Monitor
|
||||
@@ -292,7 +285,7 @@ All settings can be changed at runtime via Non-Volatile Storage (NVS) without re
|
||||
The easiest way to write NVS settings:
|
||||
|
||||
```bash
|
||||
python firmware/esp32-csi-node/provision.py --port COM7 \
|
||||
python scripts/provision.py --port COM7 \
|
||||
--ssid "MyWiFi" \
|
||||
--password "MyPassword" \
|
||||
--target-ip 192.168.1.20
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Platform probe: reproduce verify.py's hash-relevant FFT steps in isolation.
|
||||
|
||||
Runs the same scipy.fft.fft / scipy.signal calls that verify.py hashes
|
||||
(csi_processor.py:426, :438, :349) on a deterministic synthetic input,
|
||||
without dragging in src.app / pydantic Settings. Used to empirically
|
||||
locate the source of platform divergence in issue #560.
|
||||
|
||||
Usage: python3 scripts/probe-fft-platform.py
|
||||
Output: single JSON object on stdout. Run on each platform and diff.
|
||||
|
||||
If two machines print the same `first8_doppler_bytes_hex` and the same
|
||||
`first4_psd_floats` but different `sha256`, the divergence is in later
|
||||
FFT bins (SIMD reordering). If even the first values differ, it's a
|
||||
true ULP-level divergence at every bin (Apple Silicon NEON vs x86_64
|
||||
AVX, or different scipy pocketfft builds).
|
||||
"""
|
||||
import hashlib
|
||||
import json
|
||||
import platform
|
||||
import struct
|
||||
import sys
|
||||
|
||||
import numpy as np
|
||||
import scipy.fft
|
||||
import scipy.signal
|
||||
|
||||
# Deterministic synthetic input -- no IO, no .env, no Settings
|
||||
rng = np.random.RandomState(42)
|
||||
N_FRAMES = 100
|
||||
N_SUBC = 100
|
||||
amp = rng.randn(N_FRAMES, N_SUBC).astype(np.float64)
|
||||
|
||||
# Mirror the three scipy calls verify.py's hash depends on:
|
||||
# archive/v1/src/core/csi_processor.py:349 -> scipy.signal.windows.hamming
|
||||
# archive/v1/src/core/csi_processor.py:426 -> scipy.fft.fft(mean_phase_diff, n=64)
|
||||
# archive/v1/src/core/csi_processor.py:438 -> scipy.fft.fft(amp.flatten(), n=128)
|
||||
mean_phase_diff = amp.mean(axis=1)
|
||||
doppler = np.abs(scipy.fft.fft(mean_phase_diff, n=64)) ** 2
|
||||
psd = np.abs(scipy.fft.fft(amp.flatten(), n=128)) ** 2
|
||||
window = scipy.signal.windows.hamming(56)
|
||||
|
||||
# Pack the same way verify.py:features_to_bytes does (little-endian f64)
|
||||
parts = []
|
||||
for arr in (doppler, psd, window):
|
||||
flat = np.asarray(arr, dtype=np.float64).ravel()
|
||||
parts.append(struct.pack(f"<{len(flat)}d", *flat))
|
||||
blob = b"".join(parts)
|
||||
|
||||
try:
|
||||
blas_info = np.show_config(mode="dicts")
|
||||
except Exception:
|
||||
blas_info = {"error": "show_config(mode=dicts) unavailable"}
|
||||
|
||||
print(json.dumps({
|
||||
"uname": platform.uname()._asdict(),
|
||||
"python": sys.version.split()[0],
|
||||
"numpy": np.__version__,
|
||||
"scipy": __import__("scipy").__version__,
|
||||
"blob_len": len(blob),
|
||||
"sha256": hashlib.sha256(blob).hexdigest(),
|
||||
"first8_doppler_bytes_hex": doppler[:8].tobytes().hex(),
|
||||
"first4_psd_floats": psd[:4].tolist(),
|
||||
"blas_backend": blas_info if isinstance(blas_info, dict) else str(blas_info),
|
||||
}, indent=2, default=str))
|
||||
Reference in New Issue
Block a user