Compare commits

..

2 Commits

Author SHA1 Message Date
rUv 88b835dd89 fix(ci): perf job gates on the real frame-budget guard, not TDD stubs (#915)
After #914 fixed collection, the perf job actually ran the suite and
exposed that test_api_throughput.py / test_inference_speed.py are TDD
red-phase stubs (every test suffixed `_should_fail_initially`) that time
a *mock that sleeps* — not a real perf signal. They carry machine-
dependent wall-clock asserts (actual_rps >= 40, batch_time < individual_time)
that are inherently flaky on shared CI runners, plus a cross-class
fixture-scope bug (`fixture 'standard_model' not found`). Result: 3 failed,
10 errored — by design, not a regression.

Forcing those green would manufacture a false signal. Instead, gate only
on test_frame_budget.py, which times the *real* CSIProcessor pipeline
against the ADR 50 ms per-frame budget (single-frame, p95/100-frames,
+Doppler) — a genuine regression guard. Verified locally: 3 passed.

The stub files remain in-repo for local TDD; they re-enter CI when their
features are implemented and the mock-timing asserts are made deterministic.
2026-06-02 18:31:55 +02:00
rUv f8f08076eb fix(ci): perf tests — use python -m pytest so src import resolves (#914)
The Performance Tests job collected 26 items then aborted with
`ModuleNotFoundError: No module named 'src'` on test_frame_budget.py,
which does `from src.core.csi_processor import CSIProcessor`. The bare
`pytest` console script does not put the cwd (archive/v1) on sys.path;
`python -m pytest` does. pytest aborts the whole session on a collection
error, so this one import masked the entire (otherwise mock-based,
self-contained) perf suite.

Verified locally: bare-script path reproduces the exact error; `-m`
resolves it and test_frame_budget.py passes 3/3. The other two files
(test_api_throughput.py mock server, test_inference_speed.py MockPoseModel
+psutil) are fully self-contained — no test hits the running server.

Closes the last red job in the v1-API CI chain (#910/#911/#913).
2026-06-02 18:12:00 +02:00
22 changed files with 68 additions and 55 deletions
+1 -1
View File
@@ -32,7 +32,7 @@ jobs:
run:
working-directory: v2
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Install Rust toolchain
run: rustup show && rustc --version
+1 -1
View File
@@ -52,7 +52,7 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
+3 -3
View File
@@ -41,7 +41,7 @@ jobs:
image_tag: ${{ steps.determine-tag.outputs.tag }}
steps:
- name: Checkout code
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Determine deployment environment
id: determine-env
@@ -85,7 +85,7 @@ jobs:
url: https://staging.wifi-densepose.com
steps:
- name: Checkout code
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Set up kubectl
uses: azure/setup-kubectl@v3
@@ -131,7 +131,7 @@ jobs:
url: https://wifi-densepose.com
steps:
- name: Checkout code
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Set up kubectl
uses: azure/setup-kubectl@v3
+26 -13
View File
@@ -27,7 +27,7 @@ jobs:
steps:
- name: Checkout code
continue-on-error: true
uses: actions/checkout@v6
uses: actions/checkout@v4
with:
fetch-depth: 0
@@ -81,7 +81,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v6
uses: actions/checkout@v4
# `wifi-densepose-desktop` is a Tauri v2 app — `glib-sys`, `gtk-sys`,
# `webkit2gtk-sys`, etc. need the Linux dev libraries via pkg-config or the
@@ -188,7 +188,7 @@ jobs:
steps:
- name: Checkout code
continue-on-error: true
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
continue-on-error: true
@@ -253,7 +253,7 @@ jobs:
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
steps:
- name: Checkout code
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v6
@@ -282,13 +282,26 @@ jobs:
env:
MOCK_POSE_DATA: "true"
run: |
# The repo's performance suite is pytest (test_api_throughput.py,
# test_frame_budget.py, test_inference_speed.py) — there is no
# locustfile.py, so the old `locust -f tests/performance/locustfile.py`
# command always failed with "Could not find ...". Run the real suite.
# -o addopts="" drops the root pyproject's --cov/--cov-fail-under=100
# flags (pytest-cov isn't installed here and 100% cov is for unit tests).
pytest tests/performance/ -o addopts="" -v --junitxml=perf-junit.xml
# Gate only on the genuine, deterministic perf guard:
# test_frame_budget.py times the *real* CSIProcessor pipeline against
# the ADR 50 ms per-frame budget (single-frame, p95 over 100 frames,
# +Doppler) — a true regression signal.
#
# test_api_throughput.py / test_inference_speed.py are excluded: every
# test there is a TDD red-phase stub (suffix `_should_fail_initially`)
# that times a *mock that sleeps* — meaningless as a perf signal, with
# machine-dependent wall-clock asserts (e.g. `actual_rps >= 40`,
# `batch_time < individual_time`) that are inherently flaky on shared
# CI runners, plus a cross-class fixture-scope bug. Forcing them green
# would be manufacturing a false signal; they stay in-repo for local
# TDD but do not gate CI until the underlying features are implemented.
#
# `python -m pytest` (not the bare `pytest` script) puts the cwd
# (archive/v1) on sys.path so `from src.core...` resolves — the bare
# script omits cwd and raises ModuleNotFoundError: No module named 'src'.
# -o addopts="" drops the root pyproject's --cov/--cov-fail-under=100.
python -m pytest tests/performance/test_frame_budget.py \
-o addopts="" -v --junitxml=perf-junit.xml
- name: Upload performance results
if: always()
@@ -313,7 +326,7 @@ jobs:
steps:
- name: Checkout code
continue-on-error: true
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Set up Docker Buildx
continue-on-error: true
@@ -385,7 +398,7 @@ jobs:
contents: write # gh-pages deploy needs write (GITHUB_TOKEN is read-only by default -> 403)
steps:
- name: Checkout code
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v6
+1 -1
View File
@@ -34,7 +34,7 @@ jobs:
snapshot:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Fetch /traffic/clones + /traffic/views from GitHub
env:
+3 -3
View File
@@ -27,7 +27,7 @@ jobs:
name: Build x86_64
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Setup Rust
uses: dtolnay/rust-toolchain@stable
@@ -77,7 +77,7 @@ jobs:
name: Build aarch64 (arm)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Setup Rust
uses: dtolnay/rust-toolchain@stable
@@ -144,7 +144,7 @@ jobs:
github.event_name == 'push' &&
vars.HAS_GCP_CREDENTIALS == 'true'
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Download x86_64 artifact
uses: actions/download-artifact@v4
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
a11y:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with: { targets: wasm32-unknown-unknown }
+1 -1
View File
@@ -25,7 +25,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout main
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Install Rust + wasm32 target
uses: dtolnay/rust-toolchain@stable
+3 -3
View File
@@ -27,7 +27,7 @@ jobs:
target: [aarch64-apple-darwin, x86_64-apple-darwin]
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v6
@@ -82,7 +82,7 @@ jobs:
runs-on: windows-latest
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v6
@@ -130,7 +130,7 @@ jobs:
contents: write
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Download all artifacts
uses: actions/download-artifact@v4
+2 -2
View File
@@ -21,7 +21,7 @@ jobs:
runs-on: ubuntu-latest
if: github.ref_type == 'tag'
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Check firmware version.txt == tag
run: |
# Tag form: vX.Y.Z-esp32 → expect version.txt to contain X.Y.Z
@@ -70,7 +70,7 @@ jobs:
artifact_pt: partition-table-c6.bin
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Build firmware (${{ matrix.variant }})
working-directory: firmware/esp32-csi-node
+4 -4
View File
@@ -99,7 +99,7 @@ jobs:
- boundary-min
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Download QEMU artifact
uses: actions/download-artifact@v4
@@ -213,7 +213,7 @@ jobs:
name: Fuzz Testing (ADR-061 Layer 6)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Install clang
run: |
@@ -262,7 +262,7 @@ jobs:
name: NVS Matrix Generation
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Install NVS generator
run: pip install esp-idf-nvs-partition-gen
@@ -316,7 +316,7 @@ jobs:
image: espressif/idf:v5.4
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Download QEMU artifact
uses: actions/download-artifact@v4
+1 -1
View File
@@ -21,7 +21,7 @@ jobs:
name: Verify fix markers
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- uses: actions/setup-python@v6
with:
+1 -1
View File
@@ -40,7 +40,7 @@ jobs:
RUST_BACKTRACE: 1
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Install mosquitto + clients and start with allow_anonymous
run: |
+1 -1
View File
@@ -25,7 +25,7 @@ jobs:
build-and-publish:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- uses: docker/setup-buildx-action@v3
+3 -3
View File
@@ -75,7 +75,7 @@ jobs:
arch: AMD64
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
# Linux aarch64 needs QEMU for cross-build on x86_64 runners.
- name: Set up QEMU
@@ -120,7 +120,7 @@ jobs:
startsWith(github.ref, 'refs/tags/v2.')
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Install maturin
run: pip install maturin>=1.7
- name: Build sdist
@@ -143,7 +143,7 @@ jobs:
startsWith(github.ref, 'refs/tags/v1.99')
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.12'
+1 -1
View File
@@ -28,7 +28,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout main
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Stage viewer for Pages
run: |
+4 -4
View File
@@ -39,7 +39,7 @@ jobs:
- { label: 'ruflo+itar', flags: '--features ruflo,itar-unrestricted' }
- { label: 'full+train', flags: '--features full,train' }
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- name: Cache cargo
uses: actions/cache@v4
@@ -59,7 +59,7 @@ jobs:
name: clippy (-D warnings, --no-deps)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
# v2/rust-toolchain.toml pins channel "1.89" with profile "minimal" (no
# clippy). dtolnay@stable installs clippy on the floating "stable"
# toolchain, but the override makes cargo use the separate "1.89"
@@ -92,7 +92,7 @@ jobs:
name: build train_marl bin
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- name: Cache cargo
uses: actions/cache@v4
@@ -126,7 +126,7 @@ jobs:
name: ITAR / publish guard
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: publish = false is present (no accidental crates.io publish)
run: |
CARGO=v2/crates/ruview-swarm/Cargo.toml
+7 -7
View File
@@ -26,7 +26,7 @@ jobs:
steps:
- name: Checkout code
continue-on-error: true
uses: actions/checkout@v6
uses: actions/checkout@v4
with:
fetch-depth: 0
@@ -95,7 +95,7 @@ jobs:
steps:
- name: Checkout code
continue-on-error: true
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Set up Python
continue-on-error: true
@@ -162,7 +162,7 @@ jobs:
steps:
- name: Checkout code
continue-on-error: true
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Set up Docker Buildx
continue-on-error: true
@@ -243,7 +243,7 @@ jobs:
steps:
- name: Checkout code
continue-on-error: true
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Run Checkov IaC scan
continue-on-error: true
@@ -304,7 +304,7 @@ jobs:
steps:
- name: Checkout code
continue-on-error: true
uses: actions/checkout@v6
uses: actions/checkout@v4
with:
fetch-depth: 0
@@ -339,7 +339,7 @@ jobs:
steps:
- name: Checkout code
continue-on-error: true
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Set up Python
continue-on-error: true
@@ -376,7 +376,7 @@ jobs:
steps:
- name: Checkout code
continue-on-error: true
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Check security policy files
continue-on-error: true
+1 -1
View File
@@ -48,7 +48,7 @@ jobs:
name: build · push · smoke-test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
with:
submodules: recursive
+1 -1
View File
@@ -29,7 +29,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout main
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Stage demos for Pages
run: |
+1 -1
View File
@@ -13,7 +13,7 @@ jobs:
update:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
with:
submodules: true
fetch-depth: 0
+1 -1
View File
@@ -29,7 +29,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6