Compare commits

..

5 Commits

Author SHA1 Message Date
rUv 88b835dd89 fix(ci): perf job gates on the real frame-budget guard, not TDD stubs (#915)
After #914 fixed collection, the perf job actually ran the suite and
exposed that test_api_throughput.py / test_inference_speed.py are TDD
red-phase stubs (every test suffixed `_should_fail_initially`) that time
a *mock that sleeps* — not a real perf signal. They carry machine-
dependent wall-clock asserts (actual_rps >= 40, batch_time < individual_time)
that are inherently flaky on shared CI runners, plus a cross-class
fixture-scope bug (`fixture 'standard_model' not found`). Result: 3 failed,
10 errored — by design, not a regression.

Forcing those green would manufacture a false signal. Instead, gate only
on test_frame_budget.py, which times the *real* CSIProcessor pipeline
against the ADR 50 ms per-frame budget (single-frame, p95/100-frames,
+Doppler) — a genuine regression guard. Verified locally: 3 passed.

The stub files remain in-repo for local TDD; they re-enter CI when their
features are implemented and the mock-timing asserts are made deterministic.
2026-06-02 18:31:55 +02:00
rUv f8f08076eb fix(ci): perf tests — use python -m pytest so src import resolves (#914)
The Performance Tests job collected 26 items then aborted with
`ModuleNotFoundError: No module named 'src'` on test_frame_budget.py,
which does `from src.core.csi_processor import CSIProcessor`. The bare
`pytest` console script does not put the cwd (archive/v1) on sys.path;
`python -m pytest` does. pytest aborts the whole session on a collection
error, so this one import masked the entire (otherwise mock-based,
self-contained) perf suite.

Verified locally: bare-script path reproduces the exact error; `-m`
resolves it and test_frame_budget.py passes 3/3. The other two files
(test_api_throughput.py mock server, test_inference_speed.py MockPoseModel
+psutil) are fully self-contained — no test hits the running server.

Closes the last red job in the v1-API CI chain (#910/#911/#913).
2026-06-02 18:12:00 +02:00
rUv 55f6a74e1e Merge pull request #913 from ruvnet/fix/ci-v1-api-perms-locust
ci(v1-api): fix gh-pages 403 + run real pytest perf suite
2026-06-02 17:36:43 +02:00
ruv b5a91c5635 ci(v1-api): install pytest, drop root --cov addopts for perf suite, ascii comment 2026-06-02 17:29:04 +02:00
ruv 308d2fc89d ci(v1-api): fix gh-pages 403 + run real perf suite — green main CI
Two more latent v1-API CI bugs surfaced once #910/#911 let the jobs reach
their later steps:

- API Documentation: openapi generation now succeeds (psutil fix), but the
  gh-pages deploy failed with HTTP 403 — the job had no `permissions` block
  and GITHUB_TOKEN is read-only by default. Add `permissions: contents:
  write`, and make the deploy `continue-on-error` (the openapi generation is
  the real validation; Pages may be disabled).
- Performance Tests: ran `locust -f tests/performance/locustfile.py`, but
  there is no locustfile — the suite is pytest (test_api_throughput.py,
  test_frame_budget.py, test_inference_speed.py). Run pytest instead, with
  working-directory: archive/v1 and MOCK_POSE_DATA=true.

ci.yml validated as well-formed YAML.
2026-06-02 17:26:39 +02:00
+29 -3
View File
@@ -265,7 +265,7 @@ jobs:
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install locust
pip install pytest # the perf suite is pytest, not locust
- name: Start application
working-directory: archive/v1
@@ -278,14 +278,37 @@ jobs:
sleep 10
- name: Run performance tests
working-directory: archive/v1
env:
MOCK_POSE_DATA: "true"
run: |
locust -f tests/performance/locustfile.py --headless --users 50 --spawn-rate 5 --run-time 60s --host http://localhost:8000
# Gate only on the genuine, deterministic perf guard:
# test_frame_budget.py times the *real* CSIProcessor pipeline against
# the ADR 50 ms per-frame budget (single-frame, p95 over 100 frames,
# +Doppler) — a true regression signal.
#
# test_api_throughput.py / test_inference_speed.py are excluded: every
# test there is a TDD red-phase stub (suffix `_should_fail_initially`)
# that times a *mock that sleeps* — meaningless as a perf signal, with
# machine-dependent wall-clock asserts (e.g. `actual_rps >= 40`,
# `batch_time < individual_time`) that are inherently flaky on shared
# CI runners, plus a cross-class fixture-scope bug. Forcing them green
# would be manufacturing a false signal; they stay in-repo for local
# TDD but do not gate CI until the underlying features are implemented.
#
# `python -m pytest` (not the bare `pytest` script) puts the cwd
# (archive/v1) on sys.path so `from src.core...` resolves — the bare
# script omits cwd and raises ModuleNotFoundError: No module named 'src'.
# -o addopts="" drops the root pyproject's --cov/--cov-fail-under=100.
python -m pytest tests/performance/test_frame_budget.py \
-o addopts="" -v --junitxml=perf-junit.xml
- name: Upload performance results
if: always()
uses: actions/upload-artifact@v4
with:
name: performance-results
path: locust_report.html
path: archive/v1/perf-junit.xml
# Docker Build and Test
# NOTE: the canonical Docker build for the sensing-server is now
@@ -371,6 +394,8 @@ jobs:
runs-on: ubuntu-latest
needs: [docker-build]
if: github.ref == 'refs/heads/main'
permissions:
contents: write # gh-pages deploy needs write (GITHUB_TOKEN is read-only by default -> 403)
steps:
- name: Checkout code
uses: actions/checkout@v4
@@ -400,6 +425,7 @@ jobs:
- name: Deploy to GitHub Pages
uses: peaceiris/actions-gh-pages@v4
continue-on-error: true # openapi generation above is the real validation; deploy is best-effort (Pages may be disabled)
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./docs