mirror of
https://github.com/ruvnet/RuView
synced 2026-08-08 20:11:43 +00:00
ruv
9ad550d95f
Candle native port — wifi-densepose-occworld-candle v0.3.0: - config.rs: OccWorldConfig (14 params matching occworld.py) - vqvae.rs: ClassEmbedding(18→64), VQCodebook(512×512, squared-L2), QuantConv/PostQuantConv(1×1 Conv2d), fold_3d_to_2d helpers ResNet encoder/decoder are documented stubs (Phase 5 checkpoint pending) - transformer.rs: full Candle MHA transformer (2 layers, temporal+spatial cross-attention, FFN, pre-norm residuals) - inference.rs: OccWorldCandle::dummy() + ::load() + predict() InferenceOutput: sem_pred(1,15,200,200,16) + trajectory_priors - 14/14 tests pass (12 lib + 2 doctests) GCP GPU scripts — scripts/gcp/: - provision_training.sh: a2-highgpu-8g (8×A100 40GB) for Phase 5 retraining - run_training.sh: rsync + torchrun 8-GPU train + checkpoint download - provision_cosmos.sh: a2-ultragpu-1g (A100 80GB) for Cosmos evaluation - cosmos_eval.sh: run Cosmos-Transfer2.5 inference, download results - teardown.sh: safe checkpoint download + instance delete Co-Authored-By: claude-flow <ruv@ruv.net>
53 lines
2.3 KiB
Rust
53 lines
2.3 KiB
Rust
//! `wifi-densepose-occworld-candle` — OccWorld TransVQVAE inference in Candle.
|
|
//!
|
|
//! Ports the 72.4 M-parameter OccWorld world model (VQVAE tokeniser +
|
|
//! autoregressive transformer) from Python to native Rust using the
|
|
//! Hugging Face Candle framework. The goal is to eliminate the
|
|
//! 208 ms Python/IPC overhead of the existing `wifi-densepose-worldmodel`
|
|
//! bridge and enable tight integration with the streaming engine.
|
|
//!
|
|
//! ## Module structure
|
|
//!
|
|
//! | Module | Contents |
|
|
//! |-----------------|-------------------------------------------------------|
|
|
//! | `config` | `OccWorldConfig` — hyper-parameters |
|
|
//! | `error` | `OccWorldError` — unified error enum |
|
|
//! | `vqvae` | Class embedding, VQ codebook, quant convolutions |
|
|
//! | `transformer` | Autoregressive transformer (`PlanUAutoRegTransformer`) |
|
|
//! | `model` | SafeTensors weight loading + key mapping |
|
|
//! | `inference` | `OccWorldCandle` end-to-end inference engine |
|
|
//!
|
|
//! ## Implementation status
|
|
//!
|
|
//! The VQVAE encoder/decoder ResNet blocks are **stubs** that return random
|
|
//! tensors of the correct shape. All other components (class embedding,
|
|
//! VQ codebook, quant/post-quant convolutions, transformer, trajectory
|
|
//! extraction) are fully implemented. The stubs will be replaced in Phase 5
|
|
//! once the SafeTensors checkpoint is available.
|
|
//!
|
|
//! ## Usage
|
|
//!
|
|
//! ```no_run
|
|
//! use wifi_densepose_occworld_candle::inference::OccWorldCandle;
|
|
//! use wifi_densepose_occworld_candle::config::OccWorldConfig;
|
|
//! use candle_core::{Device, DType, Tensor};
|
|
//! use std::path::Path;
|
|
//!
|
|
//! let cfg = OccWorldConfig::default();
|
|
//! let engine = OccWorldCandle::dummy(cfg, Device::Cpu).expect("dummy init");
|
|
//! let past = Tensor::zeros((1, 15, 200, 200, 16), DType::U8, &Device::Cpu).unwrap();
|
|
//! let out = engine.predict(&past).expect("predict");
|
|
//! println!("predicted {} frames in {:.1} ms", out.sem_pred.dim(1).unwrap(), out.inference_ms);
|
|
//! ```
|
|
|
|
pub mod config;
|
|
pub mod error;
|
|
pub mod inference;
|
|
pub mod model;
|
|
pub mod transformer;
|
|
pub mod vqvae;
|
|
|
|
pub use config::OccWorldConfig;
|
|
pub use error::OccWorldError;
|
|
pub use inference::{InferenceOutput, OccWorldCandle, TrajectoryWaypoint};
|