mirror of
https://github.com/ruvnet/RuView
synced 2026-08-11 20:41:44 +00:00
feat: vendor midstream and sublinear-time-solver libraries
Add ruvnet/midstream (AIMDS real-time inference) and ruvnet/sublinear-time-solver (sublinear optimization algorithms) as vendored dependencies under vendor/. Co-Authored-By: claude-flow <ruv@ruv.net>
This commit is contained in:
ruv
+526
@@ -0,0 +1,526 @@
|
||||
# MidStream Real-Time Dashboard
|
||||
|
||||
**Created by rUv**
|
||||
|
||||
Comprehensive real-time dashboard for monitoring and analyzing LLM streaming with advanced temporal pattern detection, attractor analysis, and multi-modal stream introspection.
|
||||
|
||||
## 🌟 Features
|
||||
|
||||
### Real-Time Monitoring
|
||||
- **Text Streaming**: Process and analyze text messages in real-time
|
||||
- **Audio Streaming**: Monitor audio streams with transcription support
|
||||
- **Video Streaming**: Analyze video streams with object detection
|
||||
- **Multi-Modal**: Simultaneous handling of text, audio, and video streams
|
||||
|
||||
### Advanced Analysis
|
||||
- **Temporal Pattern Detection**: Identify patterns in conversation flows
|
||||
- **Attractor Analysis**: Detect fixed points, periodic cycles, and chaotic behavior
|
||||
- **Lyapunov Exponents**: Measure system stability and chaos
|
||||
- **Meta-Learning**: Adaptive learning from conversation patterns
|
||||
- **Behavior Classification**: Classify system behavior as stable, unstable, or chaotic
|
||||
|
||||
### Performance Metrics
|
||||
- **Real-Time FPS**: Frames per second monitoring
|
||||
- **Latency Tracking**: Message processing latency
|
||||
- **Stream Metrics**: Bandwidth, bitrate, and chunk statistics
|
||||
- **Token Counting**: Track LLM token usage
|
||||
- **Uptime Monitoring**: System uptime and health
|
||||
|
||||
### Streaming Support
|
||||
- **WebSocket**: Real-time bidirectional streaming
|
||||
- **Server-Sent Events (SSE)**: Unidirectional event streaming
|
||||
- **WebRTC**: Peer-to-peer audio/video streaming
|
||||
- **RTMP**: Real-Time Messaging Protocol support
|
||||
- **HLS**: HTTP Live Streaming support
|
||||
|
||||
## 📦 Installation
|
||||
|
||||
```bash
|
||||
cd npm
|
||||
npm install
|
||||
npm run build:ts
|
||||
```
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### Basic Dashboard
|
||||
|
||||
```typescript
|
||||
import { MidStreamDashboard } from 'midstream-cli';
|
||||
|
||||
const dashboard = new MidStreamDashboard();
|
||||
dashboard.start(100); // Refresh every 100ms
|
||||
|
||||
// Process a message
|
||||
dashboard.processMessage('Hello, world!', 5);
|
||||
|
||||
// Process streaming data
|
||||
const audioData = Buffer.alloc(1024);
|
||||
dashboard.processStream('audio-1', audioData, 'audio');
|
||||
```
|
||||
|
||||
### Interactive Dashboard
|
||||
|
||||
```typescript
|
||||
import { InteractiveDashboard } from 'midstream-cli';
|
||||
|
||||
const dashboard = new InteractiveDashboard();
|
||||
dashboard.startInteractive();
|
||||
```
|
||||
|
||||
### Run Demo
|
||||
|
||||
```bash
|
||||
# Full demo with all features
|
||||
npm run demo
|
||||
|
||||
# Text-only demo
|
||||
npm run demo:text
|
||||
|
||||
# Audio streaming demo
|
||||
npm run demo:audio
|
||||
|
||||
# Video streaming demo
|
||||
npm run demo:video
|
||||
|
||||
# OpenAI Realtime API demo
|
||||
npm run demo:openai
|
||||
```
|
||||
|
||||
## 📊 Dashboard Components
|
||||
|
||||
### System Metrics Panel
|
||||
```
|
||||
Messages Processed: 150
|
||||
Total Tokens: 2,340
|
||||
FPS: 60
|
||||
Latency: 12ms
|
||||
Uptime: 0h 5m 23s
|
||||
```
|
||||
|
||||
### Temporal Analysis Panel
|
||||
```
|
||||
Attractor Type: PERIODIC
|
||||
Lyapunov Exp: -0.0234
|
||||
Stability: STABLE
|
||||
Chaos: ORDERED
|
||||
Avg Reward: 0.847
|
||||
```
|
||||
|
||||
### Pattern Detection Panel
|
||||
```
|
||||
• greeting (95%)
|
||||
• question (87%)
|
||||
• acknowledgment (92%)
|
||||
• follow-up (78%)
|
||||
• closing (88%)
|
||||
```
|
||||
|
||||
### Streaming Status Panel
|
||||
```
|
||||
Audio: ● ACTIVE
|
||||
Video: ● ACTIVE
|
||||
Streams: 3 active
|
||||
```
|
||||
|
||||
### Stream Metrics Panel
|
||||
```
|
||||
audio-stream-1 (audio): 150 chunks, 1.5 MB, 45.2 KB/s
|
||||
video-stream-1 (video): 1800 frames, 180 MB, 3.2 MB/s
|
||||
```
|
||||
|
||||
## 🎥 Restream Integration
|
||||
|
||||
### WebRTC Streaming
|
||||
|
||||
```typescript
|
||||
import { RestreamClient } from 'midstream-cli';
|
||||
|
||||
const client = new RestreamClient({
|
||||
webrtcSignaling: 'wss://signaling.example.com',
|
||||
enableTranscription: true,
|
||||
enableObjectDetection: true,
|
||||
frameRate: 30,
|
||||
resolution: '1920x1080'
|
||||
});
|
||||
|
||||
// Listen for frames
|
||||
client.on('frame', (frame) => {
|
||||
console.log(`Frame ${frame.frameNumber}: ${frame.width}x${frame.height}`);
|
||||
});
|
||||
|
||||
// Listen for audio
|
||||
client.on('audio', (audio) => {
|
||||
console.log(`Audio chunk: ${audio.sampleRate}Hz, ${audio.channels}ch`);
|
||||
});
|
||||
|
||||
// Listen for transcriptions
|
||||
client.on('transcription', (text) => {
|
||||
console.log(`Transcription: ${text}`);
|
||||
});
|
||||
|
||||
// Connect
|
||||
await client.connectWebRTC();
|
||||
```
|
||||
|
||||
### RTMP Streaming
|
||||
|
||||
```typescript
|
||||
const client = new RestreamClient({
|
||||
rtmpUrl: 'rtmp://live.example.com/live',
|
||||
streamKey: 'your-stream-key',
|
||||
enableTranscription: true
|
||||
});
|
||||
|
||||
await client.connectRTMP();
|
||||
```
|
||||
|
||||
### HLS Streaming
|
||||
|
||||
```typescript
|
||||
const client = new RestreamClient({
|
||||
enableTranscription: true
|
||||
});
|
||||
|
||||
await client.connectHLS('https://example.com/stream.m3u8');
|
||||
```
|
||||
|
||||
### Stream Analysis
|
||||
|
||||
```typescript
|
||||
// Get real-time analysis
|
||||
const analysis = client.getAnalysis();
|
||||
|
||||
console.log(`
|
||||
Frames: ${analysis.frameCount}
|
||||
Audio Chunks: ${analysis.audioChunks}
|
||||
FPS: ${analysis.fps}
|
||||
Bitrate: ${analysis.bitrate} Kbps
|
||||
Patterns: ${analysis.patterns.length}
|
||||
`);
|
||||
```
|
||||
|
||||
## 🤖 OpenAI Realtime Integration
|
||||
|
||||
```typescript
|
||||
import { MidStreamDashboard } from 'midstream-cli';
|
||||
import { OpenAIRealtimeClient } from 'midstream-cli';
|
||||
|
||||
const dashboard = new MidStreamDashboard();
|
||||
dashboard.start();
|
||||
|
||||
const client = new OpenAIRealtimeClient({
|
||||
apiKey: process.env.OPENAI_API_KEY,
|
||||
model: 'gpt-4o-realtime-preview-2024-10-01',
|
||||
voice: 'alloy'
|
||||
});
|
||||
|
||||
// Connect dashboard to OpenAI events
|
||||
client.on('response.text.delta', (delta) => {
|
||||
dashboard.processMessage(delta, delta.length);
|
||||
});
|
||||
|
||||
client.on('response.audio.delta', (delta) => {
|
||||
const audio = Buffer.from(delta, 'base64');
|
||||
dashboard.processStream('openai-audio', audio, 'audio');
|
||||
});
|
||||
|
||||
await client.connect();
|
||||
client.sendText('Analyze this conversation...');
|
||||
```
|
||||
|
||||
## 🧪 Testing with Stream Simulator
|
||||
|
||||
```typescript
|
||||
import { StreamSimulator } from 'midstream-cli';
|
||||
|
||||
const simulator = new StreamSimulator(30); // 30 FPS
|
||||
|
||||
simulator.start(
|
||||
(frame) => {
|
||||
// Process video frame
|
||||
dashboard.processStream('video', frame.data, 'video');
|
||||
},
|
||||
(audio) => {
|
||||
// Process audio chunk
|
||||
dashboard.processStream('audio', audio.data, 'audio');
|
||||
}
|
||||
);
|
||||
|
||||
// Run for 60 seconds
|
||||
setTimeout(() => simulator.stop(), 60000);
|
||||
```
|
||||
|
||||
## 🔧 Configuration
|
||||
|
||||
### Dashboard Options
|
||||
|
||||
```typescript
|
||||
const dashboard = new MidStreamDashboard();
|
||||
|
||||
// Custom agent configuration
|
||||
const agent = dashboard.getAgent();
|
||||
// Agent is pre-configured with:
|
||||
// - maxHistory: 1000
|
||||
// - embeddingDim: 3
|
||||
// - schedulingPolicy: 'EDF'
|
||||
```
|
||||
|
||||
### Refresh Rate
|
||||
|
||||
```typescript
|
||||
// Fast refresh (100ms) - smooth but CPU intensive
|
||||
dashboard.start(100);
|
||||
|
||||
// Medium refresh (500ms) - balanced
|
||||
dashboard.start(500);
|
||||
|
||||
// Slow refresh (1000ms) - low CPU usage
|
||||
dashboard.start(1000);
|
||||
```
|
||||
|
||||
## 📈 Advanced Usage
|
||||
|
||||
### Custom Pattern Analysis
|
||||
|
||||
```typescript
|
||||
const agent = dashboard.getAgent();
|
||||
|
||||
// Detect custom pattern
|
||||
const pattern = ['greeting', 'question', 'answer'];
|
||||
const positions = agent.detectPattern(conversation, pattern);
|
||||
|
||||
console.log(`Pattern found at positions: ${positions}`);
|
||||
```
|
||||
|
||||
### Sequence Comparison
|
||||
|
||||
```typescript
|
||||
// Compare two conversation sequences
|
||||
const similarity = agent.compareSequences(
|
||||
sequence1,
|
||||
sequence2,
|
||||
'dtw' // Dynamic Time Warping
|
||||
);
|
||||
|
||||
console.log(`Similarity: ${(similarity * 100).toFixed(1)}%`);
|
||||
```
|
||||
|
||||
### Behavior Analysis
|
||||
|
||||
```typescript
|
||||
// Analyze system behavior
|
||||
const rewards = [0.8, 0.85, 0.83, 0.87, 0.84];
|
||||
const analysis = agent.analyzeBehavior(rewards);
|
||||
|
||||
console.log(`
|
||||
Attractor: ${analysis.attractorType}
|
||||
Lyapunov: ${analysis.lyapunovExponent}
|
||||
Stable: ${analysis.isStable}
|
||||
Chaotic: ${analysis.isChaotic}
|
||||
`);
|
||||
```
|
||||
|
||||
## 🎨 Customization
|
||||
|
||||
### Color Themes
|
||||
|
||||
The dashboard uses chalk for colorful console output:
|
||||
- **Cyan**: Headers and titles
|
||||
- **Green**: Success states and positive metrics
|
||||
- **Yellow**: Warnings and neutral metrics
|
||||
- **Red**: Errors and negative states
|
||||
- **Magenta**: Patterns and detections
|
||||
- **Gray**: Secondary information
|
||||
|
||||
### Custom Metrics
|
||||
|
||||
```typescript
|
||||
// Get current state
|
||||
const state = dashboard.getState();
|
||||
|
||||
// Modify or extend as needed
|
||||
console.log(`
|
||||
Messages: ${state.messageCount}
|
||||
Patterns: ${state.patternsDetected.length}
|
||||
Attractor: ${state.attractorType}
|
||||
`);
|
||||
```
|
||||
|
||||
## 🔐 Security Considerations
|
||||
|
||||
### API Keys
|
||||
Always use environment variables for API keys:
|
||||
|
||||
```bash
|
||||
# .env file
|
||||
OPENAI_API_KEY=sk-...
|
||||
AGENTIC_FLOW_API_KEY=...
|
||||
```
|
||||
|
||||
### Stream Authentication
|
||||
When using WebRTC or RTMP, ensure proper authentication:
|
||||
|
||||
```typescript
|
||||
const client = new RestreamClient({
|
||||
rtmpUrl: 'rtmps://secure.example.com/live', // Use RTMPS
|
||||
streamKey: process.env.STREAM_KEY,
|
||||
apiKey: process.env.API_KEY
|
||||
});
|
||||
```
|
||||
|
||||
### Rate Limiting
|
||||
Implement rate limiting for API calls:
|
||||
|
||||
```typescript
|
||||
// Limit message processing rate
|
||||
let lastProcess = 0;
|
||||
const minInterval = 100; // ms
|
||||
|
||||
function processWithRateLimit(message: string) {
|
||||
const now = Date.now();
|
||||
if (now - lastProcess >= minInterval) {
|
||||
dashboard.processMessage(message, message.length);
|
||||
lastProcess = now;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 📊 Performance Optimization
|
||||
|
||||
### Buffer Management
|
||||
The dashboard automatically manages buffers:
|
||||
- Recent messages: Last 5 messages
|
||||
- Frame buffer: Last 100 frames
|
||||
- Audio buffer: Last 100 chunks
|
||||
|
||||
### Memory Usage
|
||||
Monitor and optimize memory usage:
|
||||
|
||||
```typescript
|
||||
// Periodic cleanup
|
||||
setInterval(() => {
|
||||
if (global.gc) {
|
||||
global.gc();
|
||||
}
|
||||
}, 60000);
|
||||
```
|
||||
|
||||
### CPU Optimization
|
||||
Adjust refresh rate based on CPU usage:
|
||||
|
||||
```typescript
|
||||
// Start with fast refresh
|
||||
dashboard.start(100);
|
||||
|
||||
// Reduce if CPU is high
|
||||
if (cpuUsage > 80) {
|
||||
dashboard.stop();
|
||||
dashboard.start(500);
|
||||
}
|
||||
```
|
||||
|
||||
## 🐛 Troubleshooting
|
||||
|
||||
### Dashboard Not Updating
|
||||
- Check refresh rate is appropriate
|
||||
- Verify messages are being processed
|
||||
- Check console for errors
|
||||
|
||||
### Stream Not Connecting
|
||||
- Verify URL and credentials
|
||||
- Check network connectivity
|
||||
- Review firewall settings
|
||||
|
||||
### High CPU Usage
|
||||
- Increase refresh interval
|
||||
- Reduce stream resolution
|
||||
- Disable unnecessary features
|
||||
|
||||
### Memory Leaks
|
||||
- Check buffer sizes
|
||||
- Verify event listeners are cleaned up
|
||||
- Monitor with `process.memoryUsage()`
|
||||
|
||||
## 📚 API Reference
|
||||
|
||||
### MidStreamDashboard
|
||||
|
||||
#### Constructor
|
||||
```typescript
|
||||
new MidStreamDashboard()
|
||||
```
|
||||
|
||||
#### Methods
|
||||
- `start(refreshRate: number): void` - Start dashboard
|
||||
- `stop(): void` - Stop dashboard
|
||||
- `processMessage(message: string, tokens?: number): void` - Process text message
|
||||
- `processStream(streamId: string, data: Buffer, type: 'audio' | 'video' | 'text'): void` - Process stream data
|
||||
- `getAgent(): MidStreamAgent` - Get underlying agent
|
||||
- `getState(): DashboardState` - Get current state
|
||||
|
||||
### RestreamClient
|
||||
|
||||
#### Constructor
|
||||
```typescript
|
||||
new RestreamClient(config: RestreamConfig)
|
||||
```
|
||||
|
||||
#### Methods
|
||||
- `connectRTMP(): Promise<void>` - Connect to RTMP stream
|
||||
- `connectWebRTC(): Promise<void>` - Connect to WebRTC stream
|
||||
- `connectHLS(url: string): Promise<void>` - Connect to HLS stream
|
||||
- `disconnect(): void` - Disconnect from stream
|
||||
- `getAnalysis(): StreamAnalysis` - Get stream analysis
|
||||
- `getStats()` - Get stream statistics
|
||||
|
||||
#### Events
|
||||
- `connected` - Stream connected
|
||||
- `disconnected` - Stream disconnected
|
||||
- `frame` - Video frame received
|
||||
- `audio` - Audio chunk received
|
||||
- `transcription` - Audio transcribed
|
||||
- `objects_detected` - Objects detected in frame
|
||||
- `error` - Error occurred
|
||||
|
||||
### StreamSimulator
|
||||
|
||||
#### Constructor
|
||||
```typescript
|
||||
new StreamSimulator(frameRate: number)
|
||||
```
|
||||
|
||||
#### Methods
|
||||
- `start(onFrame, onAudio?): void` - Start simulation
|
||||
- `stop(): void` - Stop simulation
|
||||
- `getFrameNumber(): number` - Get current frame number
|
||||
|
||||
## 🤝 Contributing
|
||||
|
||||
Contributions are welcome! Please:
|
||||
1. Fork the repository
|
||||
2. Create a feature branch
|
||||
3. Make your changes
|
||||
4. Add tests
|
||||
5. Submit a pull request
|
||||
|
||||
## 📄 License
|
||||
|
||||
MIT License - see LICENSE file for details
|
||||
|
||||
## 👨💻 Author
|
||||
|
||||
**Created by rUv**
|
||||
|
||||
For questions or support, please open an issue on GitHub.
|
||||
|
||||
## 🙏 Acknowledgments
|
||||
|
||||
- OpenAI for Realtime API
|
||||
- WebRTC community
|
||||
- Node.js community
|
||||
- All contributors
|
||||
|
||||
---
|
||||
|
||||
**MidStream Dashboard** - Real-time introspection for the AI age
|
||||
Reference in New Issue
Block a user