0xPlaygrounds / 0xPlaygrounds/rig
feat: setup benchmarking
- Lingua principale
- Rust
- Stelle
- 8.6k
- Fork
- 959
- Merge medio
- 4h 32m
- PR unite (30g)
- 117
Descrizione
- [x] I have looked for existing issues (including closed) about this
## Feature Request
Setup bencmarking and cover CPU- and Memory-bound tasks like vector search, schema generation, document parsing, etc. (specific list of tasks can be defined separately).
### Motivation
- Validate Rig's position as a high-performance alternative to other AI Frameworks (mostly Python- and JavaScript-based)
- Tracking benchmarks will help identify bottlenecks and prevent any performance regressions
### Proposal
- Use `criterion.rs` or `divan` crate to setup benchmarking
- CI/CD integration: use CodSpeed or Bencher to run benchmarks in CI and generate a repor, attach it to PRs
- Cover any single CPU-heavy task, for example in-memory vector search or JSON schema generation
- More tasks can be covered separately based on priorities
### Alternatives
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
The issue proposes setting up benchmarking for CPU- and memory-bound tasks like vector search and schema generation. First, examine the project's Cargo.toml to see existing dependencies and the crate structure. Look for existing performance-critical modules (e.g., vector search, JSON schema generation) to understand what to benchmark. Then, research how to integrate criterion.rs or divan, and set up CI with CodSpeed or Bencher. 'Done' means benchmarks run in CI and produce reports for PRs.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- rust
- Ambito
- performance, tooling
- Tipo di issue
- Funzionalità
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 45/100