0xPlaygrounds / 0xPlaygrounds/rig

feat: setup benchmarking

Aperta
#1,623 6 commenti 0 reazioni 0 assegnatari Vedi su GitHub
feat
Lingua principale
Rust
Stelle
8.6k
Fork
959
Merge medio
4h 32m
PR unite (30g)
117

Descrizione

- [x] I have looked for existing issues (including closed) about this

## Feature Request

Setup bencmarking and cover CPU- and Memory-bound tasks like vector search, schema generation, document parsing, etc. (specific list of tasks can be defined separately).

### Motivation

- Validate Rig's position as a high-performance alternative to other AI Frameworks (mostly Python- and JavaScript-based)
- Tracking benchmarks will help identify bottlenecks and prevent any performance regressions

### Proposal

- Use `criterion.rs` or `divan` crate to setup benchmarking
- CI/CD integration: use CodSpeed or Bencher to run benchmarks in CI and generate a repor, attach it to PRs
- Cover any single CPU-heavy task, for example in-memory vector search or JSON schema generation
- More tasks can be covered separately based on priorities

### Alternatives

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

The issue proposes setting up benchmarking for CPU- and memory-bound tasks like vector search and schema generation. First, examine the project's Cargo.toml to see existing dependencies and the crate structure. Look for existing performance-critical modules (e.g., vector search, JSON schema generation) to understand what to benchmark. Then, research how to integrate criterion.rs or divan, and set up CI with CodSpeed or Bencher. 'Done' means benchmarks run in CI and produce reports for PRs.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
rust
Ambito
performance, tooling
Tipo di issue
Funzionalità
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
45/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.