0xPlaygrounds / 0xPlaygrounds/rig
feat: setup benchmarking
- Vorherrschende Sprache
- Rust
- Sterne
- 8.6k
- Forks
- 959
- Ø Merge
- 4 Std. 32 Min.
- Gemergte PRs (30 T.)
- 117
Beschreibung
- [x] I have looked for existing issues (including closed) about this
## Feature Request
Setup bencmarking and cover CPU- and Memory-bound tasks like vector search, schema generation, document parsing, etc. (specific list of tasks can be defined separately).
### Motivation
- Validate Rig's position as a high-performance alternative to other AI Frameworks (mostly Python- and JavaScript-based)
- Tracking benchmarks will help identify bottlenecks and prevent any performance regressions
### Proposal
- Use `criterion.rs` or `divan` crate to setup benchmarking
- CI/CD integration: use CodSpeed or Bencher to run benchmarks in CI and generate a repor, attach it to PRs
- Cover any single CPU-heavy task, for example in-memory vector search or JSON schema generation
- More tasks can be covered separately based on priorities
### Alternatives
Beitragsleitfaden
Rechercherichtung
The issue proposes setting up benchmarking for CPU- and memory-bound tasks like vector search and schema generation. First, examine the project's Cargo.toml to see existing dependencies and the crate structure. Look for existing performance-critical modules (e.g., vector search, JSON schema generation) to understand what to benchmark. Then, research how to integrate criterion.rs or divan, and set up CI with CodSpeed or Bencher. 'Done' means benchmarks run in CI and produce reports for PRs.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- rust
- Bereich
- performance, tooling
- Issue-Typ
- Feature
- Schwierigkeit
- 4/5
- Geschätzter Aufwand
- 3-5 Tage
- Aktivitätsstatus
- Veraltet
- Klarheit
- Größtenteils klar
- Anfängerfreundlichkeit
- 45/100