0xPlaygrounds / 0xPlaygrounds/rig

feat: setup benchmarking

Offen
#1,623 6 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
feat
Vorherrschende Sprache
Rust
Sterne
8.6k
Forks
959
Ø Merge
4 Std. 32 Min.
Gemergte PRs (30 T.)
117

Beschreibung

- [x] I have looked for existing issues (including closed) about this

## Feature Request

Setup bencmarking and cover CPU- and Memory-bound tasks like vector search, schema generation, document parsing, etc. (specific list of tasks can be defined separately).

### Motivation

- Validate Rig's position as a high-performance alternative to other AI Frameworks (mostly Python- and JavaScript-based)
- Tracking benchmarks will help identify bottlenecks and prevent any performance regressions

### Proposal

- Use `criterion.rs` or `divan` crate to setup benchmarking
- CI/CD integration: use CodSpeed or Bencher to run benchmarks in CI and generate a repor, attach it to PRs
- Cover any single CPU-heavy task, for example in-memory vector search or JSON schema generation
- More tasks can be covered separately based on priorities

### Alternatives

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

The issue proposes setting up benchmarking for CPU- and memory-bound tasks like vector search and schema generation. First, examine the project's Cargo.toml to see existing dependencies and the crate structure. Look for existing performance-critical modules (e.g., vector search, JSON schema generation) to understand what to benchmark. Then, research how to integrate criterion.rs or divan, and set up CI with CodSpeed or Bencher. 'Done' means benchmarks run in CI and produce reports for PRs.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
rust
Bereich
performance, tooling
Issue-Typ
Feature
Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Aktivitätsstatus
Veraltet
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
45/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.