jxnl / jxnl/instructor-classify

Add cross-provider cost tracking support

Open
#2 0 comments 0 reactions 0 assignees View on GitHub
enhancement good first issue
Dominant language
Python
Stars
37
Forks
4
PR merge metrics
No merged PRs in 30d

Description

Cost tracking currently only works for OpenAI and does not use the instructor usage tracking standard.

## Description
Extend the cost tracking functionality to work with all providers supported by Instructor, not just OpenAI. This involves integrating with Instructor's standardized usage tracking system.

## Tasks
- [ ] Review Instructor's usage tracking standard implementation
- [ ] Update cost analysis module to work with this standard format
- [ ] Add provider-specific cost calculations for Anthropic, Google, etc.
- [ ] Add tests for each provider's cost tracking
- [ ] Update documentation to reflect cross-provider cost tracking

## Expected Result
Users should be able to see accurate cost tracking regardless of which LLM provider they use. The evaluation framework should produce cost metrics for all supported providers.

## Difficulty
Medium - requires understanding of Instructor's usage tracking system and multiple provider APIs

Contributor guide

Open the contributing guide

Research direction

Start by reviewing Instructor's standardized usage tracking implementation and the existing cost analysis module. Trace how OpenAI usage is currently converted into cost metrics, then identify the provider-specific calculations and tests needed for Anthropic, Google, and other supported providers. Done means accurate cost metrics for all supported providers, with corresponding tests and updated documentation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.