aRustyDev / aRustyDev/ccgram

Integrate lookup + neural inference path

Open
#12 0 comments 0 reactions 0 assignees View on GitHub
infra model
Dominant language
Python
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

## Summary
Combine the n-gram lookup and neural fallback into a unified inference pipeline.

## Success Criteria
- [ ] Single entry point for generation
- [ ] Automatic routing based on confidence gate
- [ ] Seamless context handoff from lookup to neural
- [ ] Configurable routing strategy
- [ ] Logging of routing decisions for analysis
- [ ] Graceful degradation if neural model unavailable

## Inference Flow
\`\`\`
Input → Tokenize → Check n-gram confidence
↓ high confidence ↓ low confidence
Lookup Generate Neural Generate
↓ ↓
└────── Merge ────────┘

Detokenize → Output
\`\`\`

## Interface
\`\`\`python
class HybridGenerator:
def __init__(self, ngram_tables, neural_model, threshold=0.8):
...

def generate(self, input_context: str) -> GenerationResult:
"""Returns generated text and metadata (which path used, confidence, etc.)"""
...
\`\`\`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.