codeforboston / codeforboston/maple

Upgrade LLM used in summarization pipeline

Open
#2,242 0 comments 0 reactions 0 assignees View on GitHub
backend enhancement
Dominant language
TypeScript
Stars
56
Forks
175
Avg merge
2d 5h
Merged PRs (30d)
13

Description

**Summary**
We are currently using an ancient GPT4 model in the automated summarization and tagging pipeline. it should be updated to a more recent model, or abstracted to automatically update to a later model.

**Work Detail**
In a perfect world, this issue would include,

* Migration to latest low cost model and confirmation there is no performance regression.
* Evaluating and/or actually migrating from OpenAI models (as originally used) to Google models (now used primarily in new feature dev)
* Implementation of automated evals to help us detect when migrating to a new LLM introduces substantive performance issues across summarization accuracy, length, reading level, or bias.
* Only apply to new summaries going forward; should not regenerate summaries already in the database.

Contributor guide

Open the contributing guide

Research direction

Start by locating the automated summarization and tagging pipeline and the current GPT4 integration. Compare the existing OpenAI usage with the Google models used in newer feature development, then define evaluation coverage for accuracy, length, reading level, and bias. Done means a newer or automatically updated model is used for new summaries without regenerating existing database records.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai, backend, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.