hackclub / hackclub/ai

Flex processing support for OpenAI models

Open
#85 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
133
Forks
55
PR merge metrics
No merged PRs in 30d

Description

Would be cool to add support for OpenAI's Flex Processing tier when making requests to OpenAI models.

Flex processing is a service tier OpenAI offers where requests are processed with variable latency in exchange for lower costs([around 50%.](https://developers.openai.com/api/docs/pricing?latest-pricing=flex) If capacity isn't available, the request fails fast rather than queuing, which could be a worthy tradeoff? Or maybe just send the request again to the non flex? idk

[openai docs](https://developers.openai.com/api/docs/guides/flex-processing) for more info!
## why this should be a thing
I think this would be useful as OA's models are slowly getting more and more expensive over time ([gpt 5 mini ](https://openrouter.ai/openai/gpt-5-mini)to [gpt 5.4 mini](https://openrouter.ai/openai/gpt-5.4-mini)), making us restricted to mostly cheaper and OSS models with the $4 daily limit(which isnt an issue, im not being ungrateful yall doing a excellent job lol). With flex, gpt 5.4 mini is around the same price as gpt 5 mini (without flex) so basically its a almost free upgrade with a much higher intelligence. Also, plenty of real-world use cases don't actually need fast responses such as news summariser that sends you a morning digest; this doesnt need low latency and would benefit from the cost decrease.

Should be a relatively low-effort change too as you pass `service_tier: "flex"` in the request body

would be happy to create a pr if its worthwhile
also let me know if i can plead my case better!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.