Exposing/Use of a threads switch in the CLI
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 5.3k
- Forks
- 460
- Avg merge
- 1h 58m
- Merged PRs (30d)
- 2
Description
Hi - this may be a misunderstanding on my part. It looks like both the SDKs expose the ability to pass in cpuThreads via LLMPredictionConfigInput. As far as I can see, via the GUI, thread count is passed in at model load time, and then this can be adjusted at inference time. Via the CLI, a cpuThreads-esque switch isn't available at model load time, or during inference.
I'd appreciate a nudge in the right direction - the environment is Linux (non-native headless), without GUI access.
Thanks!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the CLI's model-load and inference entry points and comparing their options with the SDKs' LLMPredictionConfigInput cpuThreads setting. Confirm how the GUI applies thread counts, then define CLI coverage for both stages and verify it works in a Linux headless environment.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- nodejs, typescript
- Domain
- cli
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100