fwcd / fwcd/d2

Investigate integrating a local language model

Open
#165 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Swift
Stars
17
Forks
5
PR merge metrics
No merged PRs in 30d

Description

Highly quantized language models that can run locally are getting more and more popular with even [Chrome shipping a Gemini Nano model](https://developer.chrome.com/docs/ai/built-in) in their latest canary builds. Models like [Phi-3-mini](https://github.com/microsoft/Phi-3CookBook) already achieve impressive performance for being comparatively small and [support cross-platform inference](https://github.com/microsoft/Phi-3CookBook/blob/fb35e596083b05f35ddc73bf0d6936effb67f16f/md/03.Inference/Rust_Inference.md) using [a Rust library named `candle`](https://github.com/huggingface/candle).

It would be cool if we could bundle such a model with D2, e.g. as a command and/or as a [`Conversator`](https://github.com/fwcd/d2/blob/71acc52c7ede483eeee6b2571bfb5a88d3b653c2/Sources/D2Commands/Misc/Conversator.swift).

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading Sources/D2Commands/Misc/Conversator.swift and the linked candle Rust inference example. Investigate how a highly quantized local model such as Phi-3-mini could be bundled with D2 and whether the integration belongs in a command, a Conversator, or both. Done requires a defined integration approach and a working local-model feature.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, swift
Domain
ai
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.