huggingface / huggingface/transformers.js
[Question] How to use transformer.js in langchain
- Dominant language
- JavaScript
- Stars
- 16.3k
- Forks
- 1.2k
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 6
Description
Hi all, I'm writing a custom LLM to use transformer.js with langchain. Does a structure like this make sense? Any advice for optimizing it or best practices to apply?
Any suggestions or feedback would be greatly appreciated 😊 🚀
```
import { pipeline } from "@xenova/transformers";
import { LLM } from "langchain/llms/base";
class MyHF extends LLM {
static instance = null;
constructor(modelTask = "text2text-generation", modelName = "Xenova/LaMini-Flan-T5-783M") {
super({ maxConcurrency: 1 });
this.modelTask = modelTask;
this.modelName = modelName;
this.llmModel = MyHF.getInstance(this.modelTask, this.modelName);
}
static async getInstance(modelTask, modelName, progress_callback = null) {
if (this.instance === null) {
this.instance = pipeline(modelTask, modelName, { progress_callback });
}
return this.instance;
}
_llmType() {
return "hf";
}
async _call(prompt, options = { topk: 1 }) {
const executor = await MyHF.getInstance(this.modelTask, this.modelName);
const { generated_text } = await executor(prompt, options);
return generated_text
}
}
export default MyHF;
```
Contributor guide
Assessment
This issue has not been assessed yet.