huggingface / huggingface/transformers.js

[Question] How to use transformer.js in langchain

Open
#396 2 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
JavaScript
Stars
16.3k
Forks
1.2k
Avg merge
6d 2h
Merged PRs (30d)
6

Description

Hi all, I'm writing a custom LLM to use transformer.js with langchain. Does a structure like this make sense? Any advice for optimizing it or best practices to apply?

Any suggestions or feedback would be greatly appreciated 😊 🚀

```
import { pipeline } from "@xenova/transformers";
import { LLM } from "langchain/llms/base";

class MyHF extends LLM {
static instance = null;

constructor(modelTask = "text2text-generation", modelName = "Xenova/LaMini-Flan-T5-783M") {
super({ maxConcurrency: 1 });
this.modelTask = modelTask;
this.modelName = modelName;
this.llmModel = MyHF.getInstance(this.modelTask, this.modelName);
}

static async getInstance(modelTask, modelName, progress_callback = null) {
if (this.instance === null) {
this.instance = pipeline(modelTask, modelName, { progress_callback });
}
return this.instance;
}

_llmType() {
return "hf";
}

async _call(prompt, options = { topk: 1 }) {
const executor = await MyHF.getInstance(this.modelTask, this.modelName);
const { generated_text } = await executor(prompt, options);
return generated_text
}
}

export default MyHF;
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.