huggingface / huggingface/transformers.js
How to use xenova/transformers in VSCode Extension
- Dominant language
- JavaScript
- Stars
- 16.3k
- Forks
- 1.2k
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 6
Description
Hey guys! I am trying to use xenova/transformers in CodeStory, we roll a vscode extension as well and I am hitting issues with trying to get the import working, here's every flavor of importing the library which I have tried to date.
```
const TransformersApi = Function('return import("@xenova/transformers")')();
const { pipeline, env } = await TransformersApi;
```
```
const { pipeline, env } = await import('@xenova/transformers')
```
```
const TransformersApi = require('@xenova/transformers');
const { pipeline, env } = await TransformersApi;
```
I think the crux of the issue is the node environment which VSCode uses which does not allow any of these to work, and I keep getting the deaded:
```
Error [ERR_REQUIRE_ESM]: require() of ES Module /Applications/Aide.app/Contents/Resources/app/extensions/codestory/node_modules/@xenova/transformers/src/transformers.js from /Applications/Aide.app/Contents/Resources/app/extensions/codestory/out/llm/embeddings/sentenceTransformers.js not supported.
Instead change the require of transformers.js in /Applications/Aide.app/Contents/Resources/app/extensions/codestory/out/llm/embeddings/sentenceTransformers.js to a dynamic import() which is available in all CommonJS modules.
```
after checking the js code which is generated, it ends up including the require word:
```
__importStar(require('@xenova/transformers'))
```
when I used the first option which was a function I got a very weird error btw:
```
[Extension Host] TypeError: A dynamic import callback was not specified.
at new NodeError (node:internal/errors:399:5)
at importModuleDynamicallyCallback (node:internal/process/esm_loader:39:9)
at eval (eval at (/Applications/Aide.app/Contents/Resources/app/extensions/codestory/out/llm/embeddings/sentenceTransformers.js:46:41), :3:1)
```
this is mostly comping from the node version which VSCode uses itself.
Do you guys have any suggestions on what I can do about this? Thanks!
Contributor guide
Assessment
This issue has not been assessed yet.