huggingface / huggingface/tokenizers

`Encoding` object stub doesn't include `__len__`

Open
#1,556 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
11k
Forks
1.2k
Avg merge
3d 8h
Merged PRs (30d)
26

Description

It looks like the `Encoding` object returned by `Tokenizer.encode` has the `__len__` [dunder method](https://github.com/huggingface/tokenizers/commit/d8a73c89a70bc50e140aed131a8986a6ce9c11a5), and running `len(encoding)` works, but for some reason it's corresponding generated `.pyi` file does not. This causes type checking errors in Pyright/Pylance, even when running in non-strict mode (since this library isn't typed).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.