Support int32 in substring dedup

I'm using a tokenizer with > 100k vocab size, so the token id overflow as it is stored in uint16. I'm wondering if we can add support for int32? Is it possible to simply change the type or is there other places that need to be changed?

Start at the substring dedup implementation and trace where token IDs are stored as uint16, including the tokenizer path mentioned in the issue. Done means int32 IDs work for vocabularies over 100k without overflow; the issue does not name specific files or tests, so identify the affected coverage while tracing the implementation.

Written by the indexing model from the issue text.