microsoft / microsoft/tsdoc

Unicode support in identifiers

Open
#299 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
5k
Forks
162
Avg merge
17h 24m
Merged PRs (30d)
8

Description

Just debugged my way to this comment:

// Note: In addition to letters, numbers, underscores, and dollar signs, modern ECMAScript
// also allows Unicode categories such as letters, combining marks, digits, and connector punctuation.
// These are mostly supported in all environments except IE11, so if someone wants it, we would accept
// a PR to allow them (although the test surface might be somewhat large).
StringChecks._identifierBadCharRegExp = /[^a-z0-9_$]/i;

I mark @internal identifiers with a trailing Δ so I'm hitting a few issues with this! I'm happy to attempt a PR, but I'm curious what prior work has been done towards this, and what would be expected in terms of test coverage. I believe for my usecase there may have to be changes within the Tokenizer as well ({@link} parsing is a bit broken too), seems like there's plenty room here for stepping on toes 😅

Let me know what you reckon?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with StringChecks._identifierBadCharRegExp and the Tokenizer, especially the reported {@link} parsing path. Review prior work and define the required Unicode identifier categories and test coverage before changing behavior. Done means supported Unicode identifiers parse correctly without breaking existing identifier or link parsing.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
documentation
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.