GoogleCloudPlatform / GoogleCloudPlatform/knowledge-catalog
Clarify allowed character set for concept ids (§2) — ASCII-safe subset vs. full Unicode?
- Dominant language
- TypeScript
- Stars
- 9.2k
- Forks
- 782
- Avg merge
- 6h 36m
- Merged PRs (30d)
- 85
Description
I maintain [OKF4net](https://github.com/jchable/okf4net), an independent .NET implementation of OKF v0.2. While designing an API surface for programmatically constructing concept ids (e.g. deriving one from a free-form title), I went looking for what SPEC.md says about which characters are valid in a concept id / file path segment, and found nothing:
§2 defines: "Concept ID: The path of the concept's file within the bundle, with the .md suffix removed." — this describes what a concept id is, not what characters it may contain. I could not find a grammar, regex, whitelist, or blacklist anywhere else in SPEC.md either; §3's bundle-structure examples only show .md placeholders, and §3.1 only reserves the two literal names index.md/log.md.
That silence seems intentional given §1's "intentionally minimal" / "minimally opinionated" framing — but it leaves a real interoperability question open for independent implementations.
The concrete problem
If implementations are free to allow full Unicode in concept ids (any alphabet, no transliteration), two independent, spec-conformant implementations can disagree on whether a given bundle is valid, and a bundle built by one may be rejected — or silently mishandled — by another.
Filesystem round-tripping compounds this: macOS (HFS+/APFS in its default mode) normalizes filenames to NFD (decomposed), while Windows and most Linux filesystems preserve whatever normalization form was written (commonly NFC). A concept id containing a precomposed accented character (e.g. "café") can therefore round-trip differently depending on which OS wrote/read the bundle, which breaks concept-id equality/lookup for any implementation that compares segments byte-for-byte (ordinal) rather than normalizing first.
What would help
Some explicit guidance in §2 (even a non-normative note) on one of:
A recommended/required character set for concept id segments (e.g. an ASCII-safe subset, for maximum cross-implementation and cross-filesystem portability),
or if full Unicode is intended to be allowed, a mandated normalization form (e.g. "producers MUST write NFC-normalized segments; consumers SHOULD normalize before comparing") to close the cross-platform round-trip hazard above,
or an explicit statement that this is deliberately left to implementations/producers, if that's the intended answer — so implementers stop guessing and independently converging on different rules.
Happy to contribute a PR to SPEC.md once there's a direction, and to share what OKF4net currently does (and why) if useful as one data point.
Thanks for the good job on OKF !
Contributor guide
Research direction
Start by reading SPEC.md §2 and the bundle-structure guidance in §3, then check whether any existing tests or examples encode character or normalization rules. Confirm the maintainers’ intended interoperability policy before editing; done means SPEC.md explicitly documents the chosen character-set or normalization guidance and its scope.
Written by the indexing model from the issue text.
Assessment
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100