multiformats / multiformats/py-multibase

`ENCODINGS_LOOKUP` mixes string and bytes keys in the same dict

Open
#44 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
25
Forks
9
PR merge metrics
No merged PRs in 30d

Description

ENCODINGS_LOOKUP is a single dict that uses both str keys (encoding names like "base16") and bytes keys (prefix codes like b"f"). This is confusing, type-unsafe, and could cause subtle bugs if a name happens to match a prefix byte.

Problem

In multibase/multibase.py:

ENCODINGS_LOOKUP = {}
for codec in ENCODINGS:
    ENCODINGS_LOOKUP[codec.encoding] = codec  # str key: "base16"
    ENCODINGS_LOOKUP[codec.code] = codec      # bytes key: b"f"

The resulting dict has mixed key types:

ENCODINGS_LOOKUP = {
    "identity": Encoding(...),     # str
    b"\x00": Encoding(...),        # bytes
    "base2": Encoding(...),        # str
    b"0": Encoding(...),           # bytes
    "base16": Encoding(...),       # str
    b"f": Encoding(...),           # bytes
    # ...
}

Problems:

  1. Type confusion — Functions like is_encoding_supported() accept str but the dict also has bytes keys, so is_encoding_supported(b"f") returns True even though the docstring says it takes a str
  2. Potential collision — If an encoding name were a single character that matches another encoding's prefix byte, there would be a collision (currently safe but fragile)
  3. get_codec() relies on this — It checks data[:4] for emoji, then data[:1] for single-byte prefixes, both against the same mixed dict
Proposed Solution

Split into two separate dicts:

_ENCODINGS_BY_NAME: dict[str, Encoding] = {}
_ENCODINGS_BY_CODE: dict[bytes, Encoding] = {}

for codec in ENCODINGS:
    _ENCODINGS_BY_NAME[codec.encoding] = codec
    _ENCODINGS_BY_CODE[codec.code] = codec

Update all functions to use the appropriate dict:

def is_encoding_supported(encoding: str) -> bool:
    return encoding in _ENCODINGS_BY_NAME

def get_codec(data):
    data = ensure_bytes(data, "utf8")
    # Check emoji first (4-byte prefix)
    if len(data) >= 4 and data[:4] in _ENCODINGS_BY_CODE:
        return _ENCODINGS_BY_CODE[data[:4]]
    # Check single-byte prefix
    if data[:1] in _ENCODINGS_BY_CODE:
        return _ENCODINGS_BY_CODE[data[:1]]
    raise InvalidMultibaseStringError(...)

Keep ENCODINGS_LOOKUP as a backward-compatible alias if needed:

# Backward compatibility
ENCODINGS_LOOKUP = {**_ENCODINGS_BY_NAME, **_ENCODINGS_BY_CODE}
Related
  • File: multibase/multibase.py

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read multibase/multibase.py, focusing first on ENCODINGS_LOOKUP, is_encoding_supported(), and get_codec(). Trace every lookup to determine which callers need name keys versus byte-code keys, then update the separation without breaking the documented string API or codec detection. Done means the lookup responsibilities are distinct and existing behavior remains covered by the project's tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.