python / python/cpython

C API: Soft deprecate functions modifying str objects (PyUnicodeObject)

Open
#157,710 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

interpreter-core topic-C-API type-feature
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Python 3.14 added PyUnicodeWriter C API which is a safe way to create a Unicode object. IMO it's now time to soft deprecate the legacy way to create a str object using PyUnicode_New(): deprecate functions like PyUnicode_CopyCharacters() and PyUnicode_Resize().

The PyUnicode_New() API is more fragile. It's possible that a str object is "used" before it's fully initialized, which make following functions (modifying the str) fail or crash. It can also be exposed in Python by mistake before it's fully initialized.

PyUnicodeWriter has a different design than PyUnicode_New(): it's designed to append multiple strings, whereas PyUnicode_New() gives a direct access to an array of characters in a specific format (Py_UCS1, Py_UCS2 or Py_UCS4: 1, 2 or 4 bytes per character).

If PyUnicodeWriter doesn't fit well with existing code, it's possible to allocate an array of characters, write into this array, and then call PyUnicode_FromKindAndData() to create a str object.

Another option is to create a UTF-8 encode string and then call PyUnicode_FromString().

Linked PRs
  • gh-157711
  • gh-157712
  • gh-157715

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review the linked PRs gh-157711, gh-157712, and gh-157715 first, then inspect the PyUnicode_New(), PyUnicode_CopyCharacters(), and PyUnicode_Resize() C API entry points. Done means reaching agreement on which legacy functions receive soft-deprecation treatment and documenting or testing the resulting API behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, python
Domain
api
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.