oracle / oracle/graalpython

[Bug]: PEP 688 `__buffer__` missing on builtin types

Open
#1,072 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
1.6k
Forks
155
Avg merge
9h 42m
Merged PRs (30d)
36

Description

Describe the bug

GraalPy registers no __buffer__ or __release_buffer__ on its managed types. So dir(b"") returns 77 names where CPython 3.12 returns 78, bytearray().__buffer__(0) raises AttributeError, and isinstance(b"", collections.abc.Buffer) is False. The buffer protocol itself works; only the PEP 688 Python-level dunders are missing.

Operating system

Linux

CPU architecture

ARM64

GraalPy version

25.2.4 (Python 3.12.8); also reproduced on 25.0.2

JDK version

No response

Context configuration

No response

Steps to reproduce
import collections.abc

print(len(dir(b"")), "__buffer__" in dir(b""))
print(isinstance(b"", collections.abc.Buffer))
bytearray(b"hello").__buffer__(0)

GraalPy 25.2.4:

77 False
False
Traceback (most recent call last):
  File "/tmp/buf.py", line 5, in <module>
    bytearray(b"hello").__buffer__(0)
AttributeError: 'bytearray' object has no attribute '__buffer__'
Expected behavior

CPython 3.12.13 raises nothing, and __buffer__(0) returns a memoryview:

78 True
True
Stack trace

Additional context

Every builtin buffer type is affected, not just bytes. "__buffer__" in dir(t) / "__release_buffer__" in dir(t):

type GraalPy 25.2.4 CPython 3.12.13
bytes False / False True / False
bytearray False / False True / True
memoryview False / False True / True
array.array False / False True / True
mmap.mmap False / False True / True

CPython's test_buffer.TestPythonBufferProtocol.test_call_builtins covers this case, calling both dunders on a bytearray (test_buffer.py#L4589-L4595). It is missing from unittest_tags/test_buffer.txt, which tags only the four cases built on user-defined classes, so CI never sees the gap.

Root Cause

Nothing in the Java core registers the dunder: git grep __buffer__ -- graalpython/com.oracle.graal.python/src returns no hits at 730597a0. bytes.__dict__ has no entry for it, and dir() only merges the MRO type dicts (ObjectBuiltins.java#L869 into TypeBuiltins.java#L1240), so it reports 77.

CPython synthesizes the wrapper in add_operators, which PyType_Ready calls to walk slotdefs and add a descriptor to tp_dict for every slot a type defines. The two slots are declared at Objects/typeobject.c#L9501-L9506, and bytes gets bf_getbuffer from bytes_as_buffer, installed as tp_as_buffer at #L2969:

static PyBufferProcs bytes_as_buffer = {
    (getbufferproc)bytes_buffer_getbuffer,
    NULL,
};

bf_releasebuffer is NULL there, so __release_buffer__ is not added. That is why the dir() difference is exactly one name.

The protocol underneath is already there: PBytesLike exports PythonBufferAccessLibrary (PBytesLike.java#L62-L64) and memoryview(b"ab") works. The C extension layer carries the same BUFSLOT definitions (cext/src/typeobject.c#L10461-L10466), but only for native types. Managed builtin types are the gap.

See PEP 688 and object.__buffer__.

Fix Suggestion

Each managed buffer type needs __buffer__, and __release_buffer__ only where the CPython counterpart defines bf_releasebuffer. bytes takes the first alone, the other four take both. Registering the dunders as builtins per type and synthesizing them at type initialization for every type exporting PythonBufferAccessLibrary, the way add_operators walks slotdefs, both seem workable; exporting that library does not by itself separate the two groups above, so which approach fits GraalPy's type setup is better judged by the maintainers.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with PBytesLike.java and the managed type initialization paths, then compare the referenced CPython test_buffer.py case with unittest_tags/test_buffer.txt. Trace how builtin type dictionaries and buffer access are exposed, including the ObjectBuiltins.java and TypeBuiltins.java paths named in the issue. Done means the tagged test passes and the listed builtin buffer types expose the same dunders as CPython.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, python
Domain
backend, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.