pybind / pybind/pybind11

[BUG]: Failed to pickle objects using protocol 0 from Python 3.9

Open
#3,745 3 comments 0 reactions 1 assignee View on GitHub

@henryiii is already working on this.

Since Feb 17, 2022.

wontfix
Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

Problem description

Serialization of objects using protocol 0 (original "human-readable" protocol) does not work anymore from Python 3.9.

Reproducible example code

core.cpp

#include <pybind11/pybind11.h>

class Foo {
 public:
  explicit Foo() {}

  inline auto getstate() const { return pybind11::tuple(); }

  inline static Foo setstate(pybind11::tuple state) {
    if (state.size() != 0) throw std::runtime_error("Invalid state!");
    return Foo();
  }
};

auto init_module(pybind11::module& m) {
  pybind11::class_<Foo>(m, "Foo")
      .def(pybind11::init<>())
      .def(pybind11::pickle(
          [](const Foo& f) { return f.getstate(); },
          [](pybind11::tuple state) { return Foo::setstate(state); }));
}

PYBIND11_MODULE(core, m) { init_module(m); }

compilation

clang++ -O3 -Wall -shared -std=c++17 -fPIC $(python3 -m pybind11 --includes) core.cpp -o core$(python3-config --extension-suffix)

foo.py

import pickle

import core


def main():
    instance = core.Foo()
    # It always works.
    result = pickle.dumps(instance, protocol=-1)
    # Starting with python 3.9, the protocol=0 throws the error:
    # terminate called after throwing an instance of 'std::runtime_error'
    #   what():  instance allocation failed: new instance has no pybind11-registered base types
    # Aborted (core dumped)
    result = pickle.dumps(instance, protocol=0)


if __name__ == '__main__':
    main()

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.