pybind / pybind/pybind11_protobuf

proto_caster's as shared library have broken symbol visibility

Open
#160 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
67
Forks
37
PR merge metrics
No merged PRs in 30d

Description

Both native and wrapped proto_casters use several functions from proto_cast_util.cc. Unfortunately, most of these functions are not actually usable, as the pybind11 default hidden visibility is propagated to these functions.

This can be verified by analyzing the generated object file:

> readelf -a -CW ./build/CMakeFiles/pybind11_native_proto_caster.dir/pybind11_protobuf/proto_cast_util.cc.o  | grep -E 'FUNC .*GLOBAL'   592: 0000000000001630    41 FUNC    GLOBAL HIDDEN   160 pybind11_protobuf::PyBytesAsStringView(pybind11::bytes)
   593: 0000000000001660     3 FUNC    GLOBAL HIDDEN   160 pybind11_protobuf::PyProtoGetCppMessagePointer(pybind11::handle)
   594: 0000000000001670   172 FUNC    GLOBAL HIDDEN   160 pybind11_protobuf::PyProtoDescriptorFullName[abi:cxx11](pybind11::handle)
   595: 0000000000001720   158 FUNC    GLOBAL HIDDEN   160 pybind11_protobuf::PyProtoHasMatchingFullName(pybind11::handle, google::protobuf::Descriptor const*)
   748: 0000000000001ed0  1090 FUNC    GLOBAL HIDDEN   160 pybind11_protobuf::PyProtoSerializePartialToString(pybind11::handle, bool)
   829: 00000000000029e0   457 FUNC    GLOBAL HIDDEN   160 pybind11_protobuf::CProtoCopyToPyProto(google::protobuf::Message*, pybind11::handle)
   839: 0000000000003790  1089 FUNC    GLOBAL HIDDEN   160 pybind11_protobuf::AllocateCProtoFromPythonSymbolDatabase(pybind11::handle, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
   845: 0000000000003be0   104 FUNC    GLOBAL DEFAULT  160 pybind11_protobuf::InitializePybindProtoCastUtil()
   846: 0000000000003c50   274 FUNC    GLOBAL DEFAULT  160 pybind11_protobuf::ImportProtoDescriptorModule(google::protobuf::Descriptor const*)
   847: 00000000000041b0   201 FUNC    GLOBAL HIDDEN   160 pybind11_protobuf::GenericPyProtoCast(google::protobuf::Message*, pybind11::return_value_policy, pybind11::handle, bool)
   848: 0000000000004280     8 FUNC    GLOBAL HIDDEN   160 pybind11_protobuf::GenericProtoCast(google::protobuf::Message*, pybind11::return_value_policy, pybind11::handle, bool)

All method using a parameter from the pybind11:: namespace becomes "hidden", and when linking the object file into the shared library the HIDDEN symbols are omitted from the symbol table.

See https://pybind11.readthedocs.io/en/stable/faq.html#someclass-declared-with-greater-visibility-than-the-type-of-its-field-someclass-member-wattributes

google-or-tools e.g. patches the pybind11_protobuf build to create a static library (which is not affected by symbol visibility) to work around this problem:
https://github.com/google/or-tools/blob/2384738a951561bf905f3435651310841c309b4c/patches/pybind11_protobuf.patch#L32

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with proto_cast_util.cc and the pybind11_protobuf build configuration, then reproduce the reported symbol visibility with the readelf command from the issue. Check the shared-library build and confirm that the functions used by native and wrapped proto_casters remain available in its symbol table.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.