microsoft / microsoft/onnxruntime

onnx and ORT mismatch in their handling of Protobuf_USE_STATIC_LIBS and ORT can produce incorrect warnings and failures

Open
#12,867 12 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

build feature request
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 8h
Merged PRs (30d)
179

Description

Describe the bug
onnx supports setting ONNX_USE_PROTOBUF_SHARED_LIBS to ON or OFF on Windows and by extension setting Protobuf_USE_STATIC_LIBS to ON or OFF. The default is ONNX_USE_PROTOBUF_SHARED_LIBS set to OFF and Protobuf_USE_STATIC_LIBS set to ON.
onnxruntime, on the other hand, does not change Protobuf_USE_STATIC_LIBS, so it is set to OFF by default in the onnxruntime build and set to ON by default in the onnx build within onnxruntime resulting in build failures.
The onnxruntime cmake also produces a warning if Protobuf_USE_STATIC_LIBS is not set to ON regardless of what it is set to for onnx, so it will produce a warning even if it is intended to be set to OFF for both onnx and onnxruntime.
Instead, onnxruntime should use the same defaults as onnx, or possibly even set ONNX_USE_PROTOBUF_SHARED_LIBS itself.

For a related issue in onnx, which has been fixed, see: https://github.com/onnx/onnx/issues/3345.

Urgency
None

System information

To Reproduce
Build ORT and onnx together on Windows without setting Protobuf_USE_STATIC_LIBS to ON

Expected behavior
The build succeeds without any protobuf related issues or warnings.

Instead we get warnings and failures like:

LINK : warning LNK4286: symbol '??0Message@protobuf@google@@QEAA@XZ (public: __cdecl google::protobuf::Message::Message(void))' defined in 'libprotobuf.lib(wrappers.pb.cc.obj)' is imported by 'onnx_proto.lib(onnx-data.pb.cc.obj)'
LINK : warning LNK4217: symbol '??0Message@protobuf@google@@IEAA@PEAVArena@12@_N@Z (protected: __cdecl google::protobuf::Message::Message(class google::protobuf::Arena *,bool))' defined in 'libprotobuf.lib(wrappers.pb.cc.obj)' is imported by 'onnx_proto.lib(onnx-ml.pb.cc.obj)' in function '"public: virtual class onnx::ValueInfoProto * __cdecl onnx::ValueInfoProto::New(void)const " (?New@ValueInfoProto@onnx@@UEBAPEAV12@XZ)'
LINK : warning LNK4286: symbol '??0Message@protobuf@google@@IEAA@PEAVArena@12@_N@Z (protected: __cdecl google::protobuf::Message::Message(class google::protobuf::Arena *,bool))' defined in 'libprotobuf.lib(wrappers.pb.cc.obj)' is imported by 'onnx_proto.lib(onnx-data.pb.cc.obj)'
LINK : warning LNK4217: symbol '?MaybeComputeUnknownFieldsSize@Message@protobuf@google@@IEBA_K_KPEAVCachedSize@internal@23@@Z (protected: unsigned __int64 __cdecl google::protobuf::Message::MaybeComputeUnknownFieldsSize(unsigned __int64,class google::protobuf::internal::CachedSize *)const )' defined in 'libprotobuf.lib(message.cc.obj)' is imported by 'onnx_proto.lib(onnx-ml.pb.cc.obj)' in function '"public: virtual unsigned __int64 __cdecl onnx::OperatorSetIdProto::ByteSizeLong(void)const " (?ByteSizeLong@OperatorSetIdProto@onnx@@UEBA_KXZ)'
LINK : warning LNK4286: symbol '?MaybeComputeUnknownFieldsSize@Message@protobuf@google@@IEBA_K_KPEAVCachedSize@internal@23@@Z (protected: unsigned __int64 __cdecl google::protobuf::Message::MaybeComputeUnknownFieldsSize(unsigned __int64,class google::protobuf::internal::CachedSize *)const )' defined in 'libprotobuf.lib(message.cc.obj)' is imported by 'onnx_proto.lib(onnx-data.pb.cc.obj)'
LINK : warning LNK4217: symbol '??1Message@protobuf@google@@UEAA@XZ (public: virtual __cdecl google::protobuf::Message::~Message(void))' defined in 'libprotobuf.lib(wrappers.pb.cc.obj)' is imported by 'onnx_proto.lib(onnx-ml.pb.cc.obj)' in function '"int `private: static class onnx::OperatorSetIdProto * __cdecl google::protobuf::Arena::CreateMaybeMessage<class onnx::OperatorSetIdProto>(class protobuf::Arena::dtor$0 *)'::`1'::dtor$0" (?dtor$0@?0???$CreateMaybeMessage@VOperatorSetIdProto@onnx@@$$V@Arena@protobuf@google@@CAPEAVOperatorSetIdProto@onnx@@PEAV012@@Z@4HA)'
LINK : warning LNK4286: symbol '??1Message@protobuf@google@@UEAA@XZ (public: virtual __cdecl google::protobuf::Message::~Message(void))' defined in 'libprotobuf.lib(wrappers.pb.cc.obj)' is imported by 'onnx_proto.lib(onnx-data.pb.cc.obj)'
LINK : warning LNK4217: symbol '?InternalSerializeUnknownFieldsToArray@WireFormat@internal@protobuf@google@@SAPEAEAEBVUnknownFieldSet@34@PEAEPEAVEpsCopyOutputStream@io@34@@Z (public: static unsigned char * __cdecl google::protobuf::internal::WireFormat::InternalSerializeUnknownFieldsToArray(class google::protobuf::UnknownFieldSet const &,unsigned char *,class google::protobuf::io::EpsCopyOutputStream *))' defined in 'libprotobuf.lib(wire_format.cc.obj)' is imported by 'onnx_proto.lib(onnx-ml.pb.cc.obj)' in function '"public: virtual unsigned char * __cdecl onnx::OperatorSetIdProto::_InternalSerialize(unsigned char *,class google::protobuf::io::EpsCopyOutputStream *)const " (?_InternalSerialize@OperatorSetIdProto@onnx@@UEBAPEAEPEAEPEAVEpsCopyOutputStream@io@protobuf@google@@@Z)'
LINK : warning LNK4286: symbol '?InternalSerializeUnknownFieldsToArray@WireFormat@internal@protobuf@google@@SAPEAEAEBVUnknownFieldSet@34@PEAEPEAVEpsCopyOutputStream@io@34@@Z (public: static unsigned char * __cdecl google::protobuf::internal::WireFormat::InternalSerializeUnknownFieldsToArray(class google::protobuf::UnknownFieldSet const &,unsigned char *,class google::protobuf::io::EpsCopyOutputStream *))' defined in 'libprotobuf.lib(wire_format.cc.obj)' is imported by 'onnx_proto.lib(onnx-data.pb.cc.obj)'
onnx_proto.lib(onnx-ml.pb.cc.obj) : error LNK2019: unresolved external symbol "__declspec(dllimport) public: __cdecl google::protobuf::internal::ArenaStringPtr::ArenaStringPtr(class google::protobuf::internal::ExplicitlyConstructed<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > > const *)" (__imp_??0ArenaStringPtr@internal@protobuf@google@@QEAA@PEBV?$ExplicitlyConstructed@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@123@@Z) referenced in function "public: __cdecl onnx::TensorProto::TensorProto(struct google::protobuf::internal::ConstantInitialized)" (??0TensorProto@onnx@@QEAA@UConstantInitialized@internal@protobuf@google@@@Z)
onnx_proto.lib(onnx-data.pb.cc.obj) : error LNK2001: unresolved external symbol "__declspec(dllimport) public: __cdecl google::protobuf::internal::ArenaStringPtr::ArenaStringPtr(class google::protobuf::internal::ExplicitlyConstructed<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > > const *)" (__imp_??0ArenaStringPtr@internal@protobuf@google@@QEAA@PEBV?$ExplicitlyConstructed@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@123@@Z)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the ONNX Runtime CMake handling of Protobuf_USE_STATIC_LIBS and ONNX_USE_PROTOBUF_SHARED_LIBS, then reproduce the Windows build with ONNX and ONNX Runtime together. Done means the build succeeds without protobuf linker warnings or failures when the shared/static setting is intentionally consistent.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.