PROTOCOL_H2 serialization not working with braft
- Dominant language
- C++
- Stars
- 17.6k
- Forks
- 4.1k
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 69
Description
**Describe the bug (描述bug)**
I am trying to run braft atomic application with brpc code while setting protocol as HTTP2 in channel options. Raft leader node encounters following error in HTTP2 serialization:
W1030 22:23:53.358311 78030 replicator.cpp:393] Group Atomic fail to issue RPC to 10.48.221.246:8300:0 _consecutive_error_times=1, [E1003]request_attachment must be empty when request is not NULL
While looking at the code I found following problem:
Http serialization method (SerializeHttpRequest) has logic that says that if client has provided non-null pbreq, Controller must not have non-empty attachment.
void SerializeHttpRequest(butil::IOBuf* /*not used*/,
Controller* cntl,
const google::protobuf::Message* pbreq) {
....
if (pbreq != NULL) {
// If request is not NULL, message body will be serialized proto/json,
if (!cntl->request_attachment().empty()) {
return cntl->SetFailed(EREQUEST, "request_attachment must be empty "
"when request is not NULL");
}
}
If we use brpc library with braft library, replicator code in braft (replicator.cpp), calls brpc channel method by providing both protocol buffer request and controller attachment:
void Replicator::_send_entries() {
....
std::unique_ptr request(new AppendEntriesRequest);
....
for (int i = 0; i < max_entries_size; ++i) {
prepare_entry_rc = _prepare_entry(i, &em, **&cntl->request_attachment()**);
....
}
....
stub.append_entries(**cntl.release(), request.release(),**
response.release(), done);
....
}
Basically, Raft replicator sends log entries metadata within protobuf and log entries data serialized within controller attachment.
Channel code is passing these two entities along to protocol serialization code as is:
void Channel::CallMethod(const google::protobuf::MethodDescriptor* method,
google::protobuf::RpcController* controller_base,
const google::protobuf::Message* request,
google::protobuf::Message* response,
google::protobuf::Closure* done) {
.....
// Note that here request is non-null and also cntl has
_serialize_request(&cntl->_request_buf, cntl, request);
....
}
As a fix, we either need to fix Braft replicator to be channel protocol aware or fix Brpc channel code to restructure the request before calling protocol serialization code, or fix protocol serialization to handle a condition where both protocol request and attachment are set.
**To Reproduce (复现方法)**
1. Compile braft atomic application with brpc.
2. Set protocol as 'PROTOCOL_H2' in channel options.
3. Run raft servers and client.
**Expected behavior (期望行为)**
Brpc when used with Braft with protocol set to HTTP, should work!
**Versions (各种版本)**
OS: Centos 7
Compiler: Clang
brpc: 0.9.7
protobuf: 3
**Additional context/screenshots (更多上下文/截图)**
Contributor guide
Research direction
Start with braft's replicator.cpp and the brpc entry points SerializeHttpRequest and Channel::CallMethod. Reproduce the atomic application failure with PROTOCOL_H2, then trace how the protobuf request and controller attachment reach HTTP/2 serialization. Done means the documented braft scenario works with HTTP/2 without the request_attachment error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend-api-design, distributed-systems, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100