Why not using RDMA WRITE( with imm) or RDMA READ to avoid additional memory copy?
- Dominant language
- C++
- Stars
- 1.6k
- Forks
- 540
- PR merge metrics
- No merged PRs in 30d
Description
Hi developers of dmlc.
Recently I've been working with ps-lite. I find out that in the operation of `KVWorker::Pull`, there maybe some memory copy operations that may cause performance overhead:
```
template
void KVServer::Response(const KVMeta& req, const KVPairs& res) {
Message msg;
msg.meta.app_id = obj_->app_id();
msg.meta.customer_id = req.customer_id;
msg.meta.request = false;
msg.meta.push = req.push;
msg.meta.pull = req.pull;
msg.meta.head = req.cmd;
msg.meta.timestamp = req.timestamp;
msg.meta.recver = req.sender;
if (res.keys.size()) {
msg.AddData(res.keys);
msg.AddData(res.vals);
if (res.lens.size()) {
msg.AddData(res.lens);
}
}
Postoffice::Get()->van()->Send(msg);
}
```
In my opinion, code ` msg.AddData` holds back the communication between woker and server.
If using RDMA WRITE( with imm) or RDMA READ, we can exchange the address by out-of-band communication in the beginning, then do PULL and PUSH on these addresses.
This is just my thought. Please correct me if I'm wrong or I miss some details.
By the way, the function `Van::PackMetaPB` used by IBVERBS misses `pb->set_pull(meta.pull);` which is in the function of `Van::PackMeta` used by ZMQ. This will cause the error when running the test case `ps-lite/tests/test_kv_app.cc` using IBVERBS.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.