dfinity / dfinity/candid

Rust library: Optimize `Vec` of certain types

Open
#224 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement Rust
Dominant language
Rust
Stars
301
Forks
85
Avg merge
1h 4m
Merged PRs (30d)
4

Description

It seems that Candid serialization/deserialization in Rust can get quite expensive, especially if Vec<u8> is involved. Users can avoid that by using serde::ByteBuf instead, which allows Candid to copy the whole blob in and out, but it’s inconvenient if Vec<u8> is idiomatic.

The obvious solution, i.e. impl Deserialize for Vec<u8> in addition to impl Deserialize for Vec<T>, doesn't work (unless we get https://github.com/rust-lang/rust/issues/31844).

But we could do what Haskell does to handle a very similar problem with the same constraints (there the pretty-printing of [Char]).

This would require extending CandidType to have, in addition to the existing method

    fn idl_serialize<S>(&self, serializer: S) -> Result<(), S::Error>

a new method (rought syntax)

    fn idl_serialize_vec<S>(xs : &[Self], serializer: S) -> Result<(), S::Error>
  • Vec[t]’s implementation of idl_serialize calls idl_serialize_vec on the conatined type
  • Most impls define idl_serialize_vec by calling the usual, polymorphic, encoding function for Vec (i.e. what idl_serialize does for Vec[t] now)
  • But some, like impl CandidType for u8, can now do optimized stuff, e.g. use serialize_blob.

For encoding we can do that, since we have a custom trait. For decoding this might require introducing a CandidDeserialize instead of using Serde’s, not sure.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the CandidType trait, idl_serialize, Vec handling, and the serialize_blob path mentioned in the issue. Review how serde::ByteBuf currently avoids the Vec cost, then determine whether encoding and decoding require the proposed CandidDeserialize design; done means a settled approach for optimizing Vec without the conflicting Deserialize implementations.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.