paritytech / paritytech/parity-scale-codec

Suggestion for new decode API forcing developers to explicitly write down all decode options/limits

Open
#785 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
287
Forks
103
Avg merge
6d 13h
Merged PRs (30d)
1

Description

As of now we have quite a number of decoding APIs such as the following:

  • decode
  • decode_all
  • decode_all_with_depth_limit
  • decode_with_mem_limit

However some options/combinations are missing, for example there is no decode_all_with_mem_limit and it is not possible to combine a a depth limit with a mem limit (at least not without manually chaining MemTrackingInput and DepthTrackingInput). Additionally to that, the current API does not force developers to explicitly decide (and write down) whether the _all variant is needed and whether a depth limit and/or a mem limit is required. This can easily lead to security vulnerabilities such as using decode instead of decode_all (something which has happened in the past).

In order to improve the situation I'd suggest a new API to force developers to explicitly write down the choices (decode vs decode_all, with or without depth limit, with or without mem limit). The following code outlines how the API could look like. Regarding the choice to use separate types for all combinations: This is required to implement some options only for types implementing DecodeWithMemTracking - and it also allows the compiler to only include the implementation variants actually used.

use parity_scale_codec::{Decode, DecodeWithMemTracking, Encode, Error, Input};

#[derive(Encode, Decode)]
struct Foo{
    a: u32,
    b: u32
}

fn main() {
    let foo = Foo{a: 1, b: 2};
    let encoded = foo.encode();
    let _decoded: Foo = DecodeMode::ForceDecodeAll.no_depth_limit().no_mem_limit().decode(&mut &encoded[..]).unwrap();
    // This will fail at compile time with the message "the trait bound `Foo: DecodeWithMemTracking` is not satisfied"
    // Correct since Foo does not implement/derive DecodeWithMemTracking
    // let _decoded: Foo = DecodeMode::ForceDecodeAll.no_depth_limit().with_mem_limit(1_000_000).decode(&mut &encoded[..]).unwrap();
}

pub enum DecodeMode{
  ForceDecodeAll,
  AllowPartialDecoding
}

impl DecodeMode{
    fn with_depth_limit(self, depth_limit: u32) -> DecodeModeWithDepthLimit{
        DecodeModeWithDepthLimit{
            decode_mode: self,
            depth_limit,
        }
    }
    fn no_depth_limit(self) -> DecodeModeNoDepthLimit{
        DecodeModeNoDepthLimit{
            decode_mode: self,
        }
    }
}

pub struct DecodeModeWithDepthLimit{
    decode_mode: DecodeMode,
    depth_limit: u32
}

impl DecodeModeWithDepthLimit{
    pub fn with_mem_limit(self, mem_limit: usize) -> DecodeModeDepthAndMemLimit{
        DecodeModeDepthAndMemLimit{
            decode_mode: self.decode_mode,
            depth_limit: self.depth_limit,
            mem_limit
        }
    }
    pub fn no_mem_limit(self) -> DecodeModeDepthLimitOnly{
        DecodeModeDepthLimitOnly{
            decode_mode: self.decode_mode,
            depth_limit: self.depth_limit
        }
    }
}

pub struct DecodeModeNoDepthLimit{
    decode_mode: DecodeMode
}

impl DecodeModeNoDepthLimit{
    pub fn with_mem_limit(self, mem_limit: usize) -> DecodeModeMemLimitOnly{
        DecodeModeMemLimitOnly{
            decode_mode: self.decode_mode,
            mem_limit
        }
    }
    pub fn no_mem_limit(self) -> DecodeModeNoLimits{
        DecodeModeNoLimits{
            decode_mode: self.decode_mode,
        }
    }
}


pub struct DecodeModeNoLimits{
    decode_mode: DecodeMode
}
impl DecodeModeNoLimits{
    fn decode<I,T>(&self, input: &mut I) -> Result<T, Error>
    where I: Input, T: Decode
    {
        let t = T::decode(input)?;
        // TODO: Check if input is empty based on self.decode_mode
        Ok(t)
    }
}
pub struct DecodeModeDepthLimitOnly{
    decode_mode: DecodeMode,
    depth_limit: u32
}

impl DecodeModeDepthLimitOnly{
    fn decode<I,T>(&self, input: &mut I) -> Result<T, Error>
    where I: Input, T: Decode
    {
        todo!()
    }
}

pub struct DecodeModeMemLimitOnly{
    decode_mode: DecodeMode,
    mem_limit: usize
}

impl DecodeModeMemLimitOnly{
    fn decode<I,T>(&self, input: &mut I) -> Result<T, Error>
    where I: Input, T: DecodeWithMemTracking
    {
        todo!()
    }
}
pub struct DecodeModeDepthAndMemLimit{
    decode_mode: DecodeMode,
    depth_limit: u32,
    mem_limit: usize
}

impl DecodeModeDepthAndMemLimit{
    fn decode<I,T>(&self, input: &mut I) -> Result<T, Error>
    where I: Input, T: DecodeWithMemTracking
    {
        todo!()
    }
}

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the existing decode, decode_all, decode_all_with_depth_limit, and decode_with_mem_limit entry points, along with MemTrackingInput, DepthTrackingInput, Decode, and DecodeWithMemTracking. Compare them with the proposed DecodeMode combinations and determine the intended behavior for partial decoding and each limit combination. Done means a coherent API that makes these choices explicit and supports the requested combinations without requiring manual input chaining.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.