oxidecomputer / oxidecomputer/dropshot
Make page tokens shorter
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 1.2k
- Forks
- 104
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 22
Description
This is helpful but not essential for https://github.com/oxidecomputer/console/issues/1102, putting page tokens in console URLs. The real blocker is #436.
Page tokens are base64ed JSON and they come out pretty long. Here's an example:
> atob('eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJuYW1lX2FzY2VuZGluZyIsInByb2plY3QiOiJhbGFuIiwibGFzdF9zZWVuIjoienp6LWluc3QtMTE0In19')
'{"v":"v1","page_start":{"sort_by":"name_ascending","project":"alan","last_seen":"zzz-inst-114"}}'
oxiderack.com/projects/mock-project/instances?page=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJuYW1lX2FzY2VuZGluZyIsInByb2plY3QiOiJhbGFuIiwibGFzdF9zZWVuIjoienp6LWluc3QtMTE0In19 would work fine I guess, but to me it looks a bit silly. Here's where we do the serialization:
It should be pretty easy to encode the token in a more efficient format like MessagePack and maybe reduce the size of the data itself by, e.g., making some keys shorter.
| Method | Length of token |
|---|---|
| Current: base64 JSON string | 128 |
| MessagePack JSON as-is | 104 |
MessagePack JSON with page_start -> p |
92 |
| MessagePack struct directly with serde_rmp (code below) | 52 |
Rust program to compare base64 JSON and MessagePack
# Cargo.toml
[package]
name = "serialization-test"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
rmp-serde = "0.15.4"
base64 = "0.13.0"
// src/main.rs
extern crate base64;
extern crate rmp_serde as rmps;
extern crate serde;
extern crate serde_json;
use serde::{Deserialize, Serialize};
use std::error::Error;
#[derive(Debug, PartialEq, Deserialize, Serialize)]
struct MyStruct {
v: String,
page_start: PageStart,
}
#[derive(Debug, PartialEq, Deserialize, Serialize)]
struct PageStart {
sort_by: String,
project: String,
last_seen: String,
}
fn main() -> Result<(), Box<dyn Error>> {
let data = MyStruct {
v: "v1".to_owned(),
page_start: PageStart {
sort_by: "name_ascending".to_owned(),
project: "alan".to_owned(),
last_seen: "zzz-inst-114".to_owned(),
},
};
// Serialize with JSON
let json_data = serde_json::to_vec(&data)?;
let encoded_json = base64::encode(&json_data);
// Serialize with MessagePack
let msgpack_data = rmps::to_vec(&data)?;
let encoded_msgpack = base64::encode(&msgpack_data);
// Compare the lengths of the encoded strings
println!("Base64 JSON Length: {}", encoded_json.len());
println!("Base64 MessagePack Length: {}", encoded_msgpack.len());
Ok(())
}
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in dropshot/src/pagination.rs around lines 417-430, where page tokens are serialized, and review issue #436 because the description identifies it as the real blocker. Compare the current base64 JSON approach with the proposed MessagePack alternatives, then verify that pagination tokens remain usable while becoming shorter.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- api, backend-api-design
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100