scylladb / scylladb/python-rs-driver

Add custom serialization

Open
#162 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
6
Forks
7
Avg merge
5d 13h
Merged PRs (30d)
9

Description

Background: several old-driver tests deliberately send malformed/invalid wire data to verify the
server rejects it correctly — something Python's own type system won't let you construct normally (a
str can't hold invalid UTF-8, an int has no "empty" representation). The old driver gets around this
because its serializers are plain Python classes/functions that tests can monkeypatch:

Proposal: expose a way to plug in a custom Python serializer, invoked instead of the built-in one for
a given value/type.

Needs evaluation — implementation approach, not just API shape:

  1. How serialization is actually exposed to Python — the naive approach is a serialize() method on a
    user's class that returns raw bytes, letting the value "know how to serialize itself." But prior
    benchmarking here showed that exposing a Rust-backed writer structure (e.g. something like a
    CellWriter object the user writes into, rather than building a bytes object in pure Python and
    handing it back) is roughly 2x faster. We need to evaluate whether to:
    • keep it simple (serialize() -> bytes, slower, easiest to implement/use), or
    • build a proper writer-object API (faster, more implementation work, better long-term default) — and
      whether that's worth it.
  2. GIL-crossing cost: any custom serializer means calling back into Python from inside the Rust
    serialization path — this should probably be an explicit opt-in per value/statement, not something
    that touches the default hot path.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the existing Rust serialization path and the old-driver serializer classes described in the issue. Compare the proposed Python bytes callback with a Rust-backed writer such as CellWriter, including GIL-crossing costs and opt-in scope. Done means selecting and documenting an implementation approach rather than only defining an API shape.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
database
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.