questdb / questdb/py-questdb-client

row() cannot write UUID, IPv4, GEOHASH, LONG256, CHAR, DATE or BINARY columns

Open
#138 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
71
Forks
14
Avg merge
1h 7m
Merged PRs (30d)
1

Description

Summary

row() cannot write UUID, IPv4, GEOHASH, LONG256, CHAR, DATE or BINARY columns. A row-at-a-time producer has no way to populate them, even though the QWP wire protocol carries all seven and the Rust buffer underneath already implements every setter.

The columns union accepted by row() (src/questdb/_client.pyi:974) is:

Union[None, bool, int, float, str,
      TimestampMicros, TimestampNanos, datetime,
      np.ndarray, Decimal]

None of the seven types has a representation in it.

Current status: documented as a limitation

The docs state this plainly rather than treating it as a bug — documentation/connect/clients/python.md:291:

UUID, IPv4, GEOHASH, LONG256, CHAR, DATE, and BINARY columns have no row() value type. Route them through dataframe(), whose schema_overrides covers symbol, ipv4, char, and geohash, or through a SQL INSERT via query().

Both suggested routes are real, but both force the user off the streaming API: dataframe() requires batching rows into a pandas/Arrow frame, and query() means hand-writing SQL INSERT and giving up QWP's batching, auto-flush and acknowledgement machinery. Neither is a substitute for a per-row setter in a streaming producer.

Cross-client parity

Python is the only QWP client with this gap. Row-oriented ingestion API, verified in source:

Type Rust Buffer C/C++ line_sender_buffer .NET Java Sender Go QwpSender Python row()
UUID
GEOHASH
LONG256
CHAR
IPv4
BINARY
DATE

Nothing is missing underneath

The plumbing already exists on both sides of the Cython layer:

  • questdb-rs Buffer exposes all seven, with _opt null variants — column_uuid, column_long256, column_ipv4, column_date, column_char, column_binary, column_geohash (questdb-rs/src/ingress/buffer.rs:1252-1552).
  • _client.pyx already drives col_target_column_uuid and col_target_column_long256 on the dataframe() path (_client.pyx:2927-2928, :4080), and already resolves ipv4, char and geohash through schema_overrides (:4329-4345).

So this is a matter of surfacing existing capability through row(), not of adding wire or buffer support.

Suggested mapping

Three types have an unambiguous Python native that row() does not currently accept:

Python value QuestDB type
uuid.UUID UUID
ipaddress.IPv4Address IPv4
bytes / bytearray / memoryview BINARY

The remaining four need an explicit wrapper, because the natural Python type is already bound to something else or carries insufficient information:

  • GEOHASH — needs precision bits alongside the value; there is no native to infer them from. A Geohash(value, precision_bits) wrapper, or a str base32 wrapper where precision is len(value) * 5.
  • LONG256 — a Python int already maps to LONG.
  • CHAR — a 1-character str already maps to VARCHAR.
  • DATEdatetime already maps to TIMESTAMP / TIMESTAMP_NS.

Wrapper naming and whether GEOHASH precision should be locked per column on first use (as the Java client does) are worth settling before implementation.

Notes

  • dataframe()'s schema_overrides covers only symbol, ipv4, char and geohash — so even the documented workaround does not reach UUID, LONG256, DATE or BINARY by override; those depend on the frame's own dtypes.
  • Related, closed long ago against the ILP-era client: #19 (Columns of type long256 are not supported).

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the row() column union in src/questdb/_client.pyi:974, then trace its handling in _client.pyx and the existing setters in questdb-rs/src/ingress/buffer.rs:1252-1552. Review the dataframe() paths at _client.pyx:2927-2928, 4080, and 4329-4345. Done means row() can represent all seven types with settled wrapper and precision behavior while retaining the existing null handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
api
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.