weaviate / weaviate/java-client

v6: OffsetDateTime is serialised with toString(), which drops the seconds and is not RFC 3339

Open
#605 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
34
Forks
30
PR merge metrics
No merged PRs in 30d

Description

Summary

The client serialises OffsetDateTime with toString(), which omits the seconds when they are zero. 2024-03-01T00:00:00Z therefore goes on the wire as 2024-03-01T00:00Z, which is not RFC 3339 — partial-time requires hour ":" minute ":" second — and Weaviate rejects it.

Every timestamp on an exact minute boundary is affected, which is most timestamps written by hand. It breaks writes and filters alike.

Reproduction

OffsetDateTime round       = OffsetDateTime.parse("2024-03-01T00:00:00Z");
OffsetDateTime withSeconds = OffsetDateTime.parse("2024-03-01T00:00:01Z");

System.out.println(round);        // 2024-03-01T00:00Z    <- the seconds are gone
System.out.println(withSeconds);  // 2024-03-01T00:00:01Z

Against a collection with a single date property when:

FAIL  REST insert  round        -> invalid date property 'when' on class 'DBeaverDateProbe':
                                   requires a string with a RFC3339 formatted date,
                                   but the given value is '2024-03-01T00:00Z'
OK    REST insert  withSeconds
FAIL  gRPC batch   round        -> invalid date property 'when' on class 'DBeaverDateProbe':
                                   requires a string with a RFC3339 formatted date,
                                   but the given value is '2024-03-01T00:00Z'
OK    gRPC batch   withSeconds

and on the query side:

FAIL  Filter.property("when").gt(round)        -> trying parse time as RFC3339 string:
                                                  parsing time "2024-03-01T00:00Z" as
                                                  "2006-01-02T15:04:05Z07:00": cannot parse "Z" as ":"
OK    Filter.property("when").gt(withSeconds)  -> 2 rows
OK    Filter.property("when").gt("2024-03-01T00:00:00Z")   // String overload -> 2 rows
FAIL  Filter.createdAt().gt(round)             -> same parse error
OK    Filter.createdAt().gt(withSeconds)

The only difference between each passing and failing pair is whether the second happens to be zero, which isolates the cause.

Where it comes from

Disassembling every non-protocol class in client6-6.3.1-all.jar and grepping for java/time/OffsetDateTime.toString gives three call sites, one per path:

class path reached by
Filter$DateOperand gRPC search every date filter, plus Filter.createdAt() / Filter.lastUpdatedAt()
InsertManyRequest.marshalValue gRPC batch data.insertMany(...)Value.newBuilder().setStringValue(dt.toString())
DateUtil$CustomTypeAdapterFactory$1 REST / Gson data.insert(...) and object reads — JsonWriter.value(dt.toString())

OffsetDateTime.toString() delegates down to LocalTime.toString(), which emits HH:mm when both second and nano are zero. It is a display format, not a wire format.

Suggested fix

Format explicitly at all three sites instead of calling toString() — for example DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ssXXX"), or ISO_OFFSET_DATE_TIME applied to a value normalised to at least second precision, so the seconds are always present.

Workaround, and where there isn't one

For filters on a property, pass the RFC3339 String rather than an OffsetDateTime: Filter.property(p).gt("2024-03-01T00:00:00Z") takes the String overload and works (shown above).

There is no workaround on the metadata path — Filter.createdAt() and Filter.lastUpdatedAt() return DateProperty, whose comparison methods accept OffsetDateTime only. Same for writes, where the property map value has to be an OffsetDateTime to be recognised as a date at all.

Version

  • java-client 6.3.1
  • Weaviate 1.39.0

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Locate Filter$DateOperand, InsertManyRequest.marshalValue, and DateUtil$CustomTypeAdapterFactory$1, which are the three serialization entry points described in the issue. Reproduce the round-minute cases for REST inserts, gRPC batches, and date filters, then verify that all three paths emit RFC 3339 timestamps with seconds and that writes and filters accept them.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.