redpanda-data / redpanda-data/connect

MongoDB regression: ISODate fields get persisted as strings

Open
#2,557 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug mongodb
Dominant language
Go
Stars
8.8k
Forks
969
Avg merge
1d 13h
Merged PRs (30d)
64

Description

This issue was introduced in v4.26.0 (commit 9f692589b). Steps to reproduce:

> docker run --rm -p27017:27017 --name mongodb -e MONGO_INITDB_ROOT_USERNAME=mongoadmin -e MONGO_INITDB_ROOT_PASSWORD=secret mongo
> docker exec -it mongodb bash
>> mongosh -u mongoadmin -p secret
>>> use food
>>> db.createCollection("fruits")
input:
  generate:
    mapping: |
      root.ts = "2024-04-29T12:34:05".ts_strptime("%Y-%m-%dT%T")
      root.name = "apple"
    count: 1

output:
  mongodb:
    url: mongodb://localhost:27017/?maxPoolSize=3
    username: mongoadmin
    password: secret
    database: food
    collection: fruits
    operation: insert-one
    write_concern:
      w: "majority"
      j: false
      w_timeout: 5s
    upsert: false
    document_map: |
      root = this

Run the above config with Benthos v4.25.0 and then with the current main branch (commit 51d971a) and then execute db.fruits.find() in mongosh. You'll notice that the ts field is different for the two entries.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the regression with the provided Docker, MongoDB, and Benthos configurations, comparing v4.25.0 with commit 51d971a. Start by tracing the MongoDB output path and document_map handling for the ISODate value introduced in v4.26.0. Done means the timestamp is persisted consistently as a MongoDB date rather than a string, with the regression covered by a test.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, mongodb
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.