mozilla-services / mozilla-services/syncstorage-rs

Batch commit updates always bump the modified time

Open
#2,009 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
1.8k
Forks
86
Avg merge
2d 17h
Merged PRs (30d)
21

Description

A long standing bug in all the Python and Rust implementations/backends: batch writes work slightly differently from regular writes in that they always update existing bso’s modified timestamp.

Per the e2e test’s test_bulk_update_of_ttls_without_sending_data, an existing bso’s modified timestamp should not be updated when solely updating its ttl -- “Just bumping the ttl should not bump timestamp”.

For this case Python syncstorage only updates modified if payload is updating. Rust syncstorage inherited go-syncstorage’s behavior of only updating it if payload or sortindex are updating (which I think was more correct).

However the Python/Rust implementations never included such logic during a batch commit, having always written a new modified timestamp to existing bsos.

At a quick glance: go-syncstorage had an inefficient implementation of batches that were committed via the post_bsos or put_bso code paths (reading them from the db to write them back out) so it wouldn’t have had this bug.

I would argue go-syncstorage is more correct here, treating all writes identically regardless of how they’re invoked and consider the batch write difference a bug which ideally should be fixed.

The potential downside to such a fix it is the extra logic could incur more overhead in Spanner’s batch commit, which is already pretty inefficient.

┆Issue is synchronized with this Jira Task

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the e2e test test_bulk_update_of_ttls_without_sending_data and trace the Python and Rust batch commit implementations. Compare batch writes with regular writes for existing bsos when only ttl changes, then verify that the modified timestamp remains unchanged while payload or sortindex updates still behave as expected.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.