OpenFn / OpenFn/lightning

Chat messages written in the same second can render in either order

Open
#5,081 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Elixir
Stars
296
Forks
86
Avg merge
1d 13h
Merged PRs (30d)
50

Description

Messages in a chat session come back ordered by inserted_at and nothing else, and our timestamps are stored to the second. So two messages written inside the same second tie, and the panel can render them in either order. A question can appear below the answer to it.

In practice a reply takes seconds to arrive, so this is rare against a real model. It shows up reliably in tests, where both writes land in the same millisecond, and that is how we noticed. The tests have been fixed to pick messages by role rather than by position, so this is only about what a person sees.

It is not as small as it looks, which is why it is here rather than fixed.

Every schema inherits @timestamps_opts [type: :utc_datetime] from lib/lightning/schema.ex, so second precision is a repo-wide convention rather than something specific to chat messages. There are around 34 places that order by inserted_at, so the same tie is latent in a lot more than this one panel.

Two ways out, neither free:

Adding the id as a secondary sort is one line, and lib/lightning/workflow_versions.ex already does that, so there is precedent. But our ids are random UUIDs, so it only makes the order stable, not right. A tied pair would consistently render the wrong way round instead of sometimes.

Moving the column to microsecond precision fixes it properly, but it means overriding the convention for one table or changing it everywhere, and altering a timestamp's precision rewrites the table under an exclusive lock. Rows already written stay tied either way, since their sub-second part is already gone.

So this wants a decision from someone who knows how big the table is in production, rather than a quick patch. Noting it so it is not rediscovered from scratch.

Came up while working on the AI streaming stack (#5079 and the PRs above it).

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with lib/lightning/schema.ex and the secondary ordering precedent in lib/lightning/workflow_versions.ex. Review the roughly 34 orderings by inserted_at and determine the affected production table size and locking implications. Done requires choosing and implementing a consistent ordering or timestamp-precision strategy with the associated migration and verification.

Written by the indexing model from the issue text.

Assessment

Tech stack
elixir
Domain
databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.