simonsobs / simonsobs/ocs

Require the same tags are passed each time when publishing a block

Open
#477 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
19
Forks
7
PR merge metrics
No merged PRs in 30d

Description

Shouldn't it be required that the latest tags match the previous tags?

I think, as written, someone could do this:

  • publish some data with tags={...} fully populated -- these get cached.
  • publish another point, with tags=None -- this gets cached.
  • If the buffer is now flushed, the result will have tags=None, because that was the most recent thing.

That would pass all checks, but the stored tags (when the block gets flushed) would be None. It's probably not what the user wanted. I think either:

  • Require user to pass the same tags every time they push data to this block OR
  • If tags=None, don't change the stored tags in the block.

At some level, data producers are doomed to always provide tags, so the the first thing makes more sense to me.

Granted, this would require modification to the aggregator agent (so far unchanged), because you'd need to initialize each new Block (in Provider.save_to_block) with influxdb_tags set from the first transmission. That is messier in the present instance but seems logically safer on the whole. I think the best way would be to have a constructor for Block that parses the block data dict (just as Block.append and Block.extend do), and sets all the things from that (including influxdb_tags). This would make Aggregator, in the long run, less fragile vis a vis stuff in the feed that it doesn't care about.

Originally posted by @mhasself in https://github.com/simonsobs/ocs/pull/467#discussion_r3191201673

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at Provider.save_to_block and the Block.append/extend paths, then trace how the Aggregator handles cached data when a block is flushed. The issue suggests initializing Block from the first transmission and enforcing a consistent influxdb_tags value; done means later writes cannot silently replace populated tags with None.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.