MetPX / MetPX/sarrac

should there be something to suppress posts when files are created...

Open
#112 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement wishlist work-around available worries
Dominant language
C
Stars
4
Forks
1
Avg merge
5d 1h
Merged PRs (30d)
1

Description

on systems that are more heavily loaded, or running slower, there seems to be a tendency for INOTIFY
to produce CREAT events for the file being created, and then separate modify events...

possibilities:

* we are periodically processing and event queue. should we read the queue once, looking for duplicates, and remove the first op?
* there is already a flag for create events, perhaps just set that off in flow tests, and perhaps elsewhere. problem is... what if someone actually is creating an empty file? then it just will not be propagated.

It would be an enhancement if we can reduce the number of posts needed to duplicate the same tree by omitting CREAT events that are overridden by modifies...

We could go further in the batching... say for empty files, hold onto them for a few seconds in a queue, and only create them if no other event comes later to squash them...

not sure. currently this is taken care of by putting plugins in shovels/winnows, so no effect on HPC mirror client, but it complicates deployment a bit.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the INOTIFY event queue and the handling of CREAT and MODIFY events. Clarify whether duplicate events should be removed during batching and how empty files must be preserved; done should include a defined behavior and validation that reduces redundant posts without losing legitimate creations.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.