nspcc-dev / nspcc-dev/neofs-node

Child is still alive after the big object was deleted after expiration

Open
#3,621 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug I3 neofs-storage S2 U3
Dominant language
Go
Stars
38
Forks
51
Avg merge
1d 20h
Merged PRs (30d)
38

Description

Expected Behavior

When a big object is deleted after expiration, all of its parts are also deleted.

Current Behavior

After we delete expired objects on the garbage collector tick in #3582, a situation may arise where a big object has been deleted after expiration in one node, but not yet in another, and the policy is triggered. The parent is not allowed by the expiration attribute, but the child is copied, so it becomes available.

2025-10-06T16:06:49.564+0300	error	replicator/process.go:87	could not replicate object	{"component": "Object Replicator", "node": "03e340a3b89e1e6398c7c956eb0497d2a5faf5e61d5b911845a807e5e742fb7050", "object": "F6hNZM2hkgdaPDQ8nP3TATMxGMiX8yqmYYKRi4sH97RF/8RTPQ1sb8HbMP9vvdoMPRHviFTHnrwsprK4Fhp9cpzS7", "error": "copy object using NeoFS API client of the remote node: /dns4/localhost/tcp/38359: status: code = 1024 message = failed to verify and store object locally: validate object format: object did not pass expiration check: object has expired: attribute: 4, current: 5"}
2025-10-06T16:06:49.564+0300	debug	replicator/process.go:36	finish work	{"component": "Object Replicator", "amount of unfinished replicas": 1}
2025-10-06T16:06:49.582+0300	debug	replicator/process.go:91	object successfully replicated	{"component": "Object Replicator", "node": "03e340a3b89e1e6398c7c956eb0497d2a5faf5e61d5b911845a807e5e742fb7050", "object": "F6hNZM2hkgdaPDQ8nP3TATMxGMiX8yqmYYKRi4sH97RF/EBpfw1PwNhazenkBYz1a4XhHCLX9aKvGvEKP781r4VrE"}

Possible Solution

Maybe, in the policy, check for more accurate expiration attributes.

Steps to Reproduce (for bugs)

Put a big object with multiple parts and an expiration attribute in a container with replicas on multiple nodes. Wait for the expiration to occur, then head the child objects. Sometimes these parts may still be available.

Context

Flacky test case https://rest.fs.neo.org/HXSaMJXk2g8C14ht8HSi7BBaiYZ1HeWh2xnWPGQCg4H6/4145-1759747062/index.html#suites/87d8da5f92824d091055170c29079bc0/2215af6c5636b4/.

Regression

Yes, store dead objects that will no longer be removed.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the replication flow in replicator/process.go and the expiration cleanup introduced around issue #3582. Reproduce the flaky multi-node case using the linked test report, then inspect how the policy evaluates parent and child expiration attributes. Done means expired child parts are not copied and are removed consistently across nodes.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.