sillsdev / sillsdev/languageforge-lexbox

Orphaned media resources (and reserved Files rows) are never garbage-collected

Open
#2,607 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C#
Stars
9
Forks
8
Avg merge
2d 13h
Merged PRs (30d)
49

Description

Summary

Removing the last entry reference to a media id does not garbage-collect the underlying media — the Harmony resource, the physical file, and the lexbox Files row all persist forever, referenced by nothing. There is no reference-counting or GC for media anywhere today.

Details

  • Media resources are reference-independent: RemoteResource.GetReferences() is empty and RemoveReference is a no-op, so clearing the entry field that held a sil-media://.../{fileId} URI never cascades to the resource. The resource (and, for an uploaded file, its bytes in hg and its Files row) outlives the reference.
  • No code path GCs media on reference removal. The reconcile only deletes a Remote resource that has no Files row (a different case); the hg-reconcile only removes a Files row whose physical file vanished. No scheduled job, admin action, or client action removes media that is simply no longer referenced by any entry.
  • Applies to both cases:
    • an uploaded file whose reference is removed → leaks resource + hg file + Files row;
    • a never-uploaded/pending reference whose reference is removed → leaks the pending resource (and, with the media file-not-found fix (#2605), its reserved Revision == 0 Files row, which is bounded 1:1 to the resource).
  • Net effect: create/attach-then-abandon of a media reference leaks per occurrence, unbounded over a project's lifetime.

Proposed fix (later)

Garbage-collect media once no entry references its id: reference-count sil-media://.../{fileId} usage across entry fields and, when a resource becomes unreferenced, delete the resource, its Files row, and (if uploaded) its file. The reference check against entry usage does not exist anywhere yet and is the bulk of the work; it also has to reckon with media shared across multiple entries/fields, re-add, and undo.

Out of scope / already handled

The media file-not-found fix reserves a pending Files row (bounded 1:1 to a pending resource), adds a guard so a reconcile can't hard-fail on an orphaned pending row, and ties that reserved row's deletion to its resource. It does not add the reference-counting GC above — that is this issue.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing RemoteResource.GetReferences and RemoveReference, then inspect the existing Remote-resource and hg reconciliation paths described in the issue. Map how sil-media://.../{fileId} references are used across entry fields and how uploaded and pending resources relate to Files rows. Done means unreferenced resources, rows, and uploaded files are safely removed without affecting shared, re-added, or undo-related media.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
backend, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.