rubyforgood / rubyforgood/awbw
Orphaned Active Storage blobs: nothing reclaims unattached direct uploads
Nobody has claimed this yet.
- Dominant language
- Ruby
- Stars
- 15
- Forks
- 26
- Avg merge
- 12h 42m
- Merged PRs (30d)
- 242
Description
🤖 From Claude: Written up while reviewing #2217 (follow-ups to the file-upload answer type, #2136). Deliberately kept out of that PR — it's destructive and wants its own rollout.
The problem
Direct upload writes the blob to storage before the form is submitted. Anything that happens after that leaves a blob nothing references:
- the registrant abandons the form
- the content type is rejected
- the file is over the 25 MB cap added in #2217
- the registration fails for an unrelated reason (a too-long answer, a duplicate, a Stripe error)
- a validation error elsewhere on the form and the registrant re-picks a different file
Nothing in the app reclaims these. There is no unattached, no purge_later, and no sweeper anywhere in app/, lib/, or config/.
StoryImportsController is a second, deliberate source: it stages a CSV as an unattached blob between preview and confirm, purging it explicitly on confirm or on a parse error. A preview that's never confirmed leaks one.
Why now
The exposure predates the file-upload feature — app/views/shared/_form_image_field.html.erb and app/views/assets/_primary_image_picker.html.erb both use direct_upload: true. But those are admin-only pages with a handful of uses a week. The public event registration form changes the volume by orders of magnitude, and production storage is DigitalOcean Spaces, where it's a line item.
Proposed fix
Solid Queue and config/recurring.yml are already in place with two daily jobs, so this is one job class and one YAML entry.
# app/jobs/purge_unattached_blobs_job.rb
class PurgeUnattachedBlobsJob < ApplicationJob
queue_as :default
# Direct upload stores a blob before its form is submitted, so a blob is
# legitimately unattached while someone is still filling in the form (and
# while a story CSV import sits between preview and confirm). Only sweep
# ones old enough that no in-flight submission could still claim them.
UNCLAIMED_AFTER = 2.days
def perform
ActiveStorage::Blob.unattached
.where(active_storage_blobs: { created_at: ..UNCLAIMED_AFTER.ago })
.find_each(&:purge_later)
end
end
# config/recurring.yml, under production:
purge_unattached_blobs:
class: PurgeUnattachedBlobsJob
schedule: every day at 4am
4am because 3:00 and 3:30 are already taken by reconcile_checkout_payments and issue_membership_invoices.
Details that matter
- The age window is load-bearing, not a nicety.
ActiveStorage::Blob.unattachediswhere.missing(:attachments), and a blob is supposed to be unattached during the gap between direct upload and submit. Sweeping without an age filter would delete files out from under registrants mid-form. Two days is Rails' own documented figure and comfortably exceeds any workflow here, including the story-import preview. - Qualify
created_atwith the table name.unattachedbuilds aLEFT JOINagainstactive_storage_attachments, which also has acreated_at— unqualified is ambiguous. purge_later, notpurge. Each deletion becomes its own retryable job, so one flaky Spaces call doesn't abort the sweep.- This is global and destructive. It reclaims any unattached blob, not just registration uploads. Re-check for deferred-attachment paths before it ships; the only one today is the story import described above, where sweeping is a fix rather than a conflict.
Suggested rollout
Ship it in report mode first — log the count and total bytes it would purge, run for a week, then look at the numbers before it deletes anything. "Delete N files" is not a change to land on an unexamined production dataset, and the first run is the one most likely to be surprising.
What this doesn't solve
The client-side size guard in #2217 reduces orphans (an oversized file never uploads), and accept helps for content type. Neither is enforceable — a crafted request can POST straight to /rails/active_storage/direct_uploads, which is unauthenticated by Rails default. The sweeper is the backstop. If abuse ever becomes real, the next lever is rate-limiting or authenticating that endpoint, which is a bigger change than it looks because the admin pages depend on it too.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing ActiveStorage::Blob.unattached, StoryImportsController, and the existing scheduled entries in config/recurring.yml. Add app/jobs/purge_unattached_blobs_job.rb and its production schedule, beginning in report mode as requested. Done means the age-qualified global sweep is observable, uses retryable purges, and has been checked against deferred-attachment paths before deletion is enabled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rails, ruby, yaml
- Domain
- backend, devops
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100