Design: invalidate fixture cache when source definition changes
Nobody has claimed this yet.
- Dominant language
- Ruby
- Stars
- 177
- Forks
- 4
- Avg merge
- 4d 8h
- Merged PRs (30d)
- 1
Description
Context
PR #61 fixed a NoMethodError when child fixtures were saved against a parent whose cache existed on disk but was not mounted in the current process. During review, a reviewer surfaced a deeper, pre-existing concern:
The lazy-load path can silently use stale parent cache data when the parent cache file exists but does not match the current parent fixture definition.
Fixture#generateskips regeneration when@cache.exists? && !force, andCache#exists?treats a disk file as sufficient. The new child-save path reads parent data directly from that file, andActiveRecordCoder#generatemergesparent_data.keysinto the captured model set. No definition digest, mtime check, schema/version key, or parent-definition validation was found inCache,FileCache,Fixture, orRunner.
This was deliberately scoped out of #61 because it is not new behavior — but it is a real correctness gap and warrants a separate design discussion.
Today's behavior
A fixture cache on disk is treated as valid forever, regardless of:
- Changes to the fixture's
Definitionblock in source code. - Changes to the underlying schema (new columns, dropped tables, type changes).
- Changes to
Codersubclasses (new coders registered, customencode/decodeupdated). - Changes to the Rails / Ruby version that wrote the cache.
The only safety nets today are:
- FK verification at mount time (
ActiveRecord.verify_foreign_keys_for_fixtures), which only catches a subset of schema drift and only at load. - Manual
FIXTURE_KIT_PRESERVE_CACHEopt-out anddb:test:prepare.
What needs deciding
1. What identifies a "valid" cache?
Candidates:
- Source digest of the
Definitionblock (and any procs it closes over — likely needsRubyVM::InstructionSequenceor source-location-based hashing). - mtime of the fixture file that declared the fixture. Cheaper, more brittle (touching the file invalidates).
- Schema fingerprint (digest of
db/schema.rb, orActiveRecord::Migration.current_version). - Coder fingerprint (registered coder classes + their source digest).
- FixtureKit version string (cheap, catches gem upgrades).
A useful cache fingerprint is probably a tuple of several of these, not one. The cheapest worthwhile combo is (fixture_kit_version, schema_version).
2. Where does the metadata live?
- New top-level key in the cache JSON (
"meta": { "version": ..., "schema": ..., "definition_digest": ... }). Simple, requires migrating existing caches. - Sidecar
.meta.jsonfile. Avoids touching the existing format. More disk I/O. - Filename suffix (
fixture.json.v2.abc123.json). Visible but ugly.
3. What happens on mismatch?
- Raise with a clear message ("cache version mismatch — regenerate"). Conservative.
- Silently regenerate. Convenient but hides bugs in the fingerprint logic.
- Warn + regenerate. Probably the right middle ground; logging is cheap.
4. Backward compatibility
Existing users have a populated cache directory. Options:
- Treat a cache without a
metablock as version-0 and force regeneration. One-time cost per fixture, no manual migration needed. - Add a CLI / rake task to migrate caches in place.
- Bump major version of the gem and document the cache reset as a breaking change.
Out of scope
- Auto-regeneration logic itself.
Fixture#generate(force:)already exists; this issue is about when to setforce: trueautomatically. - Cross-process locking. Worth tracking but separate.
Why this is filed as a discussion, not a PR
Each of the four questions above has reasonable answers in multiple directions, and the choices interact (a heavier fingerprint is more correct but more expensive; mismatch behavior is shaped by how cheap regeneration is in practice). Picking a strategy unilaterally and shipping a PR risks landing the wrong tradeoff. Looking for maintainer steer on at least Q1 and Q3 before coding.
Related
- #61 — fix for the lazy-load case that surfaced this concern
- #62 — follow-up: wrap corrupt cache errors (also from #61 review)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading Cache, FileCache, Fixture, and Runner, then trace how Fixture#generate and Cache#exists? decide whether to reuse disk data. Review #61 for the lazy-load path that exposed the issue. Done means reaching maintainer agreement on the cache fingerprint, metadata location, mismatch behavior, and backward-compatibility strategy before implementation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rails, ruby
- Domain
- backend, databases, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100