CulverLab / CulverLab/sparcd-exploration
CI: end-to-end run against a real S3-compatible store with size-tiered fixtures
- Dominant language
- TypeScript
- Stars
- 6
- Forks
- 3
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 106
Description
Nothing in CI proves the three apps work together against a real object store. The BDD suites mock every S3 call, so a change to signing, CORS or metadata handling passes green and breaks in the field. The bench job does boot a real store, but it only drives the uploader and only cares about timing.
The store in CI is a fixture, MinIO today, and nothing in the apps or the workflow should depend on which provider it is. Production is moving to Ceph RGW on Jetstream and other teams will bring their own, so the provider dimension stays in #197 and this workflow should be able to swap the container without changing the test.
Where:
- `.github/workflows/bdd-tests.yml:55-58` says S3 traffic is mocked, via `apps/sparcd-uploader/features/steps/s3mock.ts` and `apps/sparcd-tagger/features/steps/support/s3mock.ts`
- `.github/workflows/uploader-bench.yml:61` runs `apps/sparcd-uploader/bench/run.mjs`, which starts the ephemeral store at `bench/run.mjs:64-73`
- `apps/sparcd-uploader/bench/gen-fixtures.mjs:6-9` makes one size only: 360 JPEGs at 256KB plus 2 MP4s at 12MB
- `.github/workflows/pages.yml:84-86` ships all three apps, so all three belong in the check
What to do:
1. Take the image tiers from the generator work in #192: small, medium, large and extra large plus the existing MP4, keeping the current corpus as one named tier so bench timings stay comparable. The generator itself belongs to #192; this issue consumes it.
2. Add a workflow that reuses the store block from `run.mjs`, uploads a tier through the uploader, opens the resulting collection in the tagger and tags one image, then loads the same collection in the explorer.
3. Run it on pull requests touching apps or packages, on a smaller tier than the bench so it stays quick.
Done when:
- [ ] One workflow covers upload, tag and read against a real store
- [ ] Every image tier and the MP4 upload and verify
- [ ] A tag written by the tagger is readable back from the bucket
- [ ] The explorer opens the collection the uploader created
- [ ] The store container is a single swappable setting in the workflow
- [ ] Failures name the app and the step
Related: #192 fixtures, #153 vitest never runs in CI, #197 provider matrix, #198 meeting notes.
Serves user story: SAM1, and verifies F1, AL1, AL2, H2
Contributor guide
Assessment
This issue has not been assessed yet.