Publish an optional per-file upstream URL for large files

❌ This issue is not open for contribution. Visit Contributing guidelines to learn about the contributing process and how to find suitable issues.

Overview

Studio records nothing about where a content file can be fetched from — Kolibri derives the URL from its own configuration, so every file is assumed to be served by Studio. Content files over 512 MiB are now mirrored to Cloudflare R2 and served from a different host. Store an upstream_url on File, set it for files above that threshold, and carry it into published channels and the public import-metadata API.

Complexity: Medium
Target branch: hotfixes

Context

• kolibri_content/base_models.py is a declared verbatim copy of Kolibri's base_models.py, and kolibri_public.models.LocalFile is a bare subclass of it. One field definition reaches both the published channel database and the public import-metadata API.
• publish.py:714 is where a Studio File becomes a kolibri_content LocalFile, and mapper.py:225 is where those rows reach kolibri_public.
• import_metadata_view.py:160 validates the requested schema_version against the minimum and current versions, then selects fields from the current base model. A new field therefore appears in responses at every accepted version, not only VERSION_7.
• Studio's vendored CONTENT_SCHEMA_VERSION is VERSION_6 and tracks Kolibri's. The field arrives with VERSION_7.
• Objects are copied into R2 by a GCS finalize trigger that filters at 512 MiB, so a File given an upstream_url below that threshold would point at an object that was never copied.
• Existing objects over the threshold are copied into R2 by a one-off infrastructure backfill, which has to have run before any existing File row is given a URL.

The Change

• File should carry an upstream_url recording where the file can be fetched from.
• upstream_url should be a plain URL with no R2-specific meaning, so it can later hold a link to a host Studio never writes to.
• The vendored kolibri_content schema should gain the field at VERSION_7, matching Kolibri's definition rather than diverging from it.
• The field should reach both a published channel database and the public import-metadata API, since Kolibri imports through either.
• Existing rows should be populated without re-uploading or republishing, because the affected files are the largest ones we hold.
• The threshold should be a named constant with its coupling to the infrastructure trigger recorded, since a mismatch produces URLs pointing at objects that were never copied.

Out of Scope

• Adding the field to Kolibri's base_models.py and generating the VERSION_7 schema. That lands in Kolibri first, and this issue vendors the result.
• Copying objects into R2, and the one-off backfill of objects already over the threshold.
• Changing how files are uploaded to Studio — resumable uploads and the file_size widening are learningequality/kolibri-ecosystem#53.
• Populating upstream_url with anything other than the R2 URL. Hosts Studio never writes to come later.

Acceptance Criteria

• File.upstream_url is nullable, and is set at upload for files above the mirroring threshold.
• A file at or below the threshold is uploaded with upstream_url unset.
• LocalFile.upstream_url is present in the vendored VERSION_7 schema in both kolibri_content and kolibri_public.
• Publishing a channel writes each file's upstream_url into the channel database.
• A client requesting schema_version=6 from the import-metadata API still imports successfully with the field present in the response.
• Existing File rows above the threshold have upstream_url populated by a backfill that is idempotent and resumable.
• The backfill leaves rows at or below the threshold unchanged.
• The threshold constant matches the size filter on the infrastructure copy trigger.
• A channel published before this change still imports into Kolibri without error.

AI usage

I used Claude (Opus 5, via le-skills:writing-github-issues) to work through the design and draft this issue section by section. The decision to record where a file lives as data on the file, rather than making it an infrastructure concern, was mine, as was the field name. I had Claude read the publish and import-metadata paths to ground the constraints, and edited the drafts where they over-specified or got details wrong.

Start with kolibri_content/base_models.py and kolibri_public/models.py, then trace File conversion in publish.py:714, mapping in mapper.py:225, and field selection in import_metadata_view.py:160. Check the vendored schema_versions.py and the infrastructure trigger's 512 MiB filter. Done means the nullable field flows through uploads, publishing, both API paths, and an idempotent resumable backfill without breaking older imports.