Publishing mothra output on Cantus Ultimus

Related: https://github.com/DDMAL/mothra/issues/77
Following discussion with @kyrieb-ekat

**Objective**

Currently, Mothra and Cantus Ultimus (CU) are not directly linked to each other, which means the output from mothra has to be manually downloaded and then uploaded to CU. Ideally we want to be able to
1. initialize a project with manuscripts existing on CU, then a user on mothra can create a new project by "importing" from CU directly
2. "send" completed output from mothra to CU as "pending OMR results" for admin review before it is published

To achieve this, we need to establish a two-way connection between the two systems. Since CU currently has no write API and no non-admin user accounts, Mothra will host both new surfaces, and CU will gain a minimal authenticated write endpoint.

Step (2) comes first. CU gains a single write endpoint authenticated as Mothra-the-application, and Mothra adds a submission page where any logged-in user can select a manuscript and folio (drawn from CU's existing API) and send their completed MEI for admin review. The MEI is stored in CU's database as a pending record; a CU admin then publishes it, requests a correction, or refuses it with a reason. Nothing becomes public without an admin acting.

Step (1) can come later. Mothra can pull a manuscript's IIIF manifest from CU and uses it to populate the project's image set, replacing the current manual upload step. This depends on IIIF manifest import support in Mothra, which does not yet exist; the deposit inbox works regardless of how images arrived in Mothra.

**Goals**

- Any authenticated Mothra user can submit processed MEI for a manuscript folio, no admin needed
- Manuscript selection is driven by CU's existing manuscripts; folios are picked from CU's existing list
- Each submission arrives in CU as a pending record, associated with the manuscript, folio number, submitter's Mothra username, and an optional comment
- A CU admin can publish, request a correction, or refuse a submission with a reason
- Publishing writes the MEI and triggers `index_manuscript_mei `and the folio becomes searchable immediately
- Nothing becomes publicly visible without an explicit admin action
- Submitters can see the current status and any correction/refusal reason on their submissions from within Mothra
- A correction creates a new pending record rather than updating the old one, so history comes for free and no version field is needed.

**Non-goals**

- No email notifications in v1; submission status is visible in Mothra on request, not pushed
- No auto-publishing. Every submission requires an admin to act; removing the review step is not an option
- No custom review UI in v1; review goes through Django admin; a dedicated page can be built later if needed
- Manuscripts not in CDB are out of scope for v1
- No image transfer; CU already holds manuscript images via IIIF; only MEI data crosses the boundary
- No live state sync between the two systems, only submit and check
- No changes to MEI encoding, pitch-finding, or Neon
- No per-folio Solr flushing. Publishing re-indexes the whole manuscript (see below)

**Open questions**

Mothra needs to make an HTTP POST to CU's new write endpoint. CU needs some way to verify the request is actually coming from Mothra and not from anyone who finds the URL.

The open questions are:
- What form does the credential take? An API key in a header? An HMAC signature? Something else?
- Where is it stored? Mothra's .env, presumably alongside `DATABASE_URL` and `MOTHRA_SECRET`?
- Who manages it? Whoever deploys both systems?
- What happens if it's leaked? How do we rotate it without downtime? (The good news is that the credential is leaked, it can only create pending records on CU. It cannot publish anything, because publishing requires a human to act in Django admin. So a bad actor with the
credential can spam the review queue but can't make anything public.

**Risks**

**1. Where does published MEI actually live?**

When we publish a folio, the plan is:

i. Write the MEI file to disk somewhere
ii. Run `index_manuscript_mei`, which reads from that disk location and indexes it into Solr so it becomes searchable

The command defaults to reading from `/code/production-mei-files/`, currently a git submodule, meaning it's a separate git repo of curated, human-committed MEI files. It's the authoritative archive.

The problem is what happens if publishing writes the new MEI file to that same directory inside a running container. The indexing runs fine, the folio appears searchable. But container filesystems are ephemeral: on the next deploy or restart, that directory is reset to whatever was baked into the image. The file disappears. Solr still has the n-grams from before the restart.

To fix this, we can write the MEI to a persistent volume (like the NFS volume Mothra uses for model files), and pass `--mei-dir` pointing there. The `production-mei-files` submodule stays as the curated human archive; newly imported files from Mothra live separately on persistent storage.

**2. "Searchable immediately" needs a mechanism**

`index_manuscript_mei` reads all .mei files in the manuscript's directory and adds their n-grams to Solr with `add_many`. This is fine for a first-time submission, but if folio 001r was indexed before, and you publish a corrected version of 001r, running the command again will pile the new n-grams on top of the old ones. Both versions exist in the index simultaneously. Search results for that folio become corrupted.

To fix this, we might want to delete the old n-grams first. That's what `--flush-index` does, but the code deletes everything for the manuscript and then immediately returns. It does not re-index.
https://github.com/DDMAL/cantus/blob/2c63ad7a75f10fc538b7b854f31a529901d20050/app/public/cantusdata/management/commands/index_manuscript_mei.py#L70-L75

So publishing actually needs two steps:

i. Run `index_manuscript_mei --flush-index` to wipe all OMR records for the manuscript
ii. Run `index_manuscript_mei ` to re-indexes all .mei files in the directory (including the newly published one)

This is slower than a per-folio update because we are re-indexing the whole manuscript, not just one folio.