Preserve cohort projects, peer assignment/review, voting, and scoring

Parent epic: #5. Downstream aggregate consumer: #57. Final management parity: #59. Protected rehearsal: #60.

PM disposition

GROOMED / P0 / DEPENDENCY-BLOCKED. Do not dispatch engineering.

This issue preserves the adopted cohort-owned Project, rubric assignment, submission, required/volunteer peer-review, voting, scoring/pass/result/statistics behavior behind shared courses services and supplies one exact, PII-free project component result to #57. It does not redesign the rubric or grading formula, own Enrollment preferences, send email, inspect protected data, or execute a production migration.

Normative authority

• _docs/PROCESS.md
• 04 — Preserved learner behavior, Studio coverage, migration checks
• 06 — Studio/admin API registry, authorization, revisions, operations, and audit
• 07 — learner privacy, grading repair, retention, SSRF, and redaction
• 09 — course migration and score reconciliation
• 10 — Course/Cohort verification
• accepted copied-source characterization/provenance baseline #30 and shared operation/job primitives #31
• #230's exact target-owned malformed peer-review-link recovery contract

Exact owned model and mutation boundary

Project, its ordered criterion assignments, ProjectSubmission, PeerReview, CriteriaResponse, ProjectEvaluationScore, ProjectVote, and project statistics remain owned by one exact Cohort. A ProjectSubmission belongs to one Project and one Enrollment in that Project's Cohort; its retained learner compatibility reference must resolve to the same durable learner as the Enrollment. A PeerReview's target and reviewer submissions belong to the same Project/Cohort, differ from each other, and use the exact required/optional disposition. A CriteriaResponse belongs to that PeerReview and to a criterion assigned to the target Project's frozen ordered rubric.

Unknown, missing, duplicate, stale, cross-Project, cross-Enrollment, cross-Cohort, self-review, wrong-rubric, or mismatched learner state fails before mutation and is never repaired by family, slug/year, email, newest Cohort, or arbitrary primary-key inference. Safe archive/delete protects every definition, assignment, submission, review, response, vote, evaluation, result, and statistic with dependent history.

Preserve Project states closed, collecting submissions, peer reviewing, and completed, exact deadline boundaries, GitHub/commit submission create/update/delete rules, optional fields, results/statistics, vote idempotency, and public/learner compatibility schemas. Scope and reviewer authority are resolved before object lookup. Reads never create an Enrollment, ProjectSubmission, review, vote, or result; the characterized volunteer action is the only explicit path that may create its bounded reviewer-only shell.

Submission and URL safety

Learner submission/update and peer-review create/update are atomic service mutations. Invalid or stale input re-renders the same canonical Cohort form with bounded field errors and leaves the prior submission/review, criteria responses, state, timestamps, scores/result identity, and optional fields unchanged.

• GitHub/commit, FAQ, and learning-in-public values receive local syntax/allowlist/private-literal validation in the request. No arbitrary user hostname is synchronously fetched or resolved for score authority.
• Any network verification is separately bounded, asynchronous, SSRF-safe, redirect/size/time limited, and advisory unless an explicitly groomed later contract says otherwise. Failure never creates a partial submission/review.
• Preserve #230 exactly: malformed peer-review learning-in-public input returns HTTP 200 on the same canonical Course/Cohort evaluation form with Learning in public links must be valid HTTP or HTTPS URLs. and no write. ProjectCriteriaValidationError retains precedence and HTTP 400 for forged, stale, or cross-Project criteria; a broad catch must not weaken that integrity boundary.

Exact required and volunteer peer-assignment rules

Required assignment consumes the frozen ordered set of ordinary (volunteer_review_only=false) submissions for one Project. Let k = Project.number_of_peers_to_evaluate and N be the eligible submission count:

• k must be positive and N > k; otherwise the operation fails with no assignments or state/deadline change.
• every eligible submission receives exactly k distinct required reviewers and every eligible submitter receives exactly k distinct required targets;
• reviewer and target are never the same logical Submission/Enrollment, and no pair is duplicated or crosses a Project/Cohort;
• the assignment set is deterministic for the frozen ordered submission identities and assignment-policy revision, not dependent on unordered query results;
• one successful activation atomically creates the complete required graph, transitions the Project from collecting submissions to peer reviewing, and sets the characterized seven-day review window rounded to the next hour; and
• exact command replay returns the original graph/result, changed replay conflicts, and concurrent attempts cannot create a second or partial graph.

An optional/volunteer review targets one ordinary submission in the same Project. A learner with an ordinary submission uses it as reviewer; a learner without one may receive exactly one reviewer-only volunteer_review_only=true shell for that Project. Self-review and duplicate reviewer-target pairs are denied. Optional review responses contribute to the target Submission's rubric score exactly like other submitted reviews, but an optional review never counts toward the reviewer's required-review count, peer-review points, or reviewed_enough_peers. The reviewer-only shell is never a #57 score/eligibility component. Add/update/delete/closed-state behavior remains source-characterized and cannot delete another learner's review.

Exact adopted project scoring and pass rules

A Project can score only from the peer-reviewing state, after the peer-review due date, with a positive Cohort project_passing_score and at least one in-scope PeerReview row. Otherwise scoring fails without mutation.

For each ordinary ProjectSubmission represented in the frozen scoring input:

• project_score is the sum across the Project's exact ordered assigned rubric. For each criterion with submitted responses, take the preserved numeric score for each response, compute the median, and round a fractional median upward. Where the adopted characterization supplies no submitted evaluation for the Submission, preserve its criterion-middle fallback exactly; no Studio/API/migration adapter invents another fallback.
• project_faq_score is 1 only when the enabled FAQ field contains an accepted DataTalksClub FAQ issue/pull-request URL under the preserved validator; otherwise it is 0.
• project_learning_in_public_score is 0 when the Enrollment disables learning in public; otherwise it is the accepted normalized/de-duplicated Project-submission link count capped by learning_in_public_cap_project.
• peer_review_score = submitted_required_review_count * Project.points_for_peer_review; optional reviews never contribute.
• peer_review_learning_in_public_score is 0 when the Enrollment disables learning in public; otherwise it is the sum, over every required or optional review submitted by this reviewer, of each review's accepted normalized/de-duplicated link count capped separately by learning_in_public_cap_review. Optional reviews contribute this learning-in-public component but never the base peer-review points or required-review count.
• reviewed_enough_peers = submitted_required_review_count >= Project.number_of_peers_to_evaluate.
• total_score = project_score + project_faq_score + project_learning_in_public_score + peer_review_score + peer_review_learning_in_public_score.
• passed = (project_score >= Cohort.project_passing_score) and reviewed_enough_peers. Bonus, FAQ, learning-in-public, vote count, total score, visibility, complaint, certificate state, and volunteer-only activity never independently make a Project pass.

There is no second rubric, score, or pass formula in Studio, API, statistics, migration, #57, or a compatibility adapter. ProjectVote remains one vote per voter/Submission and never changes score, pass, or #57 eligibility.

A scoring run freezes the Project/settings/rubric revision, submission/review/response inputs, and Enrollment preference revisions; stages evaluation scores, every component/total/pass result, statistics, Project completion, and downstream invalidation intent; then activates one complete Project revision atomically. Exact command replay returns the original operation/result; changed replay conflicts. Concurrent submission/review/rubric/settings/preference change makes the candidate stale and leaves the prior complete revision active. Long/full-project assignment, scoring, rescore, reassignment, and repair are bounded durable operations with progress, safe per-row failures, cancellation before activation, lease/fence crash recovery, and no request-thread bulk loop.

Disabling learning in public through #244's accepted Enrollment-preference service immediately produces new #56 component results with both learning-in-public components zero and recomputed totals; it neither deletes links/reviews nor changes rubric/FAQ/required-review points/pass. Re-enabling does not silently recreate points: an explicit authorized recomputation under the current input/settings revision is required. #244 owns the preference mutation and UI; #56 owns only the project recomputation primitive and result.

Project statistics are derived only from activated in-scope #56 results and preserve characterized count/min/max/average/median/quartile/time/pass behavior, null/empty distinctions, and calculation timestamp metadata. Statistics and votes never become scoring inputs.

Exact handoff to #57

The shared courses query/service exposes one immutable, PII-free ProjectSubmissionScoreResult for each activated ProjectSubmission revision with exactly:

• schema version and deterministic result_key;
• Course UUID, Cohort UUID, Enrollment UUID and retained Enrollment ordering identity;
• retained Project and ProjectSubmission identities;
• activated Project score revision;
• project_score, project_faq_score, project_learning_in_public_score, peer_review_score, peer_review_learning_in_public_score, and total_score;
• submitted_required_review_count, positive required_review_count, reviewed_enough_peers, positive points_to_pass, passed, and volunteer_review_only;
• ordered rubric/evaluation input fingerprint, reviewer-activity fingerprint, scoring/settings fingerprint, and Enrollment learning-in-public preference revision; and
• reconciliation disposition and calculated timestamp as metadata, neither of which changes logical identity.

result_key is deterministic over the schema version, exact identities, accepted ordered input/result fingerprints, and score/settings/preference revision. It contains no learner name/email, repository/commit value, review/response/free text, submitted URL, profile value, or reversible learner identifier. Exact replay produces the same key and values; changed submission, review, rubric, setting, preference, scope, or schema produces a new key and score revision. A stale, unactivated, duplicated, quarantined, or foreign result cannot be consumed.

The compatibility ProjectSubmission component/total/pass fields are an atomic projection of the activated result, not a second authority. #57 sums exactly one current accepted #56 total_score per logical ProjectSubmission with volunteer_review_only=false for the exact Cohort/Enrollment, and counts passed=true only in that same non-volunteer set. It includes the ordered #56 result identities in its own fingerprint and never recomputes rubric responses or pass. Unknown, duplicate, stale, quarantined, or cross-Cohort rows block activation rather than being guessed or moved.

Shared management and notification boundary

Register at least these exact capability keys through accepted #52 policy:

• courses.project.view and courses.project.manage;
• courses.project_submission.view and courses.project_submission.repair;
• courses.peer_review.assign, courses.peer_review.reassign, and courses.peer_review.repair;
• courses.project.score;
• courses.project_statistics.view; and
• courses.project_notification.initiate.

Studio and /api/v1/admin/ use the same courses query/command services for Project/criterion definitions, deadlines, assignment/reassignment, submissions/rubrics, votes, score/rescore, statistics, repair, and notification-intent initiation. Mutations use UUID resources where supplied by the accepted foundations, revision/If-Match, Idempotency-Key, allowlisted bodies, generic scope-first errors, bounded bulk operations, and the accepted #28/#52 high-risk preview/confirmation/reauthentication/audit controls. No adapter, compatibility view, task, signal, or Django-admin action writes assignment/score/result/statistics state directly.

#56 owns only versioned domain events/triggers and immutable scalar context after commit. It creates no renderer, sender, provider call, Datamailer write/requeue, Relay call, delivery status, or recipient export. #49 supplies the ordinary delivery-intent interface to the separately groomed #50 adapter; #50 consumes accepted #56 triggers later and is not a prerequisite for this non-sending component result.

Migration, reconciliation, and evidence identity

Repository verification uses deterministic synthetic fixtures only. #60 owns any separately authorized production-like rehearsal and cutover.

The engineer handoff freezes and records:

• full base/head SHAs and change-selective graph/plan/report digests;
• adopted source pin 98a235283904b4ef9ad29e196298540756cf1bcc, #30 copied-file manifest digest, and current integration-patch-ledger digest;
• exact prerequisite merge SHAs, migration leaves, and schema/migration-plan fingerprint;
• #51 Course/Cohort/retargeted-relation/mapping fingerprint;
• #52 capability/policy/registry/OpenAPI fingerprint;
• #53 lifecycle/route/alias/archive/duplication fingerprint;
• #230 merge SHA, CMP source fix 56cac9a8f15a1f183de35a5e18040c49e758ff4c, and target-adaptation ledger identity;
• #244 Enrollment identity/preference/progress-route fingerprint;
• #56 rubric/assignment/review/vote/scoring/result/statistics/operation/domain-event schema versions plus the complete characterized-fixture checksum; and
• #57 project-component consumer-schema fingerprint current at dispatch.

Synthetic preflight/dry-run/apply/exact replay/reverse/forward reports Project/criterion-assignment/Submission/PeerReview/CriteriaResponse/Vote/evaluation/result/statistics counts, retained key and Cohort/Enrollment mapping cardinalities, required/optional/volunteer matrices, component/total/pass matrices, preference states, transformed/rejected/quarantined rows, and safe non-PII checksums. It preserves null/blank distinctions, timestamps, assignment pairs, rubric outcomes, votes, scores, pass/results/statistics, and sibling-Cohort isolation. Unknown mapping, duplicate logical submission/reviewer pair/response/vote/result, learner/Enrollment mismatch, self/cross-Project/cross-Cohort relation, unassigned criterion, invalid threshold/cap/assignment setting, unsupported state, or unexplained score/pass/statistics drift fails closed and leaves the prior revision active. Evidence contains no repository/commit value, response/review/free text, submitted URL, learner identity, profile/certificate value, or production data.

Dependencies and readiness

Accepted baseline inputs are #30 and #31.

Engineering hard-depends on accepted and integrated:

1. #51 — final Course/Cohort identity, mechanically retargeted Project/review/result relations, and migration/rollback contract;
2. #52 — exact Course/Cohort capability, object/field policy, management registry, and audit boundary;
3. #53 — final Cohort lifecycle, canonical/legacy route identity, archive protection, and definition-only duplication behavior;
4. #230 — target-owned malformed peer-review-link recovery and criterion-integrity precedence; and
5. #244 — final cohort-isolated Enrollment identity, learning-in-public preference semantics, learner progress/dashboard routes, and preservation/migration mapping.

#224 is transitive through #51; #32/#33/#40/#288 are transitive through #52/#244. Broad parent #54 is coordination only. #49/#50 are downstream notification integration. #57 consumes the accepted component result; #59 consumes management coverage; #60 consumes migration/reconciliation evidence. None is a prerequisite.

Readiness remains GROOMED / DEPENDENCY-BLOCKED. After all five hard inputs are accepted and integrated on one green current-main base, PM must revalidate the frozen identities and #57 consumer schema before an isolated engineer lane starts. A local candidate, current source model, copied characterization test, or partial migration does not imply acceptance. Any prerequisite, schema, migration, capability, route, preference, source-pin, or consumer drift returns #56 to PM.

Acceptance criteria

• Every Project/rubric/Submission/PeerReview/CriteriaResponse/Vote/evaluation/result/statistics relation and mutation is scoped to one exact Cohort/Enrollment, with all named mismatch/duplicate/self/cross-scope states rejected before mutation.
• Learner submission/update and required/volunteer review behavior remains source-characterized; URL validation is atomic and SSRF-safe, and #230's exact HTTP 200 versus criterion-integrity HTTP 400 precedence is preserved.
• Required assignment satisfies the exact N > k, balanced distinct non-self graph, deterministic/idempotent/concurrency-safe activation, state, and deadline rules; optional/volunteer activity has the exact bounded disposition above.
• The exact rubric median/fallback, FAQ, learning-in-public, peer-review, total, reviewed-enough, pass, vote-neutral, and preference/re-enable formulas above are the sole authority.
• ProjectSubmissionScoreResult has the exact PII-free schema and deterministic identity; exact replay is stable, changed/stale/foreign/quarantined input fails safely, compatibility fields are an atomic projection, and #57 can consume total/pass/volunteer disposition without recalculating reviews.
• Assignment/scoring/rescoring/reassignment/repair and statistics are revisioned, idempotent, concurrency-safe, durable when long, atomically activated, crash/resume safe, and leave the prior complete revision visible on failure.
• Safe archive/delete protects dependent history; Cohort duplication copies definitions/ordered rubric/configuration only and never learner Submissions, reviews, responses, votes, result identities, statistics, or history.
• Studio/admin API/compatibility adapters have service, capability, scope, revision, idempotency, operation, result/error, audit, and OpenAPI parity; no adapter writes domain state directly.
• Notification work stops at after-commit versioned domain triggers/scalar context; no sender/provider/Datamailer/Relay or recipient-export behavior is introduced.
• Synthetic migration/reconciliation preserves mappings, values, timestamps, assignment/review graph, votes, scores, pass/results/statistics, and sibling isolation; every named corruption/drift case fails closed while protected rehearsal remains #60.
• Focused model/service/migration/API/security/concurrency/adoption tests plus graph-selected full verification pass on one frozen candidate; the independent tester inspects every required desktop/mobile screenshot before PM acceptance.

Required verification scenarios

1. Project closed/collecting/peer-reviewing/completed and deadline/DST/leap-day boundaries; valid/invalid GitHub/commit/FAQ/learning links; create/update/delete/double-submit; #230 malformed link, forged/stale criterion, and transaction rollback.
2. Required assignment with N=0/1/k/k+1/many, exact balanced graph, unordered input, exact/changed replay, concurrent workers, crash before/during activation, self/duplicate/cross-Project/Cohort inputs, and deterministic deadline transition.
3. Learner with/without ordinary submission adds/updates/deletes optional review; volunteer shell uniqueness, self/duplicate denial, closed state, another learner's review, and proof that optional review affects only the target rubric while never granting required-review points or a #57 component.
4. Every rubric response type and option boundary; zero/one/even/odd response median and upward rounding; no-submitted-evaluation fallback; missing/unassigned/duplicate response; threshold below/at/above; insufficient/sufficient required reviews; vote zero/one/replay.
5. FAQ and Project/review learning links zero/at/beyond caps and duplicated; preference enabled/disabled/re-enabled; exact five component values, total, reviewed-enough, pass, volunteer disposition, and result-key matrices.
6. Concurrent score/rescore/review/rubric/settings/preference edits; stale revision, changed idempotency replay, crash/resume/cancel, partial-row failure, prior-revision visibility, statistics refresh, and cache/result invalidation.
7. Archive/delete with and without dependent responses/results; duplicate Cohort definitions/ordered rubric then prove no learner/history copy and independent edit.
8. Studio/admin API allowed/denied actors, stale If-Match, exact/changed replay, bulk bound/partial result, progress/cancel/result, high-risk proof, audit/log redaction, CSRF/Bearer separation, OpenAPI/registry coverage, and compatibility parity.
9. Synthetic migration dry-run/apply twice/reverse/forward with missing/duplicate/cross-Cohort relations, invalid assignment/rubric/state/settings, mapping/checksum drift, and all Datamailer/Relay/SES/provider paths patched to fail if called.
10. #57 contract test accepts only the current exact non-volunteer component result set, rejects stale/foreign/quarantined/duplicate/schema-drifted results, reproduces the exact project subtotal and passed count, and performs no rubric-table recalculation.

Independent browser evidence

Using synthetic .invalid fixtures at approximately 1440×900 and 390×844, the separate tester captures and reads:

1. canonical learner Project empty/collecting/peer-reviewing/completed states and invalid/successful create/update;
2. assigned review list, valid review update, #230 malformed-link recovery, forged-criterion safe failure, optional review add/remove, volunteer path, and sibling-Cohort safe denial;
3. result/pass/fail/statistics/vote states with exact component equation and learning-in-public preference enabled/disabled; and
4. authorized/denied Studio Project/rubric/deadline/assignment-reassignment/score-rescore/repair/statistics operation states including preview, progress, stale/conflict, failure, and success.

Include keyboard-only operation, visible focus/status/error announcement, 44 CSS px targets, 200% zoom and 320 CSS px reflow, reduced motion, and supported JavaScript-disabled form/confirmation/error paths. Screenshots contain no real or unnecessary synthetic learner identity, repository/commit value, review/response/free text, submitted URL, profile/certificate value, credential, provider, or production data.

Explicit non-goals

• No rubric/scoring/pass redesign, new criterion type, reusable/versioned curriculum, Course/Cohort/Enrollment identity, registration conversion, profile schema, leaderboard/completion/certificate algorithm, or intentional visibility change.
• No arbitrary synchronous URL fetch, destructive deletion of reviewed history, request-thread full-cohort work, score/statistics trust without reconciliation, or direct adapter/model mutation.
• No system evaluation, cross-Cohort review, score hiding, new peer-task visibility, notification-delivery semantics, delivery-intent/recipient implementation, template, sender, Relay/Datamailer/provider action, production/protected data, live migration/import, deployment, workflow dispatch, commit, merge, or push during PM/engineering/tester gates.

Lifecycle

After prerequisites integrate, one engineer leaves an uncommitted frozen candidate and complete versioned verification report. A separate tester recomputes the plan, verifies every criterion, and captures/inspects required screenshots. PM then performs product acceptance. Only after both gates pass may the engineer create a focused Closes #56 commit; the orchestrator locally merges with --no-ff, pushes, and on-call alone observes terminal CI.

This is explicitly dependency-blocked and says not to dispatch engineering. Start by reading _docs/PROCESS.md and _docs/specs/04-courses-and-cohorts.md, then consult specs 06, 07, 09, and 10 plus issues #30, #31, #230, and #244; completion requires preserving the described cohort project behavior and producing the exact PII-free result for #57.