Classification: AI peer matching experiment

Context

Deodar's Use Case 1 (submission cleanup) is parked — volume (~700–800/year) doesn't justify AI. The real problem is Use Case 2: classifying writers for peer matching. New writers need credible feedback and want peer groups at or above their own skill. Deodar can bucket 30–40 stories by hand; the question is whether AI can do this reliably at 50–100+ writers. The AI's job is classification at the entry point only — assign a writer to the right room; everything after is human-to-human.

Consent blocker & workaround

Deodar needs to take permission from writers at submission and the stories are the writers' own product, so real submissions can't be sent. Workaround: Deodar assembles a dummy set of ~30 short stories (good/middling/bad, free to share) plus a written guideline (not a rubric) on what makes writing good/bad and what characterises Indian fiction.

First experiment

1. Give the AI the 30 samples + guideline; let it bucket organically into top/middle/bottom.
2. Compare its buckets against Deodar's.
3. Ask the AI to propose its own rubric; score and give feedback per story; sample-check; iterate.

• No model training — entirely prompt engineering (3–6 page prompts workable). First round will underperform; value is in iterating the rules.
• Platform fit: the existing AI Assessments pipeline works (opinionated toward assessment, but classification uses the same rubric-in/scored-buckets-out mechanism). Kaapi stays involved for 2–3 iterations, then hands Deodar a UI to self-iterate.

Notes

• Product shape (login → upload → AI feedback emailed; gated persona → room assignment) is exploratory, not committed. Platform must disclose AI is the first-level reader.
• Volume assumptions (50–100 simultaneous writers) are aspirational; market viability unvalidated; no internal deadline.

Next steps (Kaapi)

• Help structure the prompt and rubric; run the first rounds jointly; provide self-serve platform access once early rounds show promise.