OWASP / OWASP/OpenCRE

MyOpenCRE mapping: reuse Module C Librarian retriever/reranker (RFC TODO 3)

Open
#965 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
180
Forks
137
Avg merge
3d 23h
Merged PRs (30d)
21

Description

Context

RFC #876 TODO 3 defines application/utils/mapping_job.py with bi-encoder → pgvector top-20 → cross-encoder rerank. GSoC Module C (The Librarian) implements the same pipeline:

  • C.1 candidate retriever — #937 (approved)
  • C.2 cross-encoder reranker — #957

Maintainer decision on #876: do not build a second parallel mapping stack. Wire MyOpenCRE run_mapping to reuse Module C components where possible.

Requirements

  • Implement run_mapping by calling librarian/candidate_retriever + librarian/cross_encoder (or shared interfaces) instead of duplicating encode/rerank logic
  • Map RFC user_standard_sections status enum (AUTO_MAPPED, NEEDS_REVIEW, etc.) to Librarian confidence thresholds — document mapping in RFC or inline
  • rq job remains in mapping_job.py but delegates retrieval/rerank to Librarian modules
  • Tests: fixture upload produces deterministic status counts; USER_CONFIRMED rows never auto-remapped (RFC constraint)
  • Coordinate merge order: Module C W1–W3 (#922 → #925 → #937) before or in parallel with MyOpenCRE TODO 3

Out of scope

  • Full Librarian graph-write / HITL flow (Module C C.3–C.5)
  • Per-user embedding storage (RFC explicitly forbids)

Related

  • docs/rfc/user-auth-myopencre.md TODO 3
  • #586, Module C PRs #922, #925, #937, #957

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with application/utils/mapping_job.py and docs/rfc/user-auth-myopencre.md TODO 3, then read the Librarian retriever and cross-encoder work in #937 and #957. Verify that fixture uploads produce deterministic status counts and that USER_CONFIRMED rows are never auto-remapped. Coordinate against the Module C merge sequence described in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, databases, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.