CenterForDigitalHumanities / CenterForDigitalHumanities/rerum_server_nodejs

Atlas Search index definitions are not in the repository

Open
#308 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
3
Forks
6
Avg merge
1h 25m
Merged PRs (30d)
3

Description

## Summary

`/search` depends on two Atlas Search indexes, `presi3AnnotationText` and `presi2AnnotationText` (`controllers/search.js:84,153`). Their definitions exist only in the Atlas UI. Nothing in this repository describes which fields are indexed, with which analyzers, or how the two differ.

The consequence is that the current search behavior is not reviewable, not reproducible, and not diffable from source. A field path in `controllers/search.js` that the index does not cover fails silently — it simply matches nothing — and there is no artifact to check it against.

Small issue, no runtime change, and a genuine prerequisite for #309. Worth doing on its own merits regardless of what happens to that one.

## Why this matters

**The proposed fix cannot be reviewed without them.** #309 proposes a single combined index covering both vocabularies. Reviewing that proposal means comparing the new definition against the two it replaces. Right now that comparison can only happen by two people looking at the same browser tab.

**Search behavior can change with no commit.** An index edit in the Atlas UI changes what `/search` returns, with no diff, no review, and no record. When someone later asks why results changed, there is nothing to bisect.

**The pipelines assert paths nothing verifies.** `buildDualIndexQueries()` names specific paths — `body.value`, `bodyValue`, `resource.chars`, `resource.cnt:chars`, and several `embeddedDocument` traversals through `items.annotations.items`, `resources`, `otherContent`, and `sequences.canvases.otherContent.resources`. A path the index does not cover contributes zero matches and raises no error. With the definitions checked in, a reviewer can see the mismatch. Without them, only a targeted probe finds it.

**Environment drift is invisible.** Dev and production each have their own indexes. Whether they are actually identical is currently unknown and unknowable from source.

## Affected lines

| File | Line | Current |
|------|------|---------|
| `controllers/search.js` | 84 | `index: "presi3AnnotationText"` — defined only in Atlas |
| `controllers/search.js` | 153 | `index: "presi2AnnotationText"` — defined only in Atlas |
| `controllers/search.js` | 82-222 | `buildDualIndexQueries()` asserts field paths nothing in the repo verifies |

## Proposed change

Export both index definitions as JSON and commit them. `openapi/` is the existing precedent for a checked-in contract that describes something outside the code, so a sibling location under it, or a `database/` subdirectory, both fit; pick whichever matches how the team thinks about it rather than adding a new top-level directory.

Include for each index:

- The full JSON definition as Atlas reports it, exported from the UI or via the Atlas Admin API.
- Which deployment it was exported from, and on what date.
- A short README noting that these are a record of Atlas state, not something the application applies at startup, and how to re-export them.

Then check whether dev and production actually match, and record the answer. If they have drifted, that is worth its own issue.

Optionally, add a test that reads the checked-in definitions and asserts every path named in `buildDualIndexQueries()` is covered by the corresponding index. That turns the definitions from documentation into a guard, and it is the thing that would have caught a silently dead path. It is a nice-to-have, not a requirement for closing this.

## Notes

- No runtime change. Nothing depends on this landing, and #309 depends on it having landed.
- Checked-in definitions are a record, not a deployment mechanism. Applying them automatically would need the Atlas Admin API and credentials in CI, which is a larger decision and explicitly out of scope here.
- Related: [the detailed report on #306](https://github.com/CenterForDigitalHumanities/rerum_server_nodejs/issues/306#issuecomment-5529634333) raises this under Notes as "arguably worth doing first, independently of this issue".

## Acceptance criteria

- [ ] Both index definitions are committed as JSON, exactly as Atlas reports them
- [ ] Each records the deployment it came from and the date of export
- [ ] A README explains that these record Atlas state rather than being applied by the app, and documents how to re-export
- [ ] Dev and production definitions are compared and any drift is recorded

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.