apache / apache/lucene

Write a HOWTO migrate Codec format version

Open
#12,918 1 comment 2 reactions 0 assignees View on GitHub
type:documentation
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

The change to changing PFOR encoding to FOR for doc blocks in https://github.com/apache/lucene/pull/12741, required bumping the Codec version from 95 to 99.

Bumping the Codec is not a straightforward process since it involved:
* Forking all modified changed files and bumping the version number
* Moving the original copies of the modified files to backwards_codecs
* Re-generating ForUtil
* \+ changes to [forUtil.gradle](https://github.com/apache/lucene/pull/12741/files#diff-6bfb3ba57326f955b44ec86c40070c4e518bffb45cf6226504eab043d545ffbd) to bump version and add a new task for regenerating backwards_codec
* Forking PostingsFormat subclasses
* Changes to tests:
* Bumping existing Codec unit tests
* Creating RW versions of old Codec and Codec components
* Bumping Codec components where used throughout other tests

We required a few follow-ups to clean up the initial misses:
* https://github.com/apache/lucene/pull/12781
* https://github.com/apache/lucene/pull/12776
* https://github.com/apache/lucene/pull/12908
* https://github.com/apache/lucene/pull/12904

@mikemccand mentioned in https://github.com/apache/lucene/pull/12904#issuecomment-1849951055
> Codec format migrations are so tricky ... I wish we had a HOWTO somewhere.

Since I done the last Codec bump, I can have a look at writing up the documentation so subsequent codec format bumps will go smoother.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the Codec migration described in PR 12741, its follow-ups, and the changes to forUtil.gradle. Use the listed steps around ForUtil, backwards_codecs, PostingsFormat subclasses, and Codec tests to structure the HOWTO. Done means documenting the migration sequence and the missed follow-up considerations so future Codec bumps are more predictable.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.