Turn asserts in I/O related code into hard checks [LUCENE-4196]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
In lots of codecs we only assert, that e.g. some things inside files are correctly in bounds, which leads to security problems (ok, not as bad as C-Style buffer overflows), but e.g. allocating a large array after reading a VInt from a file header and then OOM, is a security issue. So we have to check all those contracts for files as hard checks, especially as a simply check in most cases dont cost anything (and it costs not more than the assert itsself, as the assert also takes CPU power, because it needs a check one time on a static final class field).
Of course we cannot check values we read when reading postings, but the simple checks that any postings file has correct header and something like a positive number of elements, or number of elements < file size,..., a bit-fireld only contains valid bits in StoredFieldsReader, or non-duplicate filenames (CFS) are very important. We had those checks in 3.x, but in 4.0, Mike changed all of those to asserts during the flex development (in my opinion with no real reason).
---
Migrated from [LUCENE-4196](https://issues.apache.org/jira/browse/LUCENE-4196) by Uwe Schindler (@uschindler), updated Mar 22 2013
Attachments: [LUCENE-4196.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-4196/LUCENE-4196.patch)
Contributor guide
Research direction
Start by reading the issue and the attached LUCENE-4196.patch, then audit the assert-based checks in the codecs and I/O areas mentioned: postings headers, StoredFieldsReader bit fields, and compound-file filenames. Compare the existing checks with the stated file-boundary and value constraints; done means relevant file contracts fail with hard checks rather than relying on assertions, with tests covering the affected cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, search
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100