apache / apache/lucene

address "monsters" in the nightly tests [LUCENE-9840]

Open
#10,879 0 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

Sometimes you want to run nightly tests yourself (e.g. if you are gonna be busy during the week, if you want more confidence in changes before committing, etc).

But this takes around an hour even on a very fast laptop. Don't want to mess up nightly test coverage, but maybe we can attack some of the worst apples to keep it in check:

```
The slowest tests (exceeding 500 ms) during this run:
1609.65s Test2BPostings.test (:lucene:core)
548.18s Test4GBStoredFields.test (:lucene:core)
408.99s TestXYMultiPolygonShapeQueries.testRandomBig (:lucene:core)
375.53s TestGeo3DPoint.testRandomBig (:lucene:spatial3d)
368.43s TestSearcherManager.testSearcherManager (:lucene:core)
351.70s TestNRTThreads.testNRTThreads (:lucene:core)
349.51s TestControlledRealTimeReopenThread.testControlledRealTimeReopenThread (:lucene:core)
338.09s TestDemoParallelLeafReader.testRandomMultipleSchemaGensSameField (:lucene:core)
336.96s TestSearcherTaxonomyManager.testDirectory (:lucene:facet)
334.17s TestLatLonDocValuesQueries.testRandomBig (:lucene:core)
The slowest suites (exceeding 1s) during this run:
1609.69s Test2BPostings (:lucene:core)
1395.07s TestBestCompressionLucene80DocValuesFormat (:lucene:backward-codecs)
1209.91s TestBestCompressionLucene90DocValuesFormat (:lucene:core)
836.90s TestBestSpeedLucene80DocValuesFormat (:lucene:backward-codecs)
806.67s TestBestSpeedLucene90DocValuesFormat (:lucene:core)
630.09s TestSimpleTextPostingsFormat (:lucene:codecs)
600.18s TestLucene70DocValuesFormat (:lucene:backward-codecs)
548.57s Test4GBStoredFields (:lucene:core)
488.01s TestDemoParallelLeafReader (:lucene:core)
450.85s TestXYMultiPolygonShapeQueries (:lucene:core)

BUILD SUCCESSFUL in 50m 45s
192 actionable tasks: 67 executed, 125 up-to-date
```

---
Migrated from [LUCENE-9840](https://issues.apache.org/jira/browse/LUCENE-9840) by Robert Muir (@rmuir)

Contributor guide

Open the contributing guide

Research direction

Start by running the nightly tests and reviewing the listed slow tests and suites, especially Test2BPostings, Test4GBStoredFields, and TestXYMultiPolygonShapeQueries. Compare their runtime and coverage before deciding which tests can be addressed; done means reducing the nightly run time without compromising nightly test coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
performance, testing-qa
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.