look at reducing depencies for lucene/replicator [LUCENE-9851]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
Lucene/replicator brings in a lot of dependencies (some test-only) for http communication (httpclient, jetty, servet-api, etc), I think it is worth looking at reducing these.
It isn't gonna need to be used as a typical high-qps webserver, instead low level transfer of index segments. IMO this is a more delicate operation where we actually care about stuff like how/if data is compressed and want it done efficiently.
I've had good experiences in production with java 11's new http client: https://docs.oracle.com/en/java/javase/11/docs/api/java.net.http/java/net/http/package-summary.html
It is maintained by openjdk and built into java's core, so we could remove the httpclient dependency and not cause conflicts with downstream projects.
On the server side I think jetty may be overkill, we could use jdk's built-in Http/HttpsServer, too. Because we don't need fancy nonblocking io / resource pooling / whatever, its smaller number but MUCH BIGGER than average http transfers going on.
We can keep replicators apis generic so that users can still use different protocols or methods, this is really just about the **implementations** and **tests** that we supply.
---
Migrated from [LUCENE-9851](https://issues.apache.org/jira/browse/LUCENE-9851) by Robert Muir (@rmuir)
Contributor guide
Research direction
Start by locating the lucene/replicator implementations, supplied tests, and dependency declarations, then inventory which httpclient, Jetty, servlet-api, and other dependencies are runtime-only or test-only. Compare the implementation options mentioned in the issue, including Java 11's HttpClient and the JDK Http/HttpsServer; done means generic replicator APIs remain intact while unnecessary dependencies are removed and the implementations and tests still work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, networking
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100