apache / apache/lucene

look at reducing depencies for lucene/replicator [LUCENE-9851]

Open
#10,890 1 comment 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

Lucene/replicator brings in a lot of dependencies (some test-only) for http communication (httpclient, jetty, servet-api, etc), I think it is worth looking at reducing these.

It isn't gonna need to be used as a typical high-qps webserver, instead low level transfer of index segments. IMO this is a more delicate operation where we actually care about stuff like how/if data is compressed and want it done efficiently.

I've had good experiences in production with java 11's new http client: https://docs.oracle.com/en/java/javase/11/docs/api/java.net.http/java/net/http/package-summary.html
It is maintained by openjdk and built into java's core, so we could remove the httpclient dependency and not cause conflicts with downstream projects.

On the server side I think jetty may be overkill, we could use jdk's built-in Http/HttpsServer, too. Because we don't need fancy nonblocking io / resource pooling / whatever, its smaller number but MUCH BIGGER than average http transfers going on.

We can keep replicators apis generic so that users can still use different protocols or methods, this is really just about the **implementations** and **tests** that we supply.

---
Migrated from [LUCENE-9851](https://issues.apache.org/jira/browse/LUCENE-9851) by Robert Muir (@rmuir)

Contributor guide

Open the contributing guide

Research direction

Start by locating the lucene/replicator implementations, supplied tests, and dependency declarations, then inventory which httpclient, Jetty, servlet-api, and other dependencies are runtime-only or test-only. Compare the implementation options mentioned in the issue, including Java 11's HttpClient and the JDK Http/HttpsServer; done means generic replicator APIs remain intact while unnecessary dependencies are removed and the implementations and tests still work.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, networking
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.