apache / apache/lucene

Explore GPU acceleration [LUCENE-7745]

Open
#8,796 44 comments 1 reaction 0 assignees View on GitHub
legacy-jira-label:gsoc2017 legacy-jira-label:mentor legacy-jira-priority:Major type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

There are parts of Lucene that can potentially be speeded up if computations were to be offloaded from CPU to the GPU(s). With commodity GPUs having as high as 12GB of high bandwidth RAM, we might be able to leverage GPUs to speed parts of Lucene (indexing, search).

First that comes to mind is spatial filtering, which is traditionally known to be a good candidate for GPU based speedup (esp. when complex polygons are involved). In the past, Mike McCandless has mentioned that "both initial indexing and merging are CPU/IO intensive, but they are very amenable to soaking up the hardware's concurrency."

I'm opening this issue as an exploratory task, suitable for a GSoC project. I volunteer to mentor any GSoC student willing to work on this this summer.

---
Migrated from [LUCENE-7745](https://issues.apache.org/jira/browse/LUCENE-7745) by Ishan Chattopadhyaya (@chatman), 2 votes, updated May 11 2020
Attachments: [gpu-benchmarks.png](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-7745/gpu-benchmarks.png), [TermDisjunctionQuery.java](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-7745/TermDisjunctionQuery.java)

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the attached TermDisjunctionQuery.java and gpu-benchmarks.png, then inspect Lucene's indexing, merging, spatial filtering, and search paths to identify a bounded, benchmarkable target. The issue does not define completion criteria, so a contributor would need agreement on a concrete GPU experiment and success measures.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
performance, search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.