microsoft / microsoft/SynapseML
feat: ship a GPU-enabled LightGBM native so deviceType=gpu can actually train
- Dominant language
- Scala
- Stars
- 5.2k
- Forks
- 868
- Avg merge
- 22h 9m
- Merged PRs (30d)
- 45
Description
#2627 adds a `deviceType` parameter so `device_type=gpu|cuda` can be set on the LightGBM learners. The parameter works — it reaches the native layer, verified by LightGBM rejecting it *by name*. But it cannot actually train on a GPU, because the native library SynapseML ships is CPU-only.
That is the other half of GPU support, and it is a build problem rather than a code problem.
## Evidence
`build.sbt` pins `com.microsoft.ml.lightgbm:lightgbmlib` `3.3.510`. Scanning the bundled `lib_lightgbm.so`:
```
OpenCL 0 occurrences
clCreateKernel 0 occurrences
cuda_tree_learner 0 occurrences
```
Training with `device_type=gpu` fails with `GPU Tree Learner was not enabled in this build.`, and `cuda` with the CUDA equivalent. Those strings are compiled into the shipped binary.
## No dependency bump fixes this
- `com.microsoft.ml.lightgbm:lightgbmlib` publishes one artifact and `3.3.510` is the newest that exists (19 versions, newest 2023-05-15). We are already on it.
- Every other `lightgbm` artifact on Maven Central is either older, a PMML/conversion tool, or a different binding.
- The one newer binding, `io.github.metarank:lightgbm4j` (LightGBM 4.4.0), ships CPU-only natives too. Its README is explicit that GPU requires you to rebuild LightGBM yourself with `-DUSE_CUDA=1 -DUSE_SWIG=ON` and point `LIGHTGBM_NATIVE_LIB_PATH` at the result.
So there is no artifact to switch to. Someone has to build the native.
## What this would involve
1. Build LightGBM from source with `-DUSE_GPU=1` (OpenCL) and/or `-DUSE_CUDA=1`, plus `-DUSE_SWIG=ON`, for each platform we support.
2. Decide how it ships. A GPU build needs an OpenCL/CUDA runtime present at load time, so it likely cannot simply replace the current native in the main jar without breaking CPU-only users — a classifier, a separate artifact, or runtime selection with fallback.
3. Add a load path that prefers the GPU native when present and falls back to the CPU one, so nothing regresses for users without a GPU.
4. Get GPU CI capacity, or accept that this stays manually verified.
## Why it is worth doing separately
Step 2 is the real design question and it affects packaging for every SynapseML user, GPU or not. It should not ride along inside a parameter PR.
Worth noting the ordering is fine either way: #2627's GPU tests assert that a device request is *never silently downgraded to CPU*, accepting either a successful fit or a failure naming the tree learner. On a GPU-capable native they start passing by training instead of by failing, with no test changes needed.
Contributor guide
Research direction
Start with build.sbt and inspect how the bundled lib_lightgbm.so is packaged and loaded. Review the GPU tests from #2627 and determine the artifact or runtime-selection design, including CPU fallback. Done means supported GPU natives can train without silently downgrading, while CPU-only users continue to work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cmake, scala
- Domain
- build-system, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100