microsoft / microsoft/onnxruntime
[Documentation] A new Java binding for ORT
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the documentation issue
[SMILE](https://github.com/haifengl/smile/blob/master/core/ONNX.md) provides an idiomatic Java API for running ONNX models. The `smile.onnx` package wraps the ONNX Runtime C API through the Panama FFM layer. `InferenceSession` is the central object. It loads an ONNX model, optimizes its graph, and executes inference.
```
import smile.onnx.*;
import java.util.Map;
// Load the model and create an inference session.
try (var session = InferenceSession.create("resnet50.onnx")) {
// Build an input tensor (batch=1, channels=3, height=224, width=224)
float[] pixels = preprocessImage(...); // your image preprocessing
long[] shape = { 1, 3, 224, 224 };
try (OrtValue input = OrtValue.fromFloatArray(pixels, shape)) {
// Run inference
OrtValue[] outputs = session.run(Map.of("data", input));
// Read the result
float[] scores = outputs[0].toFloatArray();
int classId = argmax(scores);
System.out.println("Predicted class: " + classId);
// Release output tensors
for (OrtValue v : outputs) v.close();
}
}
```
In addition, [SMILE Serve](https://github.com/haifengl/smile/blob/master/serve/README.md) is a production-ready inference server built on [Quarkus](https://quarkus.io/). It brings together three complementary inference capabilities on the JVM:
| Capability | API prefix | Description |
|---|---|--------------------------------|
| **ONNX Runtime** | `/api/v1/onnx` | Any model in the ONNX open format (`.onnx`) |
| **Classic ML** | `/api/v1/models` | Serialized SMILE models (`.sml`) — classifiers and regressors |
| **LLM Chat** | `/api/v1/chat` | Llama 3 chat completions with conversation persistence |
Besides the current document [Get Started with ORT for Java](https://onnxruntime.ai/docs/get-started/with-java.html), shall we add a new page to introduce [SMILE](https://github.com/haifengl/smile) as an alternative for Java developer community to leverage ONNX?
### Page / URL
https://onnxruntime.ai/docs/get-started/with-java.html
Contributor guide
Research direction
Start with the existing Get Started with ORT for Java page at the provided URL, then review SMILE's linked ONNX.md and Serve README. Add the proposed SMILE alternative content with the Java example, API descriptions, and links, and verify that the documentation accurately explains the three listed inference capabilities.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- documentation, machine-learning
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 70/100