microsoft / microsoft/onnxruntime

[Documentation] A new Java binding for ORT

Open Beginner friendly
#28,239 4 comments 0 reactions 0 assignees View on GitHub
api:Java documentation
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the documentation issue

[SMILE](https://github.com/haifengl/smile/blob/master/core/ONNX.md) provides an idiomatic Java API for running ONNX models. The `smile.onnx` package wraps the ONNX Runtime C API through the Panama FFM layer. `InferenceSession` is the central object. It loads an ONNX model, optimizes its graph, and executes inference.

```
import smile.onnx.*;
import java.util.Map;

// Load the model and create an inference session.
try (var session = InferenceSession.create("resnet50.onnx")) {
// Build an input tensor (batch=1, channels=3, height=224, width=224)
float[] pixels = preprocessImage(...); // your image preprocessing
long[] shape = { 1, 3, 224, 224 };

try (OrtValue input = OrtValue.fromFloatArray(pixels, shape)) {
// Run inference
OrtValue[] outputs = session.run(Map.of("data", input));

// Read the result
float[] scores = outputs[0].toFloatArray();
int classId = argmax(scores);
System.out.println("Predicted class: " + classId);

// Release output tensors
for (OrtValue v : outputs) v.close();
}
}
```

In addition, [SMILE Serve](https://github.com/haifengl/smile/blob/master/serve/README.md) is a production-ready inference server built on [Quarkus](https://quarkus.io/). It brings together three complementary inference capabilities on the JVM:
| Capability | API prefix | Description |
|---|---|--------------------------------|
| **ONNX Runtime** | `/api/v1/onnx` | Any model in the ONNX open format (`.onnx`) |
| **Classic ML** | `/api/v1/models` | Serialized SMILE models (`.sml`) — classifiers and regressors |
| **LLM Chat** | `/api/v1/chat` | Llama 3 chat completions with conversation persistence |

Besides the current document [Get Started with ORT for Java](https://onnxruntime.ai/docs/get-started/with-java.html), shall we add a new page to introduce [SMILE](https://github.com/haifengl/smile) as an alternative for Java developer community to leverage ONNX?

### Page / URL

https://onnxruntime.ai/docs/get-started/with-java.html

Contributor guide

Open the contributing guide

Research direction

Start with the existing Get Started with ORT for Java page at the provided URL, then review SMILE's linked ONNX.md and Serve README. Add the proposed SMILE alternative content with the Java example, API descriptions, and links, and verify that the documentation accurately explains the three listed inference capabilities.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
documentation, machine-learning
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
70/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.