apache / apache/camel-quarkus

Milo integration tests native build fails due to PermanentBailoutException: Number of elements in a node list too high

Open
#9,001 0 comments 0 reactions 0 assignees View on GitHub
area/milo area/native area/testing bug native test
Dominant language
Java
Stars
302
Forks
232
Avg merge
1d 20h
Merged PRs (30d)
114

Description

### Bug description

Native image builds fail during `[5/8] Inlining methods...` with a `PermanentBailoutException` raised from the single callsite inlining stage (`-H:+AOTSingleCallsiteInline`, added by [graalvm/mandrel@11b6c9c](https://github.com/graalvm/mandrel/commit/11b6c9c87a9a) "single callsite inlining").

Quarkus 3.39 enables this option automatically for non-Oracle distributions in the version range `[25.0.4, 25.1.0)`, so applications hit it by default rather than opting into it ([quarkusio/quarkus@2071c57](https://github.com/quarkusio/quarkus/commit/2071c57b3614099fa955cb41dac8295d0247b097), on the 3.39 branch as a cherry-pick of quarkusio/quarkus#55482):

```java
if (graalVMVersion.compareTo(io.quarkus.runtime.graal.GraalVM.Version.VERSION_25_0_4) >= 0
&& graalVMVersion.compareTo(io.quarkus.runtime.graal.GraalVM.Version.VERSION_25_1_0) < 0
&& graalVMVersion.getDistribution() != Distribution.ORACLE) {
final List additionalBuildArgs = NativeConfigUtils.getNativeAdditionalBuildArgs(nativeConfig);
if (additionalBuildArgs.stream().noneMatch(arg -> arg.contains("AOTSingleCallsiteInline"))) {
log.info("Single callsite inlining has been enabled for this build. ...");
addExperimentalVMOption(nativeImageArgs, "-H:+AOTSingleCallsiteInline");
}
}
```

## CI build logs

```
[2/8] Performing analysis... [*****] (88.9s @ 4.50GB)
22,896 types, 29,324 fields, and 103,018 methods found reachable
[3/8] Building universe... (12.0s @ 4.93GB)
[4/8] Parsing methods... [***] (9.7s @ 5.24GB)
[5/8] Inlining methods... [*****] (13.7s @ 6.07GB)

Fatal error: jdk.graal.compiler.core.common.PermanentBailoutException: Number of elements in a node list too high: 65537
at jdk.graal.compiler/jdk.graal.compiler.graph.NodeList.checkMaxSize(NodeList.java:124)
at jdk.graal.compiler/jdk.graal.compiler.graph.NodeList.add(NodeList.java:180)
at jdk.graal.compiler/jdk.graal.compiler.graph.NodeList.add(NodeList.java:40)
at jdk.graal.compiler/jdk.graal.compiler.nodes.AbstractMergeNode.addForwardEnd(AbstractMergeNode.java:74)
at jdk.graal.compiler/jdk.graal.compiler.nodes.AbstractMergeNode.simplify(AbstractMergeNode.java:226)
at jdk.graal.compiler/jdk.graal.compiler.phases.common.CanonicalizerPhase.tryCanonicalize(CanonicalizerPhase.java:731)
at jdk.graal.compiler/jdk.graal.compiler.phases.common.CanonicalizerPhase.processNode(CanonicalizerPhase.java:611)
at jdk.graal.compiler/jdk.graal.compiler.phases.common.CanonicalizerPhase.processWorkSet(CanonicalizerPhase.java:384)
at jdk.graal.compiler/jdk.graal.compiler.phases.common.CanonicalizerPhase.run(CanonicalizerPhase.java:275)
at jdk.graal.compiler/jdk.graal.compiler.phases.common.CanonicalizerPhase.run(CanonicalizerPhase.java:102)
at jdk.graal.compiler/jdk.graal.compiler.phases.BasePhase.apply(BasePhase.java:468)
at jdk.graal.compiler/jdk.graal.compiler.phases.BasePhase.apply(BasePhase.java:334)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.code.CompileQueue.doInlineSingleCallsite(CompileQueue.java:992)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.code.CompileQueue$SingleCallsiteInlineTask.run(CompileQueue.java:371)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.executeCommand(CompletionExecutor.java:166)
...
------------------------------------------------------------------------------------------------------------------------
12.4s (9.0% of total time) in 755 GCs | Peak RSS: 7.04GB | CPU load: 3.41
========================================================================================================================
Failed generating 'camel-quarkus-integration-test-milo-3.39.0-SNAPSHOT-runner' after 2m 16s.
```

## What appears to be broken

The following is from reading `CompileQueue.java` at tag `mandrel-25.0.4.0-Final`; I have not identified which specific method exceeds the limit.

The trace line numbers match that tag exactly: `CompileQueue.java:992` is the `CanonicalizerPhase.create().apply(graph, providers)` call inside `doInlineSingleCallsite`, and `CompileQueue.java:371` is `doInlineSingleCallsite(debug, method)` in `SingleCallsiteInlineTask.run`.

`NodeList` caps at 65536 entries, and the graph being canonicalized after single callsite inlining has an `AbstractMergeNode` acquiring its 65537th forward end. The stage does have a size guard, but three properties look like they let a caller graph grow past any intended bound:

1. **The guard operands are stale within a round.** `makeSingleCallsiteInlineDecision` (line 1019) tests `caller.compilationInfo.sizeLastRound + callee.compilationInfo.sizeLastRound >= FALLBACK_SIZE` (line 1034), but `sizeLastRound` is only assigned at the end of `doInlineSingleCallsite` (line 996), after the whole `InlinePhase` has run. During a single decode the plugin can inline many callees into the same caller while both operands remain at their previous-round values, so growth accumulated within that round is never observed by the guard.

2. **`FALLBACK_SIZE` is not in node-count units.** It is `50000` (line 166) compared against `NodeCostUtil.computeGraphSize(graph)`, a cost estimate, so it was never a bound on `NodeList` size in the first place.

3. **`FALLBACK_SIZE` is a hardcoded constant** with no corresponding `@Option`, so it cannot be tuned downward.

Additionally, `doInlineSingleCallsite` ends in `catch (Throwable ex) { throw debug.handle(ex); }` (lines 998-1000), so the bailout propagates and terminates the image build.

## Workaround

Disabling the stage avoids the failure:

```
-Dquarkus.native.additional-build-args-append=-H:-AOTSingleCallsiteInline
```

## References

The stage reached `mandrel-25.0.4.0-Final` through graalvm/mandrel#985 ("Merge upstream graalvm-community-jdk25u/master into mandrel/25.0"), in these commits:

- [`11b6c9c`](https://github.com/graalvm/mandrel/commit/11b6c9c87a9a) (2026-01-23) — "single callsite inlining": adds `SubstrateOptions.AOTSingleCallsiteInline`, `InliningGraphDecoder`, and the `CompileQueue` stage
- [`86a715a`](https://github.com/graalvm/mandrel/commit/86a715a8c357) (2026-02-02) — "account for NeverInline annotation": adds the `callee.getWrapped().canBeInlined()` check
- [`db6ab61`](https://github.com/graalvm/mandrel/commit/db6ab616fdec) (2026-05-13) — "revert name change of UnpublishedTrivialMethods and unpublishedTrivialMethods"

The corresponding review thread is oracle/graal#12899 ("Add an inlining stage for single callsite methods"), which is still open on GitHub even though the commits above are in the release.

_Claude Code on behalf of James Netherton_

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the Milo native integration-test build with AOTSingleCallsiteInline enabled, then read CompileQueue.java around doInlineSingleCallsite and makeSingleCallsiteInlineDecision. Compare the graph-size checks with NodeList.java and the reported AbstractMergeNode failure; done means the build no longer terminates with PermanentBailoutException while the stage remains enabled.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.