Azure / Azure/azure-sdk-for-java
[BUG] Azure AI Projects maxCompletionTokens
- Dominant language
- Java
- Stars
- 2.6k
- Forks
- 2.2k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 178
Description
**Describe the bug**
There seems to be some unexpected timeout when using `maxCompletionTokens`. I have two version of a project one in python one for java. In python making a simple chat request with 800 max completion tokens works, but in java it fails in various way depending on how big of a value I give.
In Java at 800 max completion tokens I get a `finishReason` of `length`. With no text response.
In Java at 2000 max completion tokens I've gotten everything from the text response to an exception being thrown that the connection was terminated.
***Exception or Stack Trace***
Add the exception log and stack trace if available
**To Reproduce**
Steps to reproduce the behavior:
See code snippet.
***Code Snippet***
```java
public static void main(String[] args) {
String endpoint = System.getenv("AZURE_AI_FOUNDRY_ENDPOINT");
String model = "gpt-5";
OpenAIClient openAIClient = new AIProjectClientBuilder()
.endpoint(endpoint)
.credential(new DefaultAzureCredentialBuilder().build())
.buildOpenAIClient();
List messages = new ArrayList<>();
messages.add(ChatCompletionMessageParam.ofSystem(
ChatCompletionSystemMessageParam.builder()
.content("You are a helpful assistant.")
.build()));
messages.add(ChatCompletionMessageParam.ofUser(
ChatCompletionUserMessageParam.builder()
.content("What is Azure App Configuration?")
.build()));
ChatCompletionCreateParams params = ChatCompletionCreateParams.builder()
.model(model)
.messages(messages)
.maxCompletionTokens(800)
.build();
ChatCompletion response = openAIClient.chat().completions().create(params);
String aiResponse = response.choices().get(0).message().content().orElse("");
System.out.println("AI: " + aiResponse);
}
```
**Expected behavior**
This should work, it does in other languages.
**Setup (please complete the following information):**
- OS: Windows
- IDE: VS Code
- Library/Libraries: azure-ai-projects 2.0.0-beta.2
- Java version: 17
- App Server/Environment: N/A
- Frameworks: N/A
If you suspect a dependency version mismatch (e.g. you see `NoClassDefFoundError`, `NoSuchMethodError` or similar), please check out [Troubleshoot dependency version conflict article](https://aka.ms/azsdk/java/dependency/troubleshoot) first. If it doesn't provide solution for the problem, please provide:
- verbose dependency tree (`mvn dependency:tree -Dverbose`)
- exception message, full stack trace, and any available logs
**Additional context**
Add any other context about the problem here.
**Information Checklist**
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report
- [ ] Bug Description Added
- [ ] Repro Steps Added
- [ ] Setup information Added
Contributor guide
Assessment
This issue has not been assessed yet.