Bug in addAttributeToProject in enhanced client for list attributes
- Dominant language
- Java
- Stars
- 2.6k
- Forks
- 1k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 51
Description
### Describe the bug
`addAttributeToProject` generates invalid ExpressionAttributeName key when projecting list elements
## Summary
`ScanEnhancedRequest.builder().addAttributeToProject("tags[0]")` generates an ExpressionAttributeNames entry with brackets in the key (`#AMZN_MAPPED_tags[0]`), which DynamoDB rejects with a validation error. The `[0]` list dereference belongs in the projection expression itself, not inside the `#placeholder` key.
## Affected Component
- **Package:** `AwsJavaSdk-DynamoDb-Enhanced` (2.0)
- **Class:** `software.amazon.awssdk.enhanced.dynamodb.internal.ProjectionExpression`
- **Source:** [ProjectionExpression.java](https://code.amazon.com/packages/AwsJavaSdk-DynamoDb-Enhanced/blobs/aws-sdk-java-v2/release/--/src/software/amazon/awssdk/enhanced/dynamodb/internal/ProjectionExpression.java)
## Reproduction
```java
DynamoDbTable table = enhancedClient.table("MyTable", TableSchema.fromBean(Item.class));
// Documented as valid per NestedAttributeName javadoc:
// "List item 0 of ListAttribute can be created as NestedAttributeName.create("ListAttribute[0]")"
ScanEnhancedRequest request = ScanEnhancedRequest.builder()
.addAttributeToProject("tags[0]")
.build();
table.scan(request).iterator().next(); // THROWS
```
Note the docs here imply that this should work: https://docs.aws.amazon.com/java/api/latest/software/amazon/awssdk/enhanced/dynamodb/NestedAttributeName.html (the addAttributeToProject delegates down - just sugar and turns the string into a NestedAttributeName)
Quote from docs:
`List item 0 of ListAttribute can be created as NestedAttributeName.create("ListAttribute[0]")`
**Error:**
```
software.amazon.awssdk.services.dynamodb.model.DynamoDbException:
ExpressionAttributeNames contains invalid key: Syntax error; key: "#AMZN_MAPPED_tags[0]"
(Service: DynamoDb, Status Code: 400)
```
The request payload that us generated looks like this and is invalid
```
{"TableName":"BugRepro-Projection","ProjectionExpression":"#AMZN_MAPPED_tags[0]","ExpressionAttributeNames":{"#AMZN_MAPPED_tags[0]":"tags[0]"}}
```
## Expected Behavior
The SDK should generate:
```
ExpressionAttributeNames: { "#AMZN_MAPPED_tags": "tags" }
ProjectionExpression: "#AMZN_MAPPED_tags[0]"
```
i.e., the placeholder `#AMZN_MAPPED_tags` maps to the attribute name `tags`, and `[0]` is appended to the placeholder *in the expression string* to dereference the list element.
## Root Cause
In `ProjectionExpression.java`:
1. `NestedAttributeName.create("tags[0]")` produces a single element `"tags[0]"`
2. `createAttributePlaceholders` calls `PROJECTION_EXPRESSION_KEY_MAPPER` which does `AMZN_MAPPED + cleanAttributeName(k)` — but `cleanAttributeName` doesn't strip the `[0]` suffix
3. This produces the key `#AMZN_MAPPED_tags[0]`
4. `convertToNameExpression` then uses this key directly in the expression string
The fix should:
- Strip `[N]` suffixes from the attribute name when generating the placeholder key
- Keep `[N]` in the expression output appended after the placeholder
## Workaround
Use the low-level `DynamoDbClient.scan()` with a raw `projectionExpression` string:
```java
client.scan(ScanRequest.builder()
.tableName("MyTable")
.projectionExpression("tags[0]")
.build());
```
This works because DynamoDB natively supports `tags[0]` in projection expressions — the issue is solely in how the Enhanced Client maps it to ExpressionAttributeNames.
### Regression Issue
- [ ] Select this option if this issue appears to be a regression.
### Expected Behavior
See above.
### Current Behavior
See above.
### Reproduction Steps
Note, this repro will leave behind a table in your account called BugRepro-Projection which you should delete when you're done.
```
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import software.amazon.awssdk.enhanced.dynamodb.DynamoDbEnhancedClient;
import software.amazon.awssdk.enhanced.dynamodb.DynamoDbTable;
import software.amazon.awssdk.enhanced.dynamodb.NestedAttributeName;
import software.amazon.awssdk.enhanced.dynamodb.TableSchema;
import software.amazon.awssdk.enhanced.dynamodb.mapper.annotations.DynamoDbBean;
import software.amazon.awssdk.enhanced.dynamodb.mapper.annotations.DynamoDbPartitionKey;
import software.amazon.awssdk.enhanced.dynamodb.model.ScanEnhancedRequest;
import software.amazon.awssdk.enhanced.dynamodb.model.Page;
import software.amazon.awssdk.services.dynamodb.DynamoDbClient;
import software.amazon.awssdk.services.dynamodb.model.ProvisionedThroughput;
import software.amazon.awssdk.services.dynamodb.model.AttributeValue;
import software.amazon.awssdk.services.dynamodb.model.PutItemRequest;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.dynamodb.model.ResourceInUseException;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
/**
* Minimal reproduction of addAttributeToProject bug with list index projection.
*
* The Enhanced Client's addAttributeToProject("myList[0]") generates an invalid
* ExpressionAttributeName key "#AMZN_MAPPED_myList[0]" which DynamoDB rejects.
*
* Run: java -cp org.junit.runner.JUnitCore amazon.rds.mrr.AddAttributeToProjectBugRepro
* Requires: AWS credentials with DynamoDB access in us-east-1
*/
public class PotatoTest {
private static final String TABLE_NAME = "BugRepro-Projection";
private DynamoDbClient client;
private DynamoDbEnhancedClient enhancedClient;
private DynamoDbTable table;
@DynamoDbBean
public static class Item {
private String id;
private List tags;
@DynamoDbPartitionKey
public String getId() { return id; }
public void setId(String id) { this.id = id; }
public List getTags() { return tags; }
public void setTags(List tags) { this.tags = tags; }
}
@Before
public void setUp() {
client = DynamoDbClient.builder().region(Region.US_EAST_1).build();
enhancedClient = DynamoDbEnhancedClient.builder().dynamoDbClient(client).build();
table = enhancedClient.table(TABLE_NAME, TableSchema.fromBean(Item.class));
try {
table.createTable(b -> b.provisionedThroughput(
ProvisionedThroughput.builder().readCapacityUnits(5L).writeCapacityUnits(5L).build()));
client.waiter().waitUntilTableExists(b -> b.tableName(TABLE_NAME));
} catch (ResourceInUseException e) {
System.out.println("Caught ResourceInUseException");
}
// Put an item with a list attribute using low-level client
Map item = new HashMap<>();
item.put("id", AttributeValue.builder().s("item-1").build());
item.put("tags", AttributeValue.builder().l(
AttributeValue.builder().s("alpha").build(),
AttributeValue.builder().s("beta").build(),
AttributeValue.builder().s("gamma").build()
).build());
client.putItem(PutItemRequest.builder().tableName(TABLE_NAME).item(item).build());
}
@After
public void tearDown() {
}
@Test
public void addAttributeToProject_withListIndex_failsAgainstRealDynamoDB() {
// This is documented as valid usage per NestedAttributeName javadoc:
// "List item 0 of ListAttribute can be created as NestedAttributeName.create("ListAttribute[0]")"
// But it generates an invalid ExpressionAttributeName key with brackets.
ScanEnhancedRequest request = ScanEnhancedRequest.builder()
//.addAttributeToProject("tags[0]")
.addNestedAttributesToProject(NestedAttributeName.create("tags[0]"))
.build();
// Throws: DynamoDbException: ExpressionAttributeNames contains invalid key:
// Syntax error; key: "#AMZN_MAPPED_tags[0]"
Iterator> pages = table.scan(request).iterator();
while (pages.hasNext()) {
pages.next();
}
}
@Test
public void rawProjectionExpression_withListIndex_worksAgainstRealDynamoDB() {
// Workaround: use the low-level client with a raw projection expression
var response = client.scan(software.amazon.awssdk.services.dynamodb.model.ScanRequest.builder()
.tableName(TABLE_NAME)
.projectionExpression("tags[0]")
.build());
// This succeeds - DynamoDB supports tags[0] in projection expressions natively
assert response.items().size() == 1;
assert response.items().get(0).get("tags").l().get(0).s().equals("alpha");
}
@Test
public void projectionExpressionWithExpressionAttributeNames_withListIndex_worksAgainstRealDynamoDB() {
// Workaround: use the low-level client with expressionAttributeNames
// The key insight: the placeholder key must NOT contain brackets,
// but the projection expression string can append [0] to the placeholder.
var response = client.scan(software.amazon.awssdk.services.dynamodb.model.ScanRequest.builder()
.tableName(TABLE_NAME)
.projectionExpression("#t[0]")
.expressionAttributeNames(Map.of("#t", "tags"))
.build());
// This succeeds - DynamoDB allows [0] appended to a placeholder in the expression
assert response.items().size() == 1;
assert response.items().get(0).get("tags").l().get(0).s().equals("alpha");
}
}
```
### Possible Solution
_No response_
### Additional Information/Context
_No response_
### AWS Java SDK version used
latest
### JDK version used
17
### Operating System and version
osx and alinux
Contributor guide
Research direction
Start with software.amazon.awssdk.enhanced.dynamodb.internal.ProjectionExpression.java and trace how addAttributeToProject delegates through NestedAttributeName.create("tags[0]"). Use the provided reproduction to verify that the placeholder excludes [0] while the projection expression retains it, and confirm the request succeeds against DynamoDB.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100