aws / aws/aws-sdk-java-v2

Bug in addAttributeToProject in enhanced client for list attributes

Open
#7,102 0 comments 0 reactions 0 assignees View on GitHub
bug needs-triage
Dominant language
Java
Stars
2.6k
Forks
1k
Avg merge
2d 9h
Merged PRs (30d)
51

Description

### Describe the bug
`addAttributeToProject` generates invalid ExpressionAttributeName key when projecting list elements

## Summary

`ScanEnhancedRequest.builder().addAttributeToProject("tags[0]")` generates an ExpressionAttributeNames entry with brackets in the key (`#AMZN_MAPPED_tags[0]`), which DynamoDB rejects with a validation error. The `[0]` list dereference belongs in the projection expression itself, not inside the `#placeholder` key.

## Affected Component

- **Package:** `AwsJavaSdk-DynamoDb-Enhanced` (2.0)
- **Class:** `software.amazon.awssdk.enhanced.dynamodb.internal.ProjectionExpression`
- **Source:** [ProjectionExpression.java](https://code.amazon.com/packages/AwsJavaSdk-DynamoDb-Enhanced/blobs/aws-sdk-java-v2/release/--/src/software/amazon/awssdk/enhanced/dynamodb/internal/ProjectionExpression.java)

## Reproduction

```java
DynamoDbTable table = enhancedClient.table("MyTable", TableSchema.fromBean(Item.class));

// Documented as valid per NestedAttributeName javadoc:
// "List item 0 of ListAttribute can be created as NestedAttributeName.create("ListAttribute[0]")"
ScanEnhancedRequest request = ScanEnhancedRequest.builder()
.addAttributeToProject("tags[0]")
.build();

table.scan(request).iterator().next(); // THROWS
```

Note the docs here imply that this should work: https://docs.aws.amazon.com/java/api/latest/software/amazon/awssdk/enhanced/dynamodb/NestedAttributeName.html (the addAttributeToProject delegates down - just sugar and turns the string into a NestedAttributeName)

Quote from docs:
`List item 0 of ListAttribute can be created as NestedAttributeName.create("ListAttribute[0]")`

**Error:**
```
software.amazon.awssdk.services.dynamodb.model.DynamoDbException:
ExpressionAttributeNames contains invalid key: Syntax error; key: "#AMZN_MAPPED_tags[0]"
(Service: DynamoDb, Status Code: 400)
```

The request payload that us generated looks like this and is invalid
```
{"TableName":"BugRepro-Projection","ProjectionExpression":"#AMZN_MAPPED_tags[0]","ExpressionAttributeNames":{"#AMZN_MAPPED_tags[0]":"tags[0]"}}

```

## Expected Behavior

The SDK should generate:
```
ExpressionAttributeNames: { "#AMZN_MAPPED_tags": "tags" }
ProjectionExpression: "#AMZN_MAPPED_tags[0]"
```

i.e., the placeholder `#AMZN_MAPPED_tags` maps to the attribute name `tags`, and `[0]` is appended to the placeholder *in the expression string* to dereference the list element.

## Root Cause

In `ProjectionExpression.java`:
1. `NestedAttributeName.create("tags[0]")` produces a single element `"tags[0]"`
2. `createAttributePlaceholders` calls `PROJECTION_EXPRESSION_KEY_MAPPER` which does `AMZN_MAPPED + cleanAttributeName(k)` — but `cleanAttributeName` doesn't strip the `[0]` suffix
3. This produces the key `#AMZN_MAPPED_tags[0]`
4. `convertToNameExpression` then uses this key directly in the expression string

The fix should:
- Strip `[N]` suffixes from the attribute name when generating the placeholder key
- Keep `[N]` in the expression output appended after the placeholder

## Workaround

Use the low-level `DynamoDbClient.scan()` with a raw `projectionExpression` string:

```java
client.scan(ScanRequest.builder()
.tableName("MyTable")
.projectionExpression("tags[0]")
.build());
```

This works because DynamoDB natively supports `tags[0]` in projection expressions — the issue is solely in how the Enhanced Client maps it to ExpressionAttributeNames.

### Regression Issue

- [ ] Select this option if this issue appears to be a regression.

### Expected Behavior

See above.

### Current Behavior

See above.

### Reproduction Steps

Note, this repro will leave behind a table in your account called BugRepro-Projection which you should delete when you're done.

```
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import software.amazon.awssdk.enhanced.dynamodb.DynamoDbEnhancedClient;
import software.amazon.awssdk.enhanced.dynamodb.DynamoDbTable;
import software.amazon.awssdk.enhanced.dynamodb.NestedAttributeName;
import software.amazon.awssdk.enhanced.dynamodb.TableSchema;
import software.amazon.awssdk.enhanced.dynamodb.mapper.annotations.DynamoDbBean;
import software.amazon.awssdk.enhanced.dynamodb.mapper.annotations.DynamoDbPartitionKey;
import software.amazon.awssdk.enhanced.dynamodb.model.ScanEnhancedRequest;
import software.amazon.awssdk.enhanced.dynamodb.model.Page;
import software.amazon.awssdk.services.dynamodb.DynamoDbClient;
import software.amazon.awssdk.services.dynamodb.model.ProvisionedThroughput;
import software.amazon.awssdk.services.dynamodb.model.AttributeValue;
import software.amazon.awssdk.services.dynamodb.model.PutItemRequest;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.dynamodb.model.ResourceInUseException;

import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;

/**
* Minimal reproduction of addAttributeToProject bug with list index projection.
*
* The Enhanced Client's addAttributeToProject("myList[0]") generates an invalid
* ExpressionAttributeName key "#AMZN_MAPPED_myList[0]" which DynamoDB rejects.
*
* Run: java -cp org.junit.runner.JUnitCore amazon.rds.mrr.AddAttributeToProjectBugRepro
* Requires: AWS credentials with DynamoDB access in us-east-1
*/
public class PotatoTest {

private static final String TABLE_NAME = "BugRepro-Projection";

private DynamoDbClient client;
private DynamoDbEnhancedClient enhancedClient;
private DynamoDbTable table;

@DynamoDbBean
public static class Item {
private String id;
private List tags;

@DynamoDbPartitionKey
public String getId() { return id; }
public void setId(String id) { this.id = id; }
public List getTags() { return tags; }
public void setTags(List tags) { this.tags = tags; }
}

@Before
public void setUp() {
client = DynamoDbClient.builder().region(Region.US_EAST_1).build();
enhancedClient = DynamoDbEnhancedClient.builder().dynamoDbClient(client).build();
table = enhancedClient.table(TABLE_NAME, TableSchema.fromBean(Item.class));
try {
table.createTable(b -> b.provisionedThroughput(
ProvisionedThroughput.builder().readCapacityUnits(5L).writeCapacityUnits(5L).build()));
client.waiter().waitUntilTableExists(b -> b.tableName(TABLE_NAME));
} catch (ResourceInUseException e) {
System.out.println("Caught ResourceInUseException");
}

// Put an item with a list attribute using low-level client
Map item = new HashMap<>();
item.put("id", AttributeValue.builder().s("item-1").build());
item.put("tags", AttributeValue.builder().l(
AttributeValue.builder().s("alpha").build(),
AttributeValue.builder().s("beta").build(),
AttributeValue.builder().s("gamma").build()
).build());
client.putItem(PutItemRequest.builder().tableName(TABLE_NAME).item(item).build());
}

@After
public void tearDown() {
}

@Test
public void addAttributeToProject_withListIndex_failsAgainstRealDynamoDB() {
// This is documented as valid usage per NestedAttributeName javadoc:
// "List item 0 of ListAttribute can be created as NestedAttributeName.create("ListAttribute[0]")"
// But it generates an invalid ExpressionAttributeName key with brackets.
ScanEnhancedRequest request = ScanEnhancedRequest.builder()
//.addAttributeToProject("tags[0]")
.addNestedAttributesToProject(NestedAttributeName.create("tags[0]"))
.build();

// Throws: DynamoDbException: ExpressionAttributeNames contains invalid key:
// Syntax error; key: "#AMZN_MAPPED_tags[0]"
Iterator> pages = table.scan(request).iterator();
while (pages.hasNext()) {
pages.next();
}
}

@Test
public void rawProjectionExpression_withListIndex_worksAgainstRealDynamoDB() {
// Workaround: use the low-level client with a raw projection expression
var response = client.scan(software.amazon.awssdk.services.dynamodb.model.ScanRequest.builder()
.tableName(TABLE_NAME)
.projectionExpression("tags[0]")
.build());

// This succeeds - DynamoDB supports tags[0] in projection expressions natively
assert response.items().size() == 1;
assert response.items().get(0).get("tags").l().get(0).s().equals("alpha");
}

@Test
public void projectionExpressionWithExpressionAttributeNames_withListIndex_worksAgainstRealDynamoDB() {
// Workaround: use the low-level client with expressionAttributeNames
// The key insight: the placeholder key must NOT contain brackets,
// but the projection expression string can append [0] to the placeholder.
var response = client.scan(software.amazon.awssdk.services.dynamodb.model.ScanRequest.builder()
.tableName(TABLE_NAME)
.projectionExpression("#t[0]")
.expressionAttributeNames(Map.of("#t", "tags"))
.build());

// This succeeds - DynamoDB allows [0] appended to a placeholder in the expression
assert response.items().size() == 1;
assert response.items().get(0).get("tags").l().get(0).s().equals("alpha");
}

}
```

### Possible Solution

_No response_

### Additional Information/Context

_No response_

### AWS Java SDK version used

latest

### JDK version used

17

### Operating System and version

osx and alinux

Contributor guide

Open the contributing guide

Research direction

Start with software.amazon.awssdk.enhanced.dynamodb.internal.ProjectionExpression.java and trace how addAttributeToProject delegates through NestedAttributeName.create("tags[0]"). Use the provided reproduction to verify that the placeholder excludes [0] while the projection expression retains it, and confirm the request succeeds against DynamoDB.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.