aws / aws/aws-sdk-java-v2

Bug in addAttributeToProject in enhanced client for list attributes

Offen
#7,102 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
bug needs-triage
Vorherrschende Sprache
Java
Sterne
2.6k
Forks
1k
Ø Merge
2 T. 9 Std.
Gemergte PRs (30 T.)
51

Beschreibung

### Describe the bug
`addAttributeToProject` generates invalid ExpressionAttributeName key when projecting list elements

## Summary

`ScanEnhancedRequest.builder().addAttributeToProject("tags[0]")` generates an ExpressionAttributeNames entry with brackets in the key (`#AMZN_MAPPED_tags[0]`), which DynamoDB rejects with a validation error. The `[0]` list dereference belongs in the projection expression itself, not inside the `#placeholder` key.

## Affected Component

- **Package:** `AwsJavaSdk-DynamoDb-Enhanced` (2.0)
- **Class:** `software.amazon.awssdk.enhanced.dynamodb.internal.ProjectionExpression`
- **Source:** [ProjectionExpression.java](https://code.amazon.com/packages/AwsJavaSdk-DynamoDb-Enhanced/blobs/aws-sdk-java-v2/release/--/src/software/amazon/awssdk/enhanced/dynamodb/internal/ProjectionExpression.java)

## Reproduction

```java
DynamoDbTable table = enhancedClient.table("MyTable", TableSchema.fromBean(Item.class));

// Documented as valid per NestedAttributeName javadoc:
// "List item 0 of ListAttribute can be created as NestedAttributeName.create("ListAttribute[0]")"
ScanEnhancedRequest request = ScanEnhancedRequest.builder()
.addAttributeToProject("tags[0]")
.build();

table.scan(request).iterator().next(); // THROWS
```

Note the docs here imply that this should work: https://docs.aws.amazon.com/java/api/latest/software/amazon/awssdk/enhanced/dynamodb/NestedAttributeName.html (the addAttributeToProject delegates down - just sugar and turns the string into a NestedAttributeName)

Quote from docs:
`List item 0 of ListAttribute can be created as NestedAttributeName.create("ListAttribute[0]")`

**Error:**
```
software.amazon.awssdk.services.dynamodb.model.DynamoDbException:
ExpressionAttributeNames contains invalid key: Syntax error; key: "#AMZN_MAPPED_tags[0]"
(Service: DynamoDb, Status Code: 400)
```

The request payload that us generated looks like this and is invalid
```
{"TableName":"BugRepro-Projection","ProjectionExpression":"#AMZN_MAPPED_tags[0]","ExpressionAttributeNames":{"#AMZN_MAPPED_tags[0]":"tags[0]"}}

```

## Expected Behavior

The SDK should generate:
```
ExpressionAttributeNames: { "#AMZN_MAPPED_tags": "tags" }
ProjectionExpression: "#AMZN_MAPPED_tags[0]"
```

i.e., the placeholder `#AMZN_MAPPED_tags` maps to the attribute name `tags`, and `[0]` is appended to the placeholder *in the expression string* to dereference the list element.

## Root Cause

In `ProjectionExpression.java`:
1. `NestedAttributeName.create("tags[0]")` produces a single element `"tags[0]"`
2. `createAttributePlaceholders` calls `PROJECTION_EXPRESSION_KEY_MAPPER` which does `AMZN_MAPPED + cleanAttributeName(k)` — but `cleanAttributeName` doesn't strip the `[0]` suffix
3. This produces the key `#AMZN_MAPPED_tags[0]`
4. `convertToNameExpression` then uses this key directly in the expression string

The fix should:
- Strip `[N]` suffixes from the attribute name when generating the placeholder key
- Keep `[N]` in the expression output appended after the placeholder

## Workaround

Use the low-level `DynamoDbClient.scan()` with a raw `projectionExpression` string:

```java
client.scan(ScanRequest.builder()
.tableName("MyTable")
.projectionExpression("tags[0]")
.build());
```

This works because DynamoDB natively supports `tags[0]` in projection expressions — the issue is solely in how the Enhanced Client maps it to ExpressionAttributeNames.

### Regression Issue

- [ ] Select this option if this issue appears to be a regression.

### Expected Behavior

See above.

### Current Behavior

See above.

### Reproduction Steps

Note, this repro will leave behind a table in your account called BugRepro-Projection which you should delete when you're done.

```
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import software.amazon.awssdk.enhanced.dynamodb.DynamoDbEnhancedClient;
import software.amazon.awssdk.enhanced.dynamodb.DynamoDbTable;
import software.amazon.awssdk.enhanced.dynamodb.NestedAttributeName;
import software.amazon.awssdk.enhanced.dynamodb.TableSchema;
import software.amazon.awssdk.enhanced.dynamodb.mapper.annotations.DynamoDbBean;
import software.amazon.awssdk.enhanced.dynamodb.mapper.annotations.DynamoDbPartitionKey;
import software.amazon.awssdk.enhanced.dynamodb.model.ScanEnhancedRequest;
import software.amazon.awssdk.enhanced.dynamodb.model.Page;
import software.amazon.awssdk.services.dynamodb.DynamoDbClient;
import software.amazon.awssdk.services.dynamodb.model.ProvisionedThroughput;
import software.amazon.awssdk.services.dynamodb.model.AttributeValue;
import software.amazon.awssdk.services.dynamodb.model.PutItemRequest;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.dynamodb.model.ResourceInUseException;

import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;

/**
* Minimal reproduction of addAttributeToProject bug with list index projection.
*
* The Enhanced Client's addAttributeToProject("myList[0]") generates an invalid
* ExpressionAttributeName key "#AMZN_MAPPED_myList[0]" which DynamoDB rejects.
*
* Run: java -cp org.junit.runner.JUnitCore amazon.rds.mrr.AddAttributeToProjectBugRepro
* Requires: AWS credentials with DynamoDB access in us-east-1
*/
public class PotatoTest {

private static final String TABLE_NAME = "BugRepro-Projection";

private DynamoDbClient client;
private DynamoDbEnhancedClient enhancedClient;
private DynamoDbTable table;

@DynamoDbBean
public static class Item {
private String id;
private List tags;

@DynamoDbPartitionKey
public String getId() { return id; }
public void setId(String id) { this.id = id; }
public List getTags() { return tags; }
public void setTags(List tags) { this.tags = tags; }
}

@Before
public void setUp() {
client = DynamoDbClient.builder().region(Region.US_EAST_1).build();
enhancedClient = DynamoDbEnhancedClient.builder().dynamoDbClient(client).build();
table = enhancedClient.table(TABLE_NAME, TableSchema.fromBean(Item.class));
try {
table.createTable(b -> b.provisionedThroughput(
ProvisionedThroughput.builder().readCapacityUnits(5L).writeCapacityUnits(5L).build()));
client.waiter().waitUntilTableExists(b -> b.tableName(TABLE_NAME));
} catch (ResourceInUseException e) {
System.out.println("Caught ResourceInUseException");
}

// Put an item with a list attribute using low-level client
Map item = new HashMap<>();
item.put("id", AttributeValue.builder().s("item-1").build());
item.put("tags", AttributeValue.builder().l(
AttributeValue.builder().s("alpha").build(),
AttributeValue.builder().s("beta").build(),
AttributeValue.builder().s("gamma").build()
).build());
client.putItem(PutItemRequest.builder().tableName(TABLE_NAME).item(item).build());
}

@After
public void tearDown() {
}

@Test
public void addAttributeToProject_withListIndex_failsAgainstRealDynamoDB() {
// This is documented as valid usage per NestedAttributeName javadoc:
// "List item 0 of ListAttribute can be created as NestedAttributeName.create("ListAttribute[0]")"
// But it generates an invalid ExpressionAttributeName key with brackets.
ScanEnhancedRequest request = ScanEnhancedRequest.builder()
//.addAttributeToProject("tags[0]")
.addNestedAttributesToProject(NestedAttributeName.create("tags[0]"))
.build();

// Throws: DynamoDbException: ExpressionAttributeNames contains invalid key:
// Syntax error; key: "#AMZN_MAPPED_tags[0]"
Iterator> pages = table.scan(request).iterator();
while (pages.hasNext()) {
pages.next();
}
}

@Test
public void rawProjectionExpression_withListIndex_worksAgainstRealDynamoDB() {
// Workaround: use the low-level client with a raw projection expression
var response = client.scan(software.amazon.awssdk.services.dynamodb.model.ScanRequest.builder()
.tableName(TABLE_NAME)
.projectionExpression("tags[0]")
.build());

// This succeeds - DynamoDB supports tags[0] in projection expressions natively
assert response.items().size() == 1;
assert response.items().get(0).get("tags").l().get(0).s().equals("alpha");
}

@Test
public void projectionExpressionWithExpressionAttributeNames_withListIndex_worksAgainstRealDynamoDB() {
// Workaround: use the low-level client with expressionAttributeNames
// The key insight: the placeholder key must NOT contain brackets,
// but the projection expression string can append [0] to the placeholder.
var response = client.scan(software.amazon.awssdk.services.dynamodb.model.ScanRequest.builder()
.tableName(TABLE_NAME)
.projectionExpression("#t[0]")
.expressionAttributeNames(Map.of("#t", "tags"))
.build());

// This succeeds - DynamoDB allows [0] appended to a placeholder in the expression
assert response.items().size() == 1;
assert response.items().get(0).get("tags").l().get(0).s().equals("alpha");
}

}
```

### Possible Solution

_No response_

### Additional Information/Context

_No response_

### AWS Java SDK version used

latest

### JDK version used

17

### Operating System and version

osx and alinux

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Beginnen Sie mit software.amazon.awssdk.enhanced.dynamodb.internal.ProjectionExpression.java und verfolgen Sie, wie addAttributeToProject über NestedAttributeName.create("tags[0]") delegiert. Verwenden Sie die bereitgestellte Reproduktion, um zu überprüfen, dass der Platzhalter [0] ausschließt, während der Projection-Ausdruck ihn beibehält, und bestätigen Sie, dass die Anfrage erfolgreich an DynamoDB gesendet wird.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
java
Bereich
databases
Issue-Typ
Bug
Schwierigkeit
3/5
Geschätzter Aufwand
1-2 Tage
Aktivitätsstatus
Ruhig
Klarheit
Klar beschrieben
Anfängerfreundlichkeit
72/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.