GoogleCloudPlatform / GoogleCloudPlatform/spring-cloud-gcp

Difference in behaviour between performReadWriteTransaction and @Transactional

Open
#695 6 comments 5 reactions 0 assignees View on GitHub
priority: p3 spanner type: enhancement
Dominant language
Java
Stars
551
Forks
349
Avg merge
1d 13h
Merged PRs (30d)
14

Description

I am debugging a Production issue we see with Spanner and the use of `@Transactional`. Our app may throw an exception during a transaction and not retry. This can occur at anytime, but we especially noticed it when our app instances run longer than 28 days. I assume that relates to [Spanner deleting sessions](https://cloud.google.com/spanner/docs/sessions) that are more than 28 days old.

Where our app uses `@Transactional` it does not recover from these deleted sessions. Whereas performReadWriteTransaction does. Is the difference expected behaviour?

I created a fresh Spring Boot app to test and demonstrate the issue. It has these dependencies:

```
implementation 'org.springframework.boot:spring-boot-starter:2.5.6'
implementation 'com.google.cloud:spring-cloud-gcp-starter-data-spanner:2.0.5'
```

A simple entity and repository class:

```
@Table(name = "Users")
public class UserEntity {
@PrimaryKey
@Column(name = "UserId")
private String userId;
@Column(name = "CreationDate")
private Timestamp creationDate;

public UserEntity(String userId, Timestamp creationDate) {
this.userId = userId;
this.creationDate = creationDate;
}
}

@Repository
public interface UserRepository extends SpannerRepository {
}
```

A service with two methods for saving a records. One using `@Transactional` the other using the repository's `performReadWriteTransaction` method.

```
@Service
public class UserService {
private final UserRepository userRepository;

public UserService(UserRepository userRepository) {
this.userRepository = userRepository;
}

@Transactional
public void saveWithTransactional(String userId) {
final var entity = new UserEntity(userId, Timestamp.now());

try {
System.out.println("DELETE CURRENT SPANNER SESSIONS NOW!");
Thread.sleep(7000);
} catch (InterruptedException e) {
e.printStackTrace();
}
userRepository.save(entity);
}

public void saveWithRepositoryTransaction(String userId) {
final var entity = new UserEntity(userId, Timestamp.now());

userRepository.performReadWriteTransaction( tx -> {
try {
System.out.println("DELETE CURRENT SPANNER SESSIONS NOW!");
Thread.sleep(7000);
} catch (InterruptedException e) {
e.printStackTrace();
}
var user = tx.save(entity);
return user;
});
}
}
```

Finally a CommandLineRunner to run:

```
@Bean
public CommandLineRunner commandLineRunner(UserService userService) {
return args -> {
final var userId = UUID.randomUUID().toString();

// userService.saveWithRepositoryTransaction(userId);
userService.saveWithTransactional(userId);
};
}
```

I also created a PowerShell script to manually delete the sessions while the app sleeps:

```
foreach ($session in gcloud spanner databases sessions list --instance=my-instance --database=my-db --format="value(name)")
{
Write-Host "Deleting session: $session"
gcloud spanner databases sessions delete $session --instance=my-instance --database=my-db
}
Write-Host "All deleted!"
```

To make session deletion quick I set sessions to 2. To help debugging set logging to TRACE:

```
spring.cloud.gcp.spanner.minSessions=2
logging.level.com.google.cloud.spanner=TRACE
```

**RESULTS**

With `performReadWriteTransaction`:

* The save tries to `BeginTransaction` with one of the two avaialble sessions.
* Spanner responds with "Session does not exist."
* Client then calls `BatchCreateSessions` to create 1 new session.
* Then retries the save with another session from the pool. Often this is the other original session, which is also deleted.
* This again fails, but the next retry picks the new live session and completes.

With `@Transactional`:
* The save tries to `BeginTransaction` with one of the two avaialble sessions.
* Spanner responds with "Session does not exist."
* Client calls `BatchCreateSessions` to create 1 new session.
* AbortException is thrown "ABORTED: NOT_FOUND"

Is the above differences in behaviour expected?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.