firebase / firebase/firebase-tools
[firestore-emulator: datastore-mode] No concurrent transaction manages to write successfully
- Dominant language
- TypeScript
- Stars
- 4.5k
- Forks
- 1.3k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 84
Description
### [REQUIRED] Environment info
firebase-tools: Using the gcloud cli version 511.0.0, firestore emulator: 1.19.9
Platform: macOS (arm), python client library
### [REQUIRED] Test case
```python
import os
import time
import itertools
from google.cloud import datastore
import concurrent.futures
def get_new_client():
return datastore.Client(
project=os.environ.get("GCLOUDC_PROJECT_ID", "test"),
namespace=None,
_http=None,
)
def sdk_increment_integer_entity(integer_pk):
client = get_new_client()
key = client.key("Integer", integer_pk)
with client.transaction():
entity = client.get(key=key)
entity["value"] += 1
client.put(entity)
def run_test():
PORT = 10901
os.environ["DATASTORE_EMULATOR_HOST"] = "127.0.0.1:%s" % PORT
os.environ["DATASTORE_PROJECT_ID"] = "test"
initial_value = 0
concurrent_writes = 10
futures = []
local_client = get_new_client()
key = local_client.key("Integer", 1)
entity = datastore.Entity(key)
entity["value"] = initial_value
local_client.put(entity)
with concurrent.futures.ThreadPoolExecutor(max_workers=concurrent_writes) as executor:
for _ in range(concurrent_writes):
futures.append(executor.submit(sdk_increment_integer_entity, 1))
concurrent.futures.wait(futures)
futures_exceptions = [future.exception() for future in futures]
number_successful_writes = len(list(itertools.filterfalse(None, futures_exceptions)))
if number_successful_writes == 0:
print(futures_exceptions)
else:
print(f"All good, successful writes: {number_successful_writes}")
from_db = local_client.get(key)
assert number_successful_writes != 0
assert from_db["value"] == number_successful_writes
def main():
for _ in range(1, 50):
run_test()
time.sleep(2)
if __name__ == "__main__":
main()
```
### [REQUIRED] Steps to reproduce
1. Start the emulator with gcloud emulators firestore start --host-port=127.0.0.1:10901 --database-mode=datastore-mode
2. Install the python deps: pip install google-cloud-datastore==2.19.0
3. Run the above python script
### [REQUIRED] Expected behavior
Runs successfully, without errors, every time
### [REQUIRED] Actual behavior
Command fails with
```
[Aborted('Transaction lock timeout.'), Aborted('Transaction lock timeout.'), Aborted('Transaction lock timeout.'), Aborted('Transaction lock timeout.'), Aborted('Transaction lock timeout.'), Aborted('Transaction lock timeout.'), Aborted('Transaction lock timeout.'), Aborted('Transaction lock timeout.'), Aborted('Transaction lock timeout.'), Aborted('Transaction lock timeout.')]
Traceback (most recent call last):
File "/Users/artoale/dev/gcloudc/./example.py", line 69, in
main()
File "/Users/artoale/dev/gcloudc/./example.py", line 64, in main
run_test()
File "/Users/artoale/dev/gcloudc/./example.py", line 58, in run_test
assert number_successful_writes != 0
```
I've tested the same script against two real firestore in datastore mode instances (one configured with pessimistic concurrency, one with optimistic) - running the script multiple time.
Transaction failing because of lock timeout make complete sense, but I would expect at least one transaction (typically the first to acquire the lock with a pessimistic model) to pass - but the test shows that they do not
Contributor guide
Assessment
This issue has not been assessed yet.