Concurrent transaction lock Issue when no yt_wrapper directory exists
- Dominant language
- C++
- Stars
- 2.2k
- Forks
- 220
- PR merge metrics
- No merged PRs in 30d
Description
When there is no `yt_wrapper` directory exists, the following code results in an error:
```python
with client.Transaction():
client.run_map(...)
```
The error message I receive is:
```
Cannot take lock for child "yt_wrapper" of node //tmp since this child is locked by concurrent transaction 1c-4963-60001-4109
```
Backtrace
```bash
File "/usr/local/lib/python3.7/dist-packages/yt/wrapper/client_impl.py", line 2008, in run_map
job_count=job_count, memory_limit=memory_limit, spec=spec, ordered=ordered, stderr_table=stderr_table)
File "/usr/local/lib/python3.7/dist-packages/yt/wrapper/run_operation_commands.py", line 291, in run_map
return run_operation(spec_builder, sync=sync, enable_optimizations=True, client=client)
File "", line 2, in run_operation
File "/usr/local/lib/python3.7/dist-packages/yt/wrapper/common.py", line 422, in forbidden_inside_job
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/yt/wrapper/run_operation_commands.py", line 604, in run_operation
spec = spec_builder.build(client=client)
File "/usr/local/lib/python3.7/dist-packages/yt/wrapper/spec_builders.py", line 1214, in build
spec = self._do_build(client)
File "/usr/local/lib/python3.7/dist-packages/yt/wrapper/spec_builders.py", line 1538, in _do_build
client=client,
File "/usr/local/lib/python3.7/dist-packages/yt/wrapper/spec_builders.py", line 1066, in _build_simple_user_job_spec
client=client,
File "/usr/local/lib/python3.7/dist-packages/yt/wrapper/spec_builders.py", line 1093, in _build_user_job_spec
client=client,
File "/usr/local/lib/python3.7/dist-packages/yt/wrapper/spec_builders.py", line 815, in build
input_format, output_format, input_tables, output_tables, client)
File "/usr/local/lib/python3.7/dist-packages/yt/wrapper/spec_builders.py", line 604, in _prepare_job_files
local_files = file_manager.upload_files()
File "/usr/local/lib/python3.7/dist-packages/yt/wrapper/table_helpers.py", line 254, in upload_files
**file_params)
File "/usr/local/lib/python3.7/dist-packages/yt/wrapper/file_commands.py", line 480, in upload_file_to_cache
client=client)
File "/usr/local/lib/python3.7/dist-packages/yt/wrapper/cypress_commands.py", line 423, in create
result = make_formatted_request("create", params, format=None, client=client)
File "/usr/local/lib/python3.7/dist-packages/yt/wrapper/driver.py", line 182, in make_formatted_request
**kwargs)
File "/usr/local/lib/python3.7/dist-packages/yt/wrapper/driver.py", line 126, in make_request
client=client)
File "", line 2, in make_request
File "/usr/local/lib/python3.7/dist-packages/yt/wrapper/common.py", line 422, in forbidden_inside_job
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/yt/wrapper/http_driver.py", line 300, in make_request
client=client)
File "/usr/local/lib/python3.7/dist-packages/yt/wrapper/http_helpers.py", line 469, in make_request_with_retries
return RequestRetrier(method=method, url=url, **kwargs).run()
File "/usr/local/lib/python3.7/dist-packages/yt/wrapper/retries.py", line 79, in run
return self.action()
File "/usr/local/lib/python3.7/dist-packages/yt/wrapper/http_helpers.py", line 424, in action
_raise_for_status(response, request_info)
File "/usr/local/lib/python3.7/dist-packages/yt/wrapper/http_helpers.py", line 304, in _raise_for_status
raise error_exc
yt.wrapper.errors.YtCypressTransactionLockConflict: Cannot take lock for child "yt_wrapper" of node //tmp since this child is locked by concurrent transaction 1c-4963-60001-4109
***** Details:
Received HTTP response with error
origin ytsaurus-6b68f78676-7nj8f on 2023-10-24T23:49:24.633843Z
request_headers {
"User-Agent": "Python wrapper 0.13.7",
"Accept-Encoding": "gzip, identity",
"X-Started-By": "{\"pid\"=1;\"user\"=\"root\";}",
"X-YT-Header-Format": "yson",
"Content-Type": "application/x-yt-yson-text",
"X-YT-Correlation-Id": "7aaecbb4-eb9af461-c0a93ebd-b0d2cd2e"
}
response_headers {
"server": "istio-envoy",
"date": "Tue, 24 Oct 2023 23:49:24 GMT",
"content-type": "application/json",
"content-length": "402",
"x-yt-response-message": "Cannot take lock for child \"yt_wrapper\" of node //tmp since this child is locked by concurrent transaction 1c-4963-60001-4109",
"x-yt-response-code": "402",
"X-YT-Response-Parameters": {},
"x-yt-trace-id": "45496ba-72a0c1bf-9e4caecb-dc441f20",
"x-yt-error": "{\"code\":402,\"message\":\"Cannot take lock for child \\\"yt_wrapper\\\" of node //tmp since this child is locked by concurrent transaction 1c-4963-60001-4109\",\"attributes\":{\"winner_transaction\":{\"id\":\"1c-4963-60001-4109\",\"start_time\":\"2023-10-24T23:49:14.434164Z\",\"owner\":\"admin\",\"timeout\":120000,\"title\":\"my-tx\",\"parents\":[]},\"path\":\"//tmp/yt_wrapper/file_storage/yUd6fQrMuH\"}}",
"x-yt-request-id": "148d4b8c-6c302ae2-7d4eea31-3d2e202b",
"cache-control": "no-store",
"x-envoy-upstream-service-time": "27",
"Authorization": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
}
params {
"suppress_transaction_coordinator_sync": false,
"type": "file",
"recursive": true,
"ignore_existing": false,
"attributes": {
"replication_factor": 10
},
"path": "//tmp/yt_wrapper/file_storage/yUd6fQrMuH",
"output_format": {
"$attributes": {
"ignore_inner_attributes": false,
"always_create_attributes": false,
"format": "binary",
"lazy": false,
"sort_keys": false
},
"$value": "yson"
},
"mutation_id": "96373b42-435afb3e-82409a35-f11f5627",
"retry": false
}
transparent True
Cannot take lock for child "yt_wrapper" of node //tmp since this child is locked by concurrent transaction 1c-4963-60001-4109
code 402
origin ytsaurus-6b68f78676-7nj8f on 2023-10-24T23:49:24.633746Z
winner_transaction {'id': '1c-4963-60001-4109', 'start_time': '2023-10-24T23:49:14.434164Z', 'owner': 'admin', 'timeout': 120000, 'title': 'my-tx', 'parents': []}
path //tmp/yt_wrapper/file_storage/yUd6fQrMuH
```
Contributor guide
Assessment
This issue has not been assessed yet.