Find a way to avoid unexpected switching from yson bindings to python impl
- Dominant language
- C++
- Stars
- 2.2k
- Forks
- 219
- PR merge metrics
- No merged PRs in 30d
Description
After update to `ytsaurus-client==0.13.8` I have just missed this warning:
```bash
Warning! Failed to import YSON bindings: cannot import name 'dump_parquete' from 'yt_yson_bindings'
```
Luckily, I had a regular process that eventually crashed and I noticed it, here is the backtrace example:
```bash
return client_api.write_table(
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/echunaev/.local/lib/python3.11/site-packages/yt/wrapper/table_commands.py", line 234, in write_table
make_write_request(
File "/home/echunaev/.local/lib/python3.11/site-packages/yt/wrapper/heavy_commands.py", line 202, in make_write_request
for chunk in stream.into_chunks(chunk_size).split_chunks(2 * MB):
File "/home/echunaev/.local/lib/python3.11/site-packages/yt/wrapper/stream.py", line 130, in __next__
item = next(self._iter)
^^^^^^^^^^^^^^^^
File "/home/echunaev/.local/lib/python3.11/site-packages/yt/wrapper/stream.py", line 48, in _split_chunks_by_max_size
for chunk in stream:
File "/home/echunaev/.local/lib/python3.11/site-packages/yt/wrapper/stream.py", line 130, in __next__
item = next(self._iter)
^^^^^^^^^^^^^^^^
File "/home/echunaev/.local/lib/python3.11/site-packages/yt/wrapper/stream.py", line 85, in _merge_items_into_chunks
for item in items:
File "/home/echunaev/.local/lib/python3.11/site-packages/yt/wrapper/stream.py", line 31, in _stream_or_empty_bytes
for line in stream:
File "/home/echunaev/.local/lib/python3.11/site-packages/yt/wrapper/stream.py", line 130, in __next__
item = next(self._iter)
^^^^^^^^^^^^^^^^
File "/home/echunaev/.local/lib/python3.11/site-packages/yt/wrapper/table_helpers.py", line 78, in
stream = (format.dumps_row(row) for row in stream)
^^^^^^^^^^^^^^^^^^^^^
File "/home/echunaev/.local/lib/python3.11/site-packages/yt/wrapper/format.py", line 320, in dumps_row
self.dump_row(row, stream)
File "/home/echunaev/.local/lib/python3.11/site-packages/yt/wrapper/format.py", line 306, in dump_row
self._dump_row(row, stream)
File "/home/echunaev/.local/lib/python3.11/site-packages/yt/wrapper/format.py", line 750, in _dump_row
self._check_bindings()
File "/home/echunaev/.local/lib/python3.11/site-packages/yt/wrapper/format.py", line 718, in _check_bindings
raise YtFormatError('YSON bindings required. Try to use other format or install bindings. '
yt.wrapper.format.YtFormatError: YSON bindings required. Try to use other format or install bindings. Bindings are shipped as additional package and can be installed as pip package "ytsaurus-yson"
```
But if I didn’t have such a process, I would have been debugging for a long time why everything began to work more slowly.
I have already made dirty workaround in my code like this:
```python
try:
from yt_yson_bindings import dump_parquete # pyright: ignore[reportGeneralTypeIssues]
except ImportError:
import yt_yson_bindings
def dump_parquete(*args: Any, **kwargs: Any):
raise RuntimeError("yt_yson_bindings are incompatible, dump_parquete can not be used")
# Workaround for incompatible yson bindings version.
yt_yson_bindings.dump_parquete = dump_parquete # pyright: ignore[reportGeneralTypeIssues]
```
So bindings release is not critical for me, but I really want to have some protection that will prevent unexpected switching to python parsing.
For example something like this:
```python
TYPE = None
try:
from yt_yson_bindings import load, loads, dump, dumps, dump_parquete # noqa
TYPE = "BINARY"
except ImportError as error:
have_yson_bindings = False
try:
import yt_yson_bindings as _yt_yson_bindings
have_yson_bindings = True
except ImportError:
pass
if have_yson_bindings:
allow_incompatible_yson_bindings = os.getenv("YT_ALLOW_INCOMPATIBLE_YSON_BINDINGS")
if allow_incompatible_yson_bindings:
print("Warning! Failed to import YSON bindings: " + message, file=_sys.stderr)
else:
raise ImportError(f"found incompatible yt_yson_bindings version: {_yt_yson_bindings.__version__}, please update it to XXX or set YT_ALLOW_INCOMPATIBLE_YSON_BINDINGS env (or remove it) ")
```
Instead of this:
https://github.com/ytsaurus/ytsaurus/blob/68798316ebb0ce810ee4bda39867d4466d2cb807/yt/python/yt/yson/__init__.py#L40-L51
```
Contributor guide
Assessment
This issue has not been assessed yet.