invoke-ai / invoke-ai/invoke-training
PyArrow has no attribute 'PyExtensionType'
- Dominant language
- Python
- Stars
- 206
- Forks
- 35
- PR merge metrics
- No merged PRs in 30d
Description
When attempting to load SDXL for performing textual inversion, I receive the following error:
```
Traceback (most recent call last):
File ".venv/bin/invoke-train", line 6, in
from invoke_training.scripts.invoke_train import main
File "invoke-training/src/invoke_training/scripts/invoke_train.py", line 8, in
from invoke_training.pipelines.invoke_train import train
File "invoke-training/src/invoke_training/pipelines/invoke_train.py", line 4, in
from invoke_training.pipelines._experimental.sd_dpo_lora.train import train as train_sd_ddpo_lora
File "invoke-training/src/invoke_training/pipelines/_experimental/sd_dpo_lora/train.py", line 21, in
from invoke_training._shared.accelerator.accelerator_utils import (
File "invoke-training/src/invoke_training/_shared/accelerator/accelerator_utils.py", line 5, in
import datasets
File ".venv/lib/python3.12/site-packages/datasets/__init__.py", line 22, in
from .arrow_dataset import Dataset
File ".venv/lib/python3.12/site-packages/datasets/arrow_dataset.py", line 67, in
from .arrow_writer import ArrowWriter, OptimizedTypedSequence
File ".venv/lib/python3.12/site-packages/datasets/arrow_writer.py", line 27, in
from .features import Features, Image, Value
File ".venv/lib/python3.12/site-packages/datasets/features/__init__.py", line 18, in
from .features import Array2D, Array3D, Array4D, Array5D, ClassLabel, Features, Sequence, Value
File ".venv/lib/python3.12/site-packages/datasets/features/features.py", line 634, in
class _ArrayXDExtensionType(pa.PyExtensionType):
^^^^^^^^^^^^^^^^^^
AttributeError: module 'pyarrow' has no attribute 'PyExtensionType'. Did you mean: 'ExtensionType'?
```
According to ChatGPT (I can't say for sure how correct this is), the reasoning is that:
> In PyArrow ≥15.0.0, the PyExtensionType class was deprecated and later removed, replaced by ExtensionType.
>
> Older versions of datasets (before 2.19.0) still rely on PyExtensionType.
>
> So your datasets library and pyarrow are out of sync.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.