allenai / allenai/molmo

PlotQa dataset causes training errors in the preprocessor

Aberta
#23 2 comentários 0 reações 0 responsáveis Ver no GitHub
Linguagem predominante
Python
Estrelas
933
Forks
96
Métricas de merge de PRs
Nenhum PR com merge em 30d

Descrição

When including PlotQa in the multitask training, we encounter the following error

```
ValueError: Caught ValueError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/PIL/Image.py", line 3281, in open
fp.seek(0)
AttributeError: 'dict' object has no attribute 'seek'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/pscratch/sd/t/thes/molmo_code/olmo/data/model_preprocessor.py", line 833, in __call__
image = load_image(example["image"])
File "/pscratch/sd/t/thes/molmo_code/olmo/data/model_preprocessor.py", line 54, in load_image
with PIL.Image.open(image_path) as image:
File "/usr/local/lib/python3.10/dist-packages/PIL/Image.py", line 3283, in open
fp = io.BytesIO(fp.read())
AttributeError: 'dict' object has no attribute 'read'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop
data = fetcher.fetch(index) # type: ignore[possibly-undefined]
File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/_utils/fetch.py", line 32, in fetch
data.append(next(self.dataset_iter))
File "/pscratch/sd/t/thes/molmo_code/olmo/data/iterable_dataset_mixture.py", line 120, in __iter__
yield dataset.get(int(shuffled_order[count % len(dataset)]), epoch)
File "/pscratch/sd/t/thes/molmo_code/olmo/data/dataset.py", line 68, in get
item = self.preprocessor(item, rng)
File "/pscratch/sd/t/thes/molmo_code/olmo/data/model_preprocessor.py", line 836, in __call__
raise ValueError(f"Could not load image: {example['image']}")
ValueError: Could not load image: {'bytes': b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x03\xca\x00\x00\x02\x8a\x08\x06\x00\x00\x00\xe5@\xa2\xec\x00\x00\x97NIDATx\x9c\xec\xddy\\Tu\xe3\xfe\xff\xeb\xc0\xb0\x08\xb8\xe1\x02\xa2\xa8l!\x02i\xa6h*\x86\xd9\xddj\xa5m\xa6\x96v\xa7-\xb7\xa5u\xabuWfH\xe57+\xef,\xad M\xbd\xcd2\xdb\xee\xbaK3Ss\xcd\xd4l1\x97TpA\r\xc3\x05E`X\x1c\x98\xdf\x1f\xfe<\x1fHP\x18\x87\x19\x93\xd7\xf3\xf1\x98G\xcd93g\xae\x19f\x8es\xcd9\xe7}\x0c\xbb\xdd~\xa5\xa4$\x01\x00\x00\x00\x00\x00Yt\xaa$Opo\x0c\x00\x00\x00\x00\x00.\x0c\x1e\xee\x0e\x00\x00\x00\x00\x00\xc0\x85\x84\xa2\x0c\x00\x00\x00\x00@9\x14e\x00\x00\x00\x00\x00\xca\xa1(\x03\x00\x00\x00\x00P\x0eE\x19\x00\x00\x00\x00\x80r(\xca\x00\x00\x00\x00\x00\x94CQ\x06\x00\x00\x00\x00\xa0\x1c\x8a2\x00\x00\x00\x00\x00\xe5P\x94\x01\x00\x00\x00\x00(\x87\xa2\x0c\x00\x00\x00\x
```

This error can be simply reproduced by running

```
from olmo.data.model_preprocessor import load_image
from olmo.data.academic_datasets import PlotQa
x = PlotQa("train", in_memory=False)
load_image(x[0])
```

Our environment is as follows

Package Version
---------------------------------- -------------------------
absl-py 2.1.0
accelerate 1.3.0
aiohttp 3.9.5
aiosignal 1.3.1
alembic 1.14.1
annotated-types 0.7.0
antlr4-python3-runtime 4.9.3
anyio 4.8.0
apex 0.1
argon2-cffi 23.1.0
argon2-cffi-bindings 21.2.0
arrow 1.3.0
asciitree 0.3.3
asttokens 2.4.1
astunparse 1.6.3
async-timeout 4.0.3
attrs 23.2.0
audioread 3.0.1
beautifulsoup4 4.12.3
bleach 6.1.0
blinker 1.9.0
blis 0.7.11
boto3 1.37.2
botocore 1.37.2
cached_path 1.6.7
cachetools 5.3.3
catalogue 2.0.10
certifi 2024.6.2
cffi 1.16.0
charset-normalizer 3.3.2
click 8.1.7
cloudpathlib 0.16.0
cloudpickle 3.0.0
cmake 3.29.3
comm 0.2.2
confection 0.1.5
contourpy 1.2.1
cuda-python 12.5.0
cudf 24.4.0
cugraph 24.4.0
cugraph-dgl 24.4.0
cugraph-equivariant 24.4.0
cugraph-pyg 24.4.0
cugraph-service-client 24.4.0
cugraph-service-server 24.4.0
cuml 24.4.0
cupy-cuda12x 13.0.0
cycler 0.12.1
cymem 2.0.8
Cython 3.0.10
dask 2024.1.1
dask-cuda 24.4.0
dask-cudf 24.4.0
dask-expr 0.4.0
databricks-sdk 0.44.1
datasets 3.3.3.dev0
debugpy 1.8.1
decorator 5.1.1
deepspeed 0.16.3
defusedxml 0.7.1
Deprecated 1.2.18
dill 0.3.8
distributed 2024.1.1
dm-tree 0.1.8
docker 7.1.0
docker-pycreds 0.4.0
editdistance 0.8.1
einops 0.8.0
entrypoints 0.4
evaluate 0.4.3
exceptiongroup 1.2.1
execnet 2.1.1
executing 2.0.1
expecttest 0.1.3
Farama-Notifications 0.0.4
fasteners 0.19
fastjsonschema 2.19.1
fastrlock 0.8.2
filelock 3.14.0
flash-attn 2.4.2
Flask 3.1.0
fonttools 4.53.0
fqdn 1.5.1
frozenlist 1.4.1
fsspec 2024.5.0
gast 0.5.4
gitdb 4.0.12
GitPython 3.1.44
google-api-core 2.24.1
google-auth 2.29.0
google-auth-oauthlib 0.4.6
google-cloud-core 2.4.2
google-cloud-storage 2.19.0
google-crc32c 1.6.0
google-resumable-media 2.7.2
googleapis-common-protos 1.68.0
graphene 3.4.3
graphql-core 3.2.6
graphql-relay 3.2.0
greenlet 3.1.1
grpcio 1.62.1
gunicorn 23.0.0
gymnasium 1.0.0
h5py 3.12.1
hjson 3.1.0
huggingface-hub 0.27.1
hypothesis 5.35.1
idna 3.7
igraph 0.11.5
imageio 2.37.0
importlib_metadata 7.1.0
iniconfig 2.0.0
intel-openmp 2021.4.0
ipykernel 6.29.4
ipympl 0.9.6
ipython 8.21.0
ipython-genutils 0.2.0
ipywidgets 8.1.5
isoduration 20.11.0
itsdangerous 2.2.0
jedi 0.19.1
Jinja2 3.1.4
jmespath 1.0.1
joblib 1.4.2
json5 0.9.25
jsonpointer 3.0.0
jsonschema 4.22.0
jsonschema-specifications 2023.12.1
jupyter_client 8.6.2
jupyter_core 5.7.2
jupyter-events 0.11.0
jupyter_server 2.15.0
jupyter_server_proxy 4.4.0
jupyter_server_terminals 0.5.3
jupyter-tensorboard 0.2.0
jupyterlab 2.3.2
jupyterlab_pygments 0.3.0
jupyterlab-server 1.2.0
jupyterlab_widgets 3.0.13
jupytext 1.16.2
kiwisolver 1.4.5
kvikio 24.4.0
langcodes 3.4.0
language_data 1.2.0
lazy_loader 0.4
librosa 0.10.1
lightning 2.5.0.post0
lightning-thunder 0.2.0.dev0
lightning-utilities 0.11.2
lintrunner 0.12.5
llvmlite 0.42.0
locket 1.0.0
looseversion 1.3.0
lz4 4.3.3
Mako 1.3.9
marisa-trie 1.1.1
Markdown 3.6
markdown-it-py 3.0.0
MarkupSafe 2.1.5
matplotlib 3.9.0
matplotlib-inline 0.1.7
mdit-py-plugins 0.4.1
mdurl 0.1.2
mistune 3.0.2
mkl 2021.1.1
mkl-devel 2021.1.1
mkl-include 2021.1.1
mlflow 2.20.3
mlflow-skinny 2.20.3
mock 5.1.0
mpi4py 4.0.1
mpmath 1.3.0
msgpack 1.0.8
multidict 6.0.5
multiprocess 0.70.16
murmurhash 1.0.10
nbclient 0.10.0
nbconvert 7.16.4
nbformat 5.10.4
nersc-tensorboard-helper 0.1
nest-asyncio 1.6.0
networkx 3.3
ninja 1.11.1.1
notebook 6.4.10
numba 0.59.1
numcodecs 0.11.0
numpy 1.24.4
nvfuser 0.2.3a0+f73ff1b
nvidia-cudnn-frontend 1.3.0
nvidia-dali-cuda120 1.38.0
nvidia-modelopt 0.11.2
nvidia-nvimgcodec-cu12 0.2.0.7
nvidia-pyindex 1.0.9
nvtx 0.2.5
nx-cugraph 24.4.0
oauthlib 3.2.2
omegaconf 2.3.0
onnx 1.16.0
opencv 4.7.0
opentelemetry-api 1.30.0
opentelemetry-sdk 1.30.0
opentelemetry-semantic-conventions 0.51b0
opt-einsum 3.3.0
optree 0.11.0
ormsgpack 1.7.0
overrides 7.7.0
packaging 24.0
pandas 2.2.1
pandocfilters 1.5.1
parameterized 0.9.0
parso 0.8.4
partd 1.4.2
pexpect 4.9.0
pillow 10.3.0
pip 24.3.1
platformdirs 4.2.2
pluggy 1.5.0
ply 3.11
polygraphy 0.49.11
pooch 1.8.1
preshed 3.0.9
prettytable 3.10.0
prometheus_client 0.20.0
prompt_toolkit 3.0.45
proto-plus 1.26.0
protobuf 4.24.4
psutil 5.9.8
ptyprocess 0.7.0
pure-eval 0.2.2
py-cpuinfo 9.0.0
pyarrow 19.0.0
pyasn1 0.6.0
pyasn1_modules 0.4.0
pybind11 2.12.0
pybind11_global 2.12.0
pycocotools 2.0+nv0.8.0
pycparser 2.22
pydantic 2.7.3
pydantic_core 2.18.4
pyg-lib 0.4.0
Pygments 2.18.0
pylibcugraph 24.4.0
pylibcugraphops 24.4.0
pylibraft 24.4.0
pylibwholegraph 24.4.0
pynvjitlink 0.2.3
pynvml 11.4.1
pyparsing 3.1.2
pytest 8.1.1
pytest-flakefinder 1.1.0
pytest-rerunfailures 14.0
pytest-shard 0.1.2
pytest-xdist 3.6.1
python-dateutil 2.9.0.post0
python-hostlist 1.23.0
python-json-logger 3.2.1
pytorch-lightning 2.5.0.post0
pytorch-quantization 2.1.2
pytorch-triton 3.0.0+989adb9a2
pytz 2024.1
PyYAML 6.0.1
pyzmq 26.0.3
raft-dask 24.4.0
rapids-dask-dependency 24.4.0a0
ray 2.41.0
referencing 0.35.1
regex 2024.5.15
requests 2.32.3
requests-oauthlib 2.0.0
rfc3339-validator 0.1.4
rfc3986-validator 0.1.1
rich 13.7.1
rmm 24.4.0
rpds-py 0.18.1
rsa 4.9
ruamel.yaml 0.18.10
ruamel.yaml.clib 0.2.12
s3transfer 0.11.3
safetensors 0.5.2
scikit-image 0.25.0
scikit-learn 1.5.0
scipy 1.13.1
Send2Trash 1.8.3
sentry-sdk 2.20.0
setproctitle 1.3.4
setuptools 68.2.2
simpervisor 1.0.0
six 1.16.0
smart-open 6.4.0
smmap 5.0.2
sniffio 1.3.1
sortedcontainers 2.4.0
soundfile 0.12.1
soupsieve 2.5
soxr 0.3.7
spacy 3.7.4
spacy-legacy 3.0.12
spacy-loggers 1.0.5
sphinx_glpi_theme 0.6
SQLAlchemy 2.0.38
sqlparse 0.5.3
srsly 2.4.8
stack-data 0.6.3
sympy 1.12.1
tabulate 0.9.0
tbb 2021.12.0
tblib 3.0.0
tensorboard 2.9.0
tensorboard-data-server 0.6.1
tensorboard-plugin-wit 1.8.1
tensorboardX 2.6.2.2
tensorrt 10.1.0
terminado 0.18.1
texttable 1.7.0
thinc 8.2.3
threadpoolctl 3.5.0
thriftpy2 0.5.0
tifffile 2025.1.10
tinycss2 1.3.0
tokenizers 0.21.0
tomli 2.0.1
toolz 0.12.1
torch 2.4.0a0+f70bd71a48.nv24.6
torch-cluster 1.6.3
torch-geometric 2.6.1
torch-scatter 2.1.2
torch-sparse 0.6.18
torch-tensorrt 2.4.0a0
torchinfo 1.8.0
torchmetrics 1.6.1
torchsummary 1.5.1
torchvision 0.19.0a0
tornado 6.4
tqdm 4.66.4
traitlets 5.9.0
transformer-engine 1.7.0+4e7caa1
transformers 4.48.1
treelite 4.1.2
typer 0.9.4
types-dataclasses 0.6.6
types-python-dateutil 2.9.0.20241206
typing_extensions 4.12.0
tzdata 2024.1
ucx-py 0.37.0
uri-template 1.3.0
urllib3 2.0.7
wandb 0.19.4
wasabi 1.1.3
wcwidth 0.2.13
weasel 0.3.4
webcolors 24.11.1
webencodings 0.5.1
websocket-client 1.8.0
Werkzeug 3.1.3
wheel 0.43.0
widgetsnbextension 4.0.13
wrapt 1.17.2
xdoctest 1.0.2
xgboost 2.0.3
xxhash 3.5.0
yarl 1.9.4
zarr 2.18.2
zict 3.0.0
zipp 3.19.0

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.