Lightning-AI / Lightning-AI/pytorch-lightning
Lightning CLI StopIteration
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 31.4k
- Forks
- 3.8k
- Avg merge
- 6d 7h
- Merged PRs (30d)
- 6
Description
### Bug description
I'm trying to run pytorch lightning cli, but the script simply terminates with `StopIteration`. I've tried to debug the point the error is occurring, but I'm uncertain, by what exactly it is caused.
I'm using `lightning = "2.0.4"` and `jsonargparse = "4.22.0"`.
You can find the config file [here](https://gist.github.com/CallMeMisterOwl/ec518b41ec93a07968f2240d5a010184)
If you need more information please let me know :)
**Edit: Alright, I was able to identify the problem. It happens because `parser.link_arguments()` was pointing to a config entry that did not exist. I'm guessing one could still consider this a bug and implement proper exception handling.**
### What version are you seeing the problem on?
v2.0
### How to reproduce the bug
Init of the datamodule
```python
class SASADataModule(pl.LightningDataModule):
"""
Data module for the SASA dataset.
Manages the data loaders and data splits.
"""
def __init__(self, config: SASADataConfig): SASADataConfig):
```
The dataclass
```python
@dataclass
class SASADataConfig:
"""
Data configuration for SASA dataset
"""
data_dir: str = 'data/sasa'
embedding_path: str = 'data/sasa/sasaembeddings.h5'
np_path: str = 'data/sasa'
num_classes: Literal[1, 2, 3, 10] = 3
num_workers: int = 4
```
Init of the model
```python
class SASABaseline(pl.LightningModule):
def __init__(self,
num_classes: Literal[2, 3, 10] = 3,
class_weights: torch.Tensor = None,
lr: float = 1e-3,
weight_decay: float = 0.0,
**kwargs):
```
### Error messages and logs
```
Global seed set to 13
Traceback (most recent call last):
File "/home/d/PycharmProjects/protein_properties/src/train.py", line 34, in
main()
File "/home/d/PycharmProjects/protein_properties/src/train.py", line 21, in main
cli = MyLightningCLI(
File "/home/d/PycharmProjects/protein_properties/.venv/lib/python3.10/site-packages/lightning/pytorch/cli.py", line 343, in __init__
self.parse_arguments(self.parser, args)
File "/home/d/PycharmProjects/protein_properties/.venv/lib/python3.10/site-packages/lightning/pytorch/cli.py", line 492, in parse_arguments
self.config = parser.parse_args(args)
File "/home/d/PycharmProjects/protein_properties/.venv/lib/python3.10/site-packages/jsonargparse/_deprecated.py", line 122, in patched_parse
cfg = parse_method(*args, _skip_check=_skip_check, **kwargs)
File "/home/d/PycharmProjects/protein_properties/.venv/lib/python3.10/site-packages/jsonargparse/_core.py", line 385, in parse_args
cfg, unk = self.parse_known_args(args=args, namespace=cfg)
File "/home/d/PycharmProjects/protein_properties/.venv/lib/python3.10/site-packages/jsonargparse/_core.py", line 256, in parse_known_args
namespace, args = self._parse_known_args(args, namespace)
File "/home/d/.pyenv/versions/3.10.4/lib/python3.10/argparse.py", line 2067, in _parse_known_args
start_index = consume_optional(start_index)
File "/home/d/.pyenv/versions/3.10.4/lib/python3.10/argparse.py", line 2007, in consume_optional
take_action(action, args, option_string)
File "/home/d/.pyenv/versions/3.10.4/lib/python3.10/argparse.py", line 1935, in take_action
action(self, namespace, argument_values, option_string)
File "/home/d/PycharmProjects/protein_properties/.venv/lib/python3.10/site-packages/jsonargparse/_actions.py", line 174, in __call__
self.apply_config(parser, cfg, self.dest, values)
File "/home/d/PycharmProjects/protein_properties/.venv/lib/python3.10/site-packages/jsonargparse/_actions.py", line 197, in apply_config
cfg_file = parser.parse_path(value, **kwargs)
File "/home/d/PycharmProjects/protein_properties/.venv/lib/python3.10/site-packages/jsonargparse/_core.py", line 548, in parse_path
parsed_cfg = self.parse_string(
File "/home/d/PycharmProjects/protein_properties/.venv/lib/python3.10/site-packages/jsonargparse/_deprecated.py", line 122, in patched_parse
cfg = parse_method(*args, _skip_check=_skip_check, **kwargs)
File "/home/d/PycharmProjects/protein_properties/.venv/lib/python3.10/site-packages/jsonargparse/_core.py", line 591, in parse_string
cfg = self._load_config_parser_mode(cfg_str, cfg_path, ext_vars, previous_config.get())
File "/home/d/PycharmProjects/protein_properties/.venv/lib/python3.10/site-packages/jsonargparse/_core.py", line 635, in _load_config_parser_mode
return self._apply_actions(cfg_dict, prev_cfg=prev_cfg)
File "/home/d/PycharmProjects/protein_properties/.venv/lib/python3.10/site-packages/jsonargparse/_core.py", line 1246, in _apply_actions
value = self._check_value_key(action, value, action_dest, prev_cfg)
File "/home/d/PycharmProjects/protein_properties/.venv/lib/python3.10/site-packages/jsonargparse/_core.py", line 1296, in _check_value_key
value = action._check_type(value, cfg=cfg)
File "/home/d/PycharmProjects/protein_properties/.venv/lib/python3.10/site-packages/jsonargparse/_typehints.py", line 462, in _check_type
val = adapt_typehints(val, self._typehint, **kwargs)
File "/home/d/PycharmProjects/protein_properties/.venv/lib/python3.10/site-packages/jsonargparse/_typehints.py", line 828, in adapt_typehints
val = adapt_class_type(val, serialize, instantiate_classes, sub_add_kwargs, prev_val=prev_val)
File "/home/d/PycharmProjects/protein_properties/.venv/lib/python3.10/site-packages/jsonargparse/_typehints.py", line 1020, in adapt_class_type
action = next(a for a in parser._actions if a.dest == parent_key)
StopIteration
```
```
### Environment
Current environment
* CUDA:
- GPU:
- NVIDIA GeForce GTX 1060 6GB
- available: True
- version: 11.7
* Lightning:
- lightning: 2.0.4
- lightning-cloud: 0.5.37
- lightning-utilities: 0.9.0
- pytorch-lightning: 2.0.4
- torch: 2.0.0
- torchmetrics: 0.11.4
* Packages:
- absl-py: 1.4.0
- aiohttp: 3.8.4
- aiosignal: 1.3.1
- antlr4-python3-runtime: 4.9.3
- anyio: 3.6.2
- appdirs: 1.4.4
- argon2-cffi: 21.3.0
- argon2-cffi-bindings: 21.2.0
- arrow: 1.2.3
- asttokens: 2.2.1
- async-lru: 2.0.2
- async-timeout: 4.0.2
- attrs: 23.1.0
- babel: 2.12.1
- backcall: 0.2.0
- beautifulsoup4: 4.12.2
- biopython: 1.81
- biotite: 0.36.1
- bleach: 6.0.0
- blessed: 1.20.0
- cachetools: 5.3.1
- certifi: 2023.5.7
- cffi: 1.15.1
- charset-normalizer: 3.1.0
- click: 8.1.3
- cmake: 3.26.3
- comm: 0.1.3
- contourpy: 1.0.7
- croniter: 1.3.15
- cycler: 0.11.0
- dateutils: 0.6.12
- debugpy: 1.6.7
- decorator: 5.1.1
- deepdiff: 6.3.0
- defusedxml: 0.7.1
- docker-pycreds: 0.4.0
- docstring-parser: 0.15
- executing: 1.2.0
- fastapi: 0.99.1
- fastjsonschema: 2.16.3
- filelock: 3.12.0
- fonttools: 4.39.3
- fqdn: 1.5.1
- frozenlist: 1.3.3
- fsspec: 2023.6.0
- gitdb: 4.0.10
- gitpython: 3.1.31
- google-auth: 2.20.0
- google-auth-oauthlib: 1.0.0
- grpcio: 1.56.0
- h11: 0.14.0
- h5py: 3.8.0
- hydra-core: 1.3.2
- idna: 3.4
- importlib-resources: 5.12.0
- inquirer: 3.1.3
- ipykernel: 6.22.0
- ipython: 8.13.2
- ipython-genutils: 0.2.0
- ipywidgets: 8.0.6
- isoduration: 20.11.0
- itsdangerous: 2.1.2
- jedi: 0.18.2
- jinja2: 3.1.2
- joblib: 1.2.0
- json5: 0.9.14
- jsonargparse: 4.22.0
- jsonpointer: 2.3
- jsonschema: 4.17.3
- jupyter: 1.0.0
- jupyter-client: 8.2.0
- jupyter-console: 6.6.3
- jupyter-core: 5.3.0
- jupyter-events: 0.6.3
- jupyter-lsp: 2.2.0
- jupyter-server: 2.5.0
- jupyter-server-terminals: 0.4.4
- jupyterlab: 4.0.2
- jupyterlab-pygments: 0.2.2
- jupyterlab-server: 2.23.0
- jupyterlab-widgets: 3.0.7
- kiwisolver: 1.4.4
- lightning: 2.0.4
- lightning-cloud: 0.5.37
- lightning-utilities: 0.9.0
- lit: 16.0.3
- markdown: 3.4.3
- markdown-it-py: 3.0.0
- markupsafe: 2.1.2
- matplotlib: 3.7.1
- matplotlib-inline: 0.1.6
- mdurl: 0.1.2
- mistune: 2.0.5
- mpmath: 1.3.0
- msgpack: 1.0.5
- multidict: 6.0.4
- nbclassic: 1.0.0
- nbclient: 0.7.4
- nbconvert: 7.3.1
- nbformat: 5.8.0
- nest-asyncio: 1.5.6
- networkx: 3.1
- notebook: 6.5.4
- notebook-shim: 0.2.3
- numpy: 1.24.3
- nvidia-cublas-cu11: 11.10.3.66
- nvidia-cuda-cupti-cu11: 11.7.101
- nvidia-cuda-nvrtc-cu11: 11.7.99
- nvidia-cuda-runtime-cu11: 11.7.99
- nvidia-cudnn-cu11: 8.5.0.96
- nvidia-cufft-cu11: 10.9.0.58
- nvidia-curand-cu11: 10.2.10.91
- nvidia-cusolver-cu11: 11.4.0.1
- nvidia-cusparse-cu11: 11.7.4.91
- nvidia-nccl-cu11: 2.14.3
- nvidia-nvtx-cu11: 11.7.91
- oauthlib: 3.2.2
- omegaconf: 2.3.0
- ordered-set: 4.1.0
- packaging: 23.1
- pandas: 2.0.1
- pandocfilters: 1.5.0
- parso: 0.8.3
- pathtools: 0.1.2
- pexpect: 4.8.0
- pickleshare: 0.7.5
- pillow: 9.5.0
- pip: 23.1.2
- platformdirs: 3.5.0
- prometheus-client: 0.16.0
- prompt-toolkit: 3.0.38
- protobuf: 4.22.4
- psutil: 5.9.5
- ptyprocess: 0.7.0
- pure-eval: 0.2.2
- pyasn1: 0.5.0
- pyasn1-modules: 0.3.0
- pycparser: 2.21
- pydantic: 1.10.11
- pygments: 2.15.1
- pyjwt: 2.7.0
- pyparsing: 3.0.9
- pyrsistent: 0.19.3
- python-dateutil: 2.8.2
- python-editor: 1.0.4
- python-json-logger: 2.0.7
- python-multipart: 0.0.6
- pytorch-lightning: 2.0.4
- pytz: 2023.3
- pyyaml: 6.0
- pyzmq: 25.0.2
- qtconsole: 5.4.3
- qtpy: 2.3.1
- readchar: 4.0.5
- requests: 2.30.0
- requests-oauthlib: 1.3.1
- rfc3339-validator: 0.1.4
- rfc3986-validator: 0.1.1
- rich: 13.4.2
- rsa: 4.9
- scikit-learn: 1.2.2
- scipy: 1.9.3
- seaborn: 0.12.2
- send2trash: 1.8.2
- sentry-sdk: 1.22.1
- setproctitle: 1.3.2
- setuptools: 67.7.2
- six: 1.16.0
- smmap: 5.0.0
- sniffio: 1.3.0
- soupsieve: 2.4.1
- stack-data: 0.6.2
- starlette: 0.27.0
- starsessions: 1.3.0
- sympy: 1.11.1
- tensorboard: 2.13.0
- tensorboard-data-server: 0.7.1
- tensorboardx: 2.6.1
- terminado: 0.17.1
- threadpoolctl: 3.1.0
- tinycss2: 1.2.1
- tomli: 2.0.1
- torch: 2.0.0
- torchmetrics: 0.11.4
- tornado: 6.3.1
- tqdm: 4.65.0
- traitlets: 5.9.0
- triton: 2.0.0
- typeshed-client: 2.3.0
- typing-extensions: 4.5.0
- tzdata: 2023.3
- uri-template: 1.2.0
- urllib3: 1.26.15
- uvicorn: 0.22.0
- wandb: 0.15.2
- wcwidth: 0.2.6
- webcolors: 1.13
- webencodings: 0.5.1
- websocket-client: 1.5.1
- websockets: 11.0.3
- werkzeug: 2.3.6
- wheel: 0.40.0
- widgetsnbextension: 4.0.7
- yarl: 1.9.2
* System:
- OS: Linux
- architecture:
- 64bit
- ELF
- processor:
- python: 3.10.4
- release: 6.1.31-2-MANJARO
- version: #1 SMP PREEMPT_DYNAMIC Sun Jun 4 12:31:46 UTC 2023
### More info
_No response_
cc @carmocca @mauvilsa
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with jsonargparse/_typehints.py at adapt_class_type, the entry point shown in the traceback, and reproduce the configuration with Lightning CLI and the invalid parser.link_arguments() target. Done means the invalid reference raises an informative exception instead of StopIteration, with regression coverage for this configuration path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100