Segmentation fault when using msgspec.Meta
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.1k
- Forks
- 193
- Avg merge
- 3d 20h
- Merged PRs (30d)
- 15
Description
Description
Hey there,
First of all, thanks for this very nice library.
I wanted to report an issue, that seems to be related to msgspec, but you will be able to better assess whether that is indeed the case.
Our CI pipelines in GitHub started failing randomly (say, 20% failure rate), with a segmentation fault error. The failures disappeared after rerunning the failing jobs (one or more times depending on our luck). The failure was when running pytest (using python -m pytest). After setting python execution to a verbose mode, we noticed that, when it failed, then it always failed when destroying the following module: pysdmx.model.types. This module uses msgspec.Meta.
(many more lines above)
# destroy pysdmx.model.types
Fatal Python error: Segmentation fault
2024-08-06T14:19:01.4875904Z
2024-08-06T14:19:01.4876092Z Current thread 0x00007f71d8383000 (most recent call first):
2024-08-06T14:19:01.4876697Z Garbage-collecting
2024-08-06T14:19:01.4876943Z <no Python frame>
2024-08-06T14:19:02.9130965Z /home/runner/work/_temp/7afba60b-b9e9-4f16-9741-aa8c0d7ce98d.sh: line 1: 1920 Segmentation fault (core dumped) poetry run python -X faulthandler -v -m pytest -p no:faulthandler --full-trace tests/api/fmr
This module was very simple:
from typing import Annotated
from msgspec import Meta
NC_NAME_ID_TYPE = Annotated[str, Meta(pattern=r"^[A-Za-z][A-Za-z\d_-]*$")]
Once we removed this, our CI pipelines started working again, 100% of the time. So, our suspicion is that there might be cases when using msgspec.Meta prevents created objects from being destroyed, but this is just a guess based on the log above and the fact that the error did not occur again when we stopped using that class. However, it could also be that our suspicion is wrong. In which case, feel free to close this issue :-).
Thanks,
Xavier
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported module-destruction failure with the Annotated[str, Meta(...)] example and the python -X faulthandler -v -m pytest command. Focus on the pysdmx.model.types destruction point and garbage collection behavior; the issue is done when the intermittent segmentation fault can be explained and no longer occurs in the relevant test run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100