pytest-dev / pytest-dev/pytest-xdist

how to debug crashed gw worker?

Open
#998 2 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
287
Avg merge
9h 30m
Merged PRs (30d)
2

Description

Hi,

we are running our tests environment using pytest-xdist inside kubernetes pods. during the last three weeks we started suffering from crashed workers and we don't know how to debug what is the root cause of the failure.

I would kindly ask from someone to hint me on how to start looking at it, as I see only failures with crashed workers and not the reason why.

[gw8] Python 3.8.10 (default, May 26 2023, 14:05:08) -- [GCC 9.4.0]
13:33:50 [gw2] node down: Not properly terminated
13:33:50 [gw2] [ 50%] FAILED testsuite/ond/sdc/ono_oni_vz/ui/create_service/test_ond_41377_catalog_page_search.py::test_ond_41377_catalog_page_search
13:33:50
13:33:50 replacing crashed worker gw2
13:33:50
[gw9] linux Python 3.8.10 cwd: /home/jenkins/agent/workspace/QA/orion-orchestrator
13:33:50
13:33:51

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The report names pytest-xdist, pytest tests, Python 3.8.10, and Kubernetes pods but no repository files or targeted failing test to change. Start by correlating the crashed-worker output with the Kubernetes pod logs and the named failing test. Done would require identifying a reproducible root cause and documenting or fixing it, but the issue does not define a specific expected change.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, python
Domain
infrastructure, testing-qa
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.