inducer / inducer/relate

Out of memory inside of a container not cleanly diagnosed

Open
#539 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
435
Forks
124
Avg merge
5h 40m
Merged PRs (30d)
14

Description

It used to just time out. Now it seems to be possible that the OOM killer gets these processes and we may get this instead:

```
Traceback (most recent call last):
File "./course/page/code.py", line 635, in grade
run_timeout=self.page_desc.timeout)
File "./course/page/code.py", line 288, in request_python_run_with_retries
result = request_python_run(run_req, run_timeout, image=image)
File "./course/page/code.py", line 220, in request_python_run
http_response = connection.getresponse()
File "/usr/lib/python3.6/http/client.py", line 1331, in getresponse
response.begin()
File "/usr/lib/python3.6/http/client.py", line 297, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.6/http/client.py", line 266, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response
```

Which is tough from a grading perspective, since it doesn't assign fault.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the grading flow in course/page/code.py, especially grade, request_python_run_with_retries, and request_python_run. Examine how connection.getresponse() handles a closed container process and how the OOM case can be distinguished from other failures. Done means the grader reports the container memory failure clearly enough to assign fault rather than showing only RemoteDisconnected.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
backend, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.