huggingface / huggingface/course
[Bug]: Code Quiz evaluation failing with MaxRetryError / NameResolutionError on api-inference.huggingface.co
- Dominant language
- MDX
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 13m
- Merged PRs (30d)
- 1
Description
When submitting answers for the LLM Course (Chapter 11: Exam Time! Code Quiz), the evaluation backend fails to run and throws a network/connectivity error. Even though the written Python code is logically correct, the checker marks it as "Incorrect!" due to an internal server timeout when trying to resolve the Hugging Face Inference API.
Error checking code: (MaxRetryError('HTTPSConnectionPool(host=\'api-inference.huggingface.co\', port=443): Max retries exceeded with url: /models/Qwen/Qwen2.5-Coder-32B-Instruct/v1/chat/completions (Caused by NameResolutionError(": Failed to resolve \'api-inference.huggingface.co\' ([Errno -5] No address associated with hostname)"))'), '(Request ID: d823f9e0-209a-44ad-89bc-e0314621ffe1)')
Steps to Reproduce:
Go to the Hugging Face LLM Course Chapter 11 Exam page.
Attempt the Code Quiz inside the Gradio space iframe (burtenshaw/dummy-code-quiz).
Fill in the correct answer and click "Next" / "Submit".
The evaluation fallback fails because the Space cannot reach api-inference.huggingface.co.
Expected Behavior:
The background grading script should successfully connect to the Qwen2.5-Coder-32B-Instruct endpoint via the Inference API to evaluate the user's code block submission instead of throwing a NameResolutionError.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the failure in the burtenshaw/dummy-code-quiz Gradio Space and inspect its background grading script around requests to api-inference.huggingface.co/models/Qwen/Qwen2.5-Coder-32B-Instruct/v1/chat/completions. Check how the NameResolutionError is handled and verify completion by submitting a correct quiz answer that is evaluated successfully rather than marked incorrect.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python
- Domain
- ai, api, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100