huggingface / huggingface/course

[Bug]: Code Quiz evaluation failing with MaxRetryError / NameResolutionError on api-inference.huggingface.co

Open
#1,270 4 comments 1 reaction 0 assignees View on GitHub
Dominant language
MDX
Stars
4.2k
Forks
1.4k
Avg merge
13m
Merged PRs (30d)
1

Description

When submitting answers for the LLM Course (Chapter 11: Exam Time! Code Quiz), the evaluation backend fails to run and throws a network/connectivity error. Even though the written Python code is logically correct, the checker marks it as "Incorrect!" due to an internal server timeout when trying to resolve the Hugging Face Inference API.

Error checking code: (MaxRetryError('HTTPSConnectionPool(host=\'api-inference.huggingface.co\', port=443): Max retries exceeded with url: /models/Qwen/Qwen2.5-Coder-32B-Instruct/v1/chat/completions (Caused by NameResolutionError(": Failed to resolve \'api-inference.huggingface.co\' ([Errno -5] No address associated with hostname)"))'), '(Request ID: d823f9e0-209a-44ad-89bc-e0314621ffe1)')

Steps to Reproduce:
Go to the Hugging Face LLM Course Chapter 11 Exam page.

Attempt the Code Quiz inside the Gradio space iframe (burtenshaw/dummy-code-quiz).

Fill in the correct answer and click "Next" / "Submit".

The evaluation fallback fails because the Space cannot reach api-inference.huggingface.co.

Expected Behavior:
The background grading script should successfully connect to the Qwen2.5-Coder-32B-Instruct endpoint via the Inference API to evaluate the user's code block submission instead of throwing a NameResolutionError.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the failure in the burtenshaw/dummy-code-quiz Gradio Space and inspect its background grading script around requests to api-inference.huggingface.co/models/Qwen/Qwen2.5-Coder-32B-Instruct/v1/chat/completions. Check how the NameResolutionError is handled and verify completion by submitting a correct quiz answer that is evaluated successfully rather than marked incorrect.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python
Domain
ai, api, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.