huggingface / huggingface/dataset-viewer

CI spawning check error for PR from fork: The provided API token is invalid

Open
#1,085 12 comments 0 reactions 0 assignees View on GitHub
bug P2 tests
Dominant language
Python
Stars
899
Forks
130
Avg merge
9h 26m
Merged PRs (30d)
14

Description

See: https://github.com/huggingface/datasets-server/actions/runs/4796071992/jobs/8531405657?pr=1084#logs
```
______________________ test_real_check_spawning_response _______________________

app_config = AppConfig(assets=AssetsConfig(base_url='http://localhost/assets', storage_directory=None), cache=CacheConfig(mongo_dat...per_batch=1000, spawning_token='dummy_spawning_token', max_concurrent_requests_number=100, max_requests_per_second=50))

@pytest.mark.asyncio
async def test_real_check_spawning_response(app_config: AppConfig) -> None:
semaphore = Semaphore(value=10)
limiter = AsyncLimiter(10, time_period=1)

headers = {"Authorization": f"API {CI_SPAWNING_TOKEN}"}
async with ClientSession(headers=headers) as session:
image_url = "http://testurl.test/test_image.jpg"
image_urls = [image_url]
spawning_url = app_config.urls_scan.spawning_url
spawning_response = await check_spawning(image_urls, session, semaphore, limiter, spawning_url)
assert spawning_response and isinstance(spawning_response, dict)
> assert spawning_response["urls"] and isinstance(spawning_response["urls"], list)
E KeyError: 'urls'

tests/job_runners/split/test_opt_in_out_urls_scan_from_streaming.py:309: KeyError
```
where `spawning_response` is:
```
{'detail': 'The provided API token is invalid.'}
```

Contributor guide

Open the contributing guide

Research direction

Start with tests/job_runners/split/test_opt_in_out_urls_scan_from_streaming.py and test_real_check_spawning_response, then inspect check_spawning and the CI spawning-token configuration. Reproduce the fork PR failure and verify that the spawning response provides the expected urls list instead of an invalid-token detail.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, ci-cd, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.