algolia / algolia/docsearch-scraper
Output CI-friendly progress messages
- Dominant language
- Python
- Stars
- 322
- Forks
- 115
- PR merge metrics
- No merged PRs in 30d
Description
Currently, the scraper assumes it's writing progress messages to an ANSI-compatible terminal. As a result, the progress messages look like this in a CI environment:
```
[94m> DocSearch: [0mhttps://docs.couchbase.com/server/6.0/introduction/intro.html ([93m51 records[0m)
[94m> DocSearch: [0mhttps://docs.couchbase.com/home/contribute/includes.html ([93m23 records[0m)
[94m> DocSearch: [0mhttps://docs.couchbase.com/server/6.0/n1ql/n1ql-language-reference/index.html ([93m28 records[0m)
```
Either add an option to output plain messages or automatically detect if ANSI color codes are not supported.
The easiest way to accomplish this might be to route the messages through a logger which can be configured separately. I'd also be interested in silencing the messages completely, which a logger would also help with.
Contributor guide
Research direction
Start by locating where the scraper emits its progress messages, then reproduce the output in a CI or non-ANSI terminal environment. Done means progress output is readable without ANSI escape codes, with either plain-output or silencing behavior supported as appropriate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100