shift-org / shift-org/shift-docs
shift2bikes.org/robots.txt is 404
Open
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 30
- Forks
- 25
- Avg merge
- 9m
- Merged PRs (30d)
- 1
Description
Let's figure out how to properly build this to allow search indexing but disallow any nefarious behaviour.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by inspecting the site's build and deployment entry points to determine how /robots.txt is generated, then reproduce the 404 at shift2bikes.org/robots.txt. Use the issue's intended search-indexing and crawler restrictions as acceptance criteria; done means the URL serves a valid robots.txt rather than a 404 with the agreed rules.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- web-dev
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100