stac-utils / stac-utils/rustac
Crawl extension
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 145
- Forks
- 24
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 12
Description
Over at https://github.com/gadomski/heystac I've implemented rudimentary crawling:
- Visit each child catalog and collection
- For each collection, fetch one item either from its
itemlinks or from itsitemsendpoint
This seems generalizable to an extension, since the output could be useful it if were structured (things like crawl:start_datetime and crawl:end_datetime, roundtrip request times, etc). We should implement in stac-extensions, write up a spec and a json-schema for it, and then maybe make it its own https://github.com/stac-extensions.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the rudimentary crawling implementation in gadomski/heystac, especially its handling of child catalogs, collections, item links, and items endpoints. Define the extension’s crawl metadata and document the behavior, then add the requested specification and JSON Schema in stac-extensions; done means the proposal is structured and reviewable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- documentation
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100