algolia / algolia/algolia-index-transform
Stream pages from an index to another
Open
enhancement
- Dominant language
- JavaScript
- Stars
- 13
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
In order to process large indices while keeping the memory footprint low, it would be great to process the browse page by page without writing any file on the disk:
- Get a page
- Transform it
- Push the change
- Grab next page...
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no files, tests, or entry points. Start by tracing the existing index transformation and browse flow; done means processing one browse page at a time, transforming and pushing each page without writing files to disk, then continuing through the full index.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100