CentreForDigitalHumanities / CentreForDigitalHumanities/Word2VecElastic
Intervals may exclude final years
- Dominant language
- Python
- Stars
- 1
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
If the total number of years is not a multiple of the shift size, some of the last years in the data will not be included in any model.
For example, the EU parliament corpus has date range of 1999-2026. If I set the interval size to 10 and the shift to 5, the intervals are:
- 1999-2009
- 2004-2014
- 2009-2019
- 2014-2024
So 2025-2026 never get included.
It is not essential that intervals are all the same length, so perhaps it would be preferable if the last interval here was extended to 2014-2026. See also #8
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the interval construction and the code that selects corpus years for each model; no specific files or tests are named in the issue. Reproduce the 1999-2026 example with interval size 10 and shift 5, then verify that the final interval includes 2025-2026 without breaking the existing intervals.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100