CentreForDigitalHumanities / CentreForDigitalHumanities/Textcavator

Show user which languages have support

Open
#2,000 1 comment 0 reactions 0 assignees View on GitHub
corpus-form
Dominant language
Python
Stars
12
Forks
3
Avg merge
3d 3m
Merged PRs (30d)
9

Description

Currently, when uploading their own corpus, users can select any language that has an iso code, but not all those languages have stemming or stopword functionality in ElasticSearch. We should make it possible for users to see which languages have these functionalities and which don't. Potentially we can even allow them to upload their own stopword lists? Should not be too difficult to implement, as we are already using local stopword lists for several languages.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the corpus upload language selection and reviewing the existing local stopword lists and Elasticsearch language support. Define which selected languages provide stemming or stopwords, then make that support visible during upload; treat user-provided stopword lists as a separate scope decision.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, python
Domain
backend, data, search
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.