CentreForDigitalHumanities / CentreForDigitalHumanities/Textcavator

Corpus form: limit available field types

Open
#1,788 0 comments 0 reactions 0 assignees View on GitHub
corpus-form enhancement frontend
Dominant language
Python
Stars
12
Forks
3
Avg merge
3d 3m
Merged PRs (30d)
9

Description

**Is your feature request related to a problem? Please describe.**
Currently, the corpus form will preselect the data type for each field based on the CSV data. However, this type may not be appropriate, so the user is able to select a different type if needed. A typical example would be a field containing numeric IDs that is preselected as *number (integer)*, but should be *text (metadata)* (i.e. it should not be parsed as a quantity).

However, right now, this selection is too permissive, allowing users to select any data type for any column. In reality, the valid choices for a column are more limited, and choosing invalid types will result in errors during indexing.

**Describe the solution you'd like**

For each column, there should be a list of possible data types. For instance, a column containing only numbers could be *number (integer)*, *number (decimal)*, *text (metadata)* or *text (content)*. (Though the last option is probably not sensible.)

(You can get this list from the pre-selected type. Anything not pre-selected as text can also be text, and integers can also be floats.)

The form should only let the user pick one these values for this field; other options should be hidden or disabled.

**Describe alternatives you've considered**

Alternative: leave things as is, let the user figure out what types are valid.

This works in a lot of cases. In the example of an numeric column, a user would probably not select the *boolean* or *date* option anyway.

However, say that column contains incorrectly formatted dates. This would be pre-selected as *text (metadata)* (since I-analyzer does not recognise the values as dates), which the user may "correct" to the *date* type, resulting in an error down the line. In that situation, it would be better if the field editor already showed that the values in the column are not valid dates, prompting the users to correct their data.

**Additional context**

Screenshot:

![screenshot of corpus form in I-analyzer; shows dropdown for different field types](https://github.com/user-attachments/assets/63eb0759-97dc-43ea-bf5c-175f82540ab2)

Contributor guide

No contributing guide indexed for this repository

Research direction

No files, tests, or entry points are named. Start by locating the corpus form and its field-type editor, then trace how the preselected type is determined and how choices are presented. Done means each column exposes only valid types and invalid date or other values cannot be selected without first correcting the data.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, frontend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.