CentreForDigitalHumanities / CentreForDigitalHumanities/Textcavator

Implement 'core fields' between corpora

Open
#993 2 comments 0 reactions 0 assignees View on GitHub
corpus
Dominant language
Python
Stars
12
Forks
3
Avg merge
3d 3m
Merged PRs (30d)
9

Description

One potential solution to the issue of cross-corpus searching (#992) is to implement a selection of 'core fields' that are defined the same way in all corpora that use them. For example, we could have a common definition for a `date` field. Thus, a collection of corpora can run aggregations on `date` without issue. This avoid the issue that one corpus calls the date `date`, the other calls it `publish_date`, etc.

When adding fields to a corpus definition, curators can be shown the list of 'core fields', which the request to select those that apply, or add fields specific to that corpus.

Some potential fields:

- content
- date
- Perhaps also a separate 'year' field? May be automatically inferred if the corpus includes dates.
- author
- language
- publication (as in, the publication that the document is a part of)

Some other fields like 'page', etc. are very common, but not very interesting for cross-corpus filtering and such.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.