MetOffice / MetOffice/XBTs_classification
Try embedded encoding for cruise, platform features
- Dominant language
- Jupyter Notebook
- Stars
- 4
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
The standard way of encoding a categorical feature (with no natural ordering) is to use one hot encoding. One issue with this is when you have many different possible values, you need to have one column/feature per possible value. For cruise, there are thousands or tens of thousands of different cruise ID, and hundred of platforms and institutes. This makes one hot encoding impractical. An alternative is embededd encoding, when the category is represented as a vector.
https://towardsdatascience.com/deep-embeddings-for-categorical-variables-cat2vec-b05c8ab63ac0
https://towardsdatascience.com/categorical-embedding-and-transfer-learning-dd3c4af6345d
This requires there being some measure of which values are "close" to one another. There may not be any sensible way of evaluating this for the XBT data, so we might not be able to use this approach.
Contributor guide
No contributing guide indexed for this repository
Research direction
No file, test, or entry point is named. Start by reviewing how categorical cruise IDs, platforms, and institutes are currently represented in the project’s Jupyter notebooks, then determine whether a meaningful similarity measure exists for the XBT data. Done requires a decided embedding approach and evidence that it is practical.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook, machine-learning
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100