MetOffice / MetOffice/XBTs_classification

Try embedded encoding for cruise, platform features

Open
#28 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
4
Forks
2
PR merge metrics
No merged PRs in 30d

Description

The standard way of encoding a categorical feature (with no natural ordering) is to use one hot encoding. One issue with this is when you have many different possible values, you need to have one column/feature per possible value. For cruise, there are thousands or tens of thousands of different cruise ID, and hundred of platforms and institutes. This makes one hot encoding impractical. An alternative is embededd encoding, when the category is represented as a vector.
https://towardsdatascience.com/deep-embeddings-for-categorical-variables-cat2vec-b05c8ab63ac0
https://towardsdatascience.com/categorical-embedding-and-transfer-learning-dd3c4af6345d

This requires there being some measure of which values are "close" to one another. There may not be any sensible way of evaluating this for the XBT data, so we might not be able to use this approach.

Contributor guide

No contributing guide indexed for this repository

Research direction

No file, test, or entry point is named. Start by reviewing how categorical cruise IDs, platforms, and institutes are currently represented in the project’s Jupyter notebooks, then determine whether a meaningful similarity measure exists for the XBT data. Done requires a decided embedding approach and evidence that it is practical.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, machine-learning
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.