JuliaAI / JuliaAI/ScientificTypes.jl

Suggestion to simplify implemenation of scitype

Open
#155 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
98
Forks
7
PR merge metrics
No merged PRs in 30d

Description

I have often lamented the fact that scitype cannot be a map of machine type to type, instead of object to type, because of the infamous CategoricalValue fly in the ointment. As a workaround to performance problems with arrays, we introduced Scitype, which is a map from type to type. Wouldn't if be simpler if implementing Scitype is the "fallback" responsibility of a convention, and that we only overload scitype for problematic cases like CategoricalValue? What am I missing here?

So, something like (ignoring convention distinctions):

# fallback:
scitype(X) = Scitype(typeof(X))

Scitype(::Type) = Unknown
Scitype(::Type{<:Integer}) = Count
# and so forth

# exceptions:
function scitype(X::CategoricalValue)
    N = length(pool(X))
    if isordered(X)
       return OrderedFactor{N}
    end
    return Multiclass{N}
end

To be clear, I'm not suggesting a change in the definition of scitype, only how it is implemented, although Scitype is something we may want to make part of the public interface.

What got me thinking about this is the case of parametric types like Sampleable{S} and a type I'd like to introduce, called Iterator{S} for lazy loaded data structures. Here S is the scitype of the objects sampled, or the scitype of the objects iterated, respectively. How to we implement scitypes for these? This is tricky because we may not have an object from which to extract the parameter S, only its machine type. So in this case we are limited to using Scitype.

Thoughts @OkonSamuel @tlienart

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files or tests. Start by locating the implementations of scitype and Scitype, then inspect how CategoricalValue and parametric cases such as Sampleable are handled. Done would require an agreed design for fallback dispatch, exceptions, and whether Scitype becomes part of the public interface.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.