iotile / iotile/typedargs

Restructure types to be normal classes without needing dynamic imports

Open
#70 0 comments 0 reactions 0 assignees View on GitHub
Epic
Dominant language
Python
Stars
1
Forks
2
PR merge metrics
No merged PRs in 30d

Description

## Introduction

### Background

`typedargs` started in the python 2 days where function annotations weren't a thing and so there wasn't a good way to represent an argument, return type as a normal python class.

For that good reason and other bad reasons that haven't stood the test of time, types inside `typedargs` are stored as python modules. This fundamental issues makes a lot of things more complicated than they should be and needs to be remedied.

Since we support python 3.5+ only now, we have the ability to fix this in a backwards compatible way.

### Overall Scheme

There are 4 things that types are supposed to be able to do in `typedargs`, only one of which is required and 3 are optional:

1. Convert a value of the type to a string.
2. (optional) Support converting a string to an instance of the type
3. (optional) Support validating an instance of the type by calling validation functions
4. (optional) Support creating the type directly from binary data

The 4th option was important for legacy use cases where a given function wouldn't have access to the type class that it wanted to create in order to create it from binary data itself. This should go away with the new refactor so there are only 3 things that types need to do.

It's straightforward to define a `Protocol` for a class that supports the three desired activities:

```python
class BaseType(Protocol):
@classmethod
def FromString(cls, str_value: str) -> T:
"""Create an instance of this object from a string."""

def __str__(self):
"""This takes the place of default_formatter."""

def format_xxx(self):
"""Format an instance of this class in a certain way."""

def validate_yyy(self, *args):
"""Validate that an instance of this class conforms to certain expectations."""
```

The overall goal of this epic is to allow any class that conforms to the above protocol act as a typedargs type without needing to preregister it as a type extension.

### Goal State

The goal is to be able to enable the following code:

```python

class MyType:
@classmethod
def FromString(cls, str_val):
return MyType(str_val)

def format_fixed(self):
return "fixed string: " + str(self)

@docannotate
def test_func(arg1: MyType) -> MyType:
"""Basic test function.

Args:
arg1: The input that will be converted to MyType

Returns:
MyType format-as fixed: The converted type.
"""
```

The key points that I want to call out here are:

1. `MyType` was not defined inside of `typedargs` and yet did not need to be registered at all with the typedargs as a type extension.
2. There was just a single simple class for `MyType` that could be imported normally and was a standard python object.
3. If we wanted to include validators, we could do so and they would translate to simple `validate_xxx` calls on the MyType object.

## Technical Details

### Via Function Annotations

If a type is specified (on a parameter or return value) via a function annotation then by definition we have the class already that we need so we can just call `cls.FromString` when we need to convert a string parameter into an instance of the class. Similarly if we have an instance of the class, we can call `instance.format_xxx` or `str(instance)` as needed to convert it to a string.

Validation is similarly easy.

### Via a Docstring Annotation

In this case, we will need to use the same lookup system we use now to map the string name of the type to a class that matches `BaseType` and then do the same operations using that class.

### Handling Legacy Type Modules

We can define a generic `BaseType` subclass that we can add validator and formatter functions to dynamically from a module. Then when we import a type module, we just create the corresponding type class for it and then proceed to use that.

### Augmenting `builtin` Types

There is one more situation we need to be able to handle. When we encounter builtin types like `str` and `int`, we want to augment them with more formatters and validators than the original class supports. This means we need to keep track of which types should have a "shadow" type that is used instead of the actual type class passed in via the function annotation or found via the docstring annotation.

For example, we could have something like:

```python
shadow_types = {str: InternalStringClass, dict: InternalDictTypeFactory}

def find_type_class(type_):
if isinstance(type_, str):
type_ = lookup_type_by_name(type_)

# Allow us to maintain an internal augmented version of certain basic types
if type_ in shadow_types:
return shadow_types[type_]

return type_
```

@dvksirin This is what I was mentioning in the PR review about the direction that we want to go with typedargs. The goal is to remove the type modules and extension type system eventually and replace it with just simple classes.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.