apache / apache/hamilton

Inputs type annotation mismatch

Open
#1,180 0 comments 0 reactions 0 assignees View on GitHub
triage
Dominant language
Jupyter Notebook
Stars
2.6k
Forks
213
PR merge metrics
No merged PRs in 30d

Description

This is a scenario that happened to a user and we could surface better errors

# Current behavior
Take a look at the type annotations for `external_input`. The current dataflow definition is valid because the types are deemed equivalent (`Any` is special, it is both a subclass and super class of all types, including `MyClass`)

```python
from typing import Any

class MyClass:
def __init__(self):
...

def foo(external_input: MyClass) -> int:
return 1

def bar(external_input: Any) -> int:
return 2

def join_nodes(foo: int, bar: int) -> int:
return foo + bar
```

The problem happens when calling `Driver.execute(...)` and specifying `external_input`. If the user expects it to take `Any` and pass a value that's not `MyClass`, the input validation will say that there's a type mismatch without more details.

```python
inputs = {"external_input": 31}
dr.execute(["join_nodes"], inputs=inputs)
```

Additionally, because of the way visualizations are rendered, the two `external_input` will appear with the same type despite being annotated differently.

Also, the problem is specific to `inputs`. The following DAG wouldn't raise an error despite the object not matching the annotated type because there's no runtime type validation by default:
```python
from typing import Any

class MyClass:
def __init__(self):
...

def internal_input() -> MyClass:
return 31

def foo(internal_input: MyClass) -> int:
return 1

def bar(internal_input: Any) -> int:
return 2

def join_nodes(foo: int, bar: int) -> int:
return foo + bar
```

```python
dr.execute(["join_nodes"])
```

## Potential solutions
- we could have better errors on input type validation
- we could warn that nodes are annotated differently but with valid types at `Builder.build()` time
- display the different type annotations in the visualization (I suggest against because it would be confusing since it can only be "one type at a time")

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.