Remove Numpy1DArray and NumpyNDArray type distinction
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 441
- Forks
- 90
- Avg merge
- 1d 12m
- Merged PRs (30d)
- 5
Description
Is your feature request related to a problem? Please describe.
The csp.typing Numpy1DArray and NumpyNDArray types are confusing for users and lead to a lot of unexpected typing issues. We should be consistent with Numpy and use the base np.ndarray type for all Numpy time-series objects. With the distinction, you get strange bugs like this:
inp_arr = np.zeros(shape=(4,))
@csp.graph
def arr_1d(x: csp.ts[csp.typing.Numpy1DArray[float]]) -> csp.ts[np.ndarray]:
return x
res = csp.run(arr_1d(arr_1d(csp.const(inp_arr))), starttime=st, endtime=datetime(2020, 2, 7, 9, 1)) # should not raise
This raises an error:
csp.impl.types.instantiation_type_resolver.TSArgTypeMismatchError: In function accept_arr_1d: Expected ts[csp.typing.Numpy1DArray[float]] for argument 'x', got ts[csp.typing.NumpyNDArray[float]]
Describe the solution you'd like
For now, set the Numpy1DArray type to be equal to the NDArray type and make associated changes to the parquet reader/writer such that users need to opt-in to using shape information. By default we will treat the Numpy array as 1-dimensional.
Describe alternatives you've considered
Keeping the two types as separate will continue to lead to weird bugs as I showed above.
Additional context
This change has come up as a discussion topic before as the type distinction led to bugs in the past.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the csp.typing Numpy1DArray and NumpyNDArray definitions, then trace csp.impl.types.instantiation_type_resolver with the reproduction in the issue. Inspect the parquet reader and writer paths for their shape handling. Done means the type distinction no longer causes the shown mismatch and shape information is opt-in as described.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- data, stream-processing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100