apache / apache/hamilton

Engine crashes when client sends nan or inf float to UI

Open
#1,458 5 comments 0 reactions 0 assignees View on GitHub
bug SDK
Dominant language
Jupyter Notebook
Stars
2.6k
Forks
213
PR merge metrics
No merged PRs in 30d

Description

The requests JSON serializer defaults to not handling nan or inf floats and instead throws an exception that crashes the engine. The tracking/utils:make_json_safe function does not make objects safe enough to avoid this and doesn't detect it since the json.dumps check doesults to allowing nans.

# Current behavior

Engine crashes if a node returns nan or inf floats.

## Stack Traces

```
Traceback (most recent call last):
File "/opt/homebrew/Cellar/python@3.13/3.13.11_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/threading.py", line 1044, in _bootstrap_inner
self.run()
~~~~~~~~^^
File "/opt/homebrew/Cellar/python@3.13/3.13.11_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/threading.py", line 995, in run
self._target(*self._args, **self._kwargs)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mfornander/gitlab/dcsim/.venv/lib/python3.13/site-packages/hamilton_sdk/api/clients.py", line 236, in worker
self.flush(batch)
~~~~~~~~~~^^^^^^^
File "/Users/mfornander/gitlab/dcsim/.venv/lib/python3.13/site-packages/hamilton_sdk/api/clients.py", line 262, in flush
response = requests.put(
f"{self.base_url}/dag_runs_bulk?dag_run_id={dag_run_id}",
...<5 lines>...
verify=self.verify,
)
File "/Users/mfornander/gitlab/dcsim/.venv/lib/python3.13/site-packages/requests/api.py", line 130, in put
return request("put", url, data=data, **kwargs)
File "/Users/mfornander/gitlab/dcsim/.venv/lib/python3.13/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mfornander/gitlab/dcsim/.venv/lib/python3.13/site-packages/requests/sessions.py", line 575, in request
prep = self.prepare_request(req)
File "/Users/mfornander/gitlab/dcsim/.venv/lib/python3.13/site-packages/requests/sessions.py", line 484, in prepare_request
p.prepare(
~~~~~~~~~^
method=request.method.upper(),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<10 lines>...
hooks=merge_hooks(request.hooks, self.hooks),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/Users/mfornander/gitlab/dcsim/.venv/lib/python3.13/site-packages/requests/models.py", line 370, in prepare
self.prepare_body(data, files, json)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
File "/Users/mfornander/gitlab/dcsim/.venv/lib/python3.13/site-packages/requests/models.py", line 512, in prepare_body
raise InvalidJSONError(ve, request=self)
requests.exceptions.InvalidJSONError: Out of range float values are not JSON compliant
```

## Screenshots

n/a

## Steps to replicate behavior

Add this simple node, enable on tracking on a working UI server, and ask for the node's result.

```python
def nan_inf() -> list[float]:
return [1.0, float("nan"), float("inf")]
```

## Library & System Information

Python 3.13
Hamilton 1.89

# Expected behavior

Not crashing and showing `nan` and `inf` values as `"null"` in UI.

# Additional context

I have a simple fix for your make_json_safe function. Let me know if you want a PR.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.