KF v1.6.0-rc.1 - MNIST E2E on Kubeflow on Vanilla k8s - TypeError: write() argument must be str, not <class 'bytes'>
- Dominant language
- Jsonnet
- Stars
- 1.5k
- Forks
- 752
- PR merge metrics
- No merged PRs in 30d
Description
Hello,
Testing KF v1.6.0-rc.1 Mnist E2E on vanilla K8s, I get an error executing tfjob_client.wait_for_job.
> from kubeflow.tfjob import TFJobClient
> tfjob_client = TFJobClient()
> tfjob_client.wait_for_job(train_name, namespace=namespace, watch=True)
Using KF v.1.5, it was successful but using v1.6.0-rc.1 an exception is raised.
> TypeError: write() argument must be str, not
Replacing previous iostream file version, the execution is successful.
/opt/conda/lib/python3.8/site-packages/ipykernel/iostream.py
Comparing the previous [version ](https://github.com/ipython/ipykernel/blob/91550b37f78cb2db6590f591b3eeacd653fba9b4/ipykernel/iostream.py) with the latest [version](https://github.com/ipython/ipykernel/blob/724753a185b0954f0e662c226b86dc8146c62bcb/ipykernel/iostream.py)
The following code seems to cause the issue.
> if not isinstance(string, str):
> raise TypeError(f"write() argument must be str, not {type(string)}")
I don't know how to fix the issue but my current workaround is to replace the iostream.py file with the previous version.
Thank you
Contributor guide
Research direction
Reproduce the MNIST E2E failure at tfjob_client.wait_for_job(..., watch=True) on Kubeflow v1.6.0-rc.1. Compare /opt/conda/lib/python3.8/site-packages/ipykernel/iostream.py with the linked previous and latest versions, then verify that the TypeError no longer occurs without replacing the file manually.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter, kubernetes, python
- Domain
- infrastructure, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100