kubeflow / kubeflow/examples

KF v1.6.0-rc.1 - MNIST E2E on Kubeflow on Vanilla k8s - TypeError: write() argument must be str, not <class 'bytes'>

Open
#993 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jsonnet
Stars
1.5k
Forks
752
PR merge metrics
No merged PRs in 30d

Description

Hello,

Testing KF v1.6.0-rc.1 Mnist E2E on vanilla K8s, I get an error executing tfjob_client.wait_for_job.

> from kubeflow.tfjob import TFJobClient
> tfjob_client = TFJobClient()
> tfjob_client.wait_for_job(train_name, namespace=namespace, watch=True)

Using KF v.1.5, it was successful but using v1.6.0-rc.1 an exception is raised.

> TypeError: write() argument must be str, not

Replacing previous iostream file version, the execution is successful.
/opt/conda/lib/python3.8/site-packages/ipykernel/iostream.py

Comparing the previous [version ](https://github.com/ipython/ipykernel/blob/91550b37f78cb2db6590f591b3eeacd653fba9b4/ipykernel/iostream.py) with the latest [version](https://github.com/ipython/ipykernel/blob/724753a185b0954f0e662c226b86dc8146c62bcb/ipykernel/iostream.py)

The following code seems to cause the issue.

> if not isinstance(string, str):
> raise TypeError(f"write() argument must be str, not {type(string)}")

I don't know how to fix the issue but my current workaround is to replace the iostream.py file with the previous version.

Thank you

Contributor guide

Open the contributing guide

Research direction

Reproduce the MNIST E2E failure at tfjob_client.wait_for_job(..., watch=True) on Kubeflow v1.6.0-rc.1. Compare /opt/conda/lib/python3.8/site-packages/ipykernel/iostream.py with the linked previous and latest versions, then verify that the TypeError no longer occurs without replacing the file manually.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter, kubernetes, python
Domain
infrastructure, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.