tensorflow / tensorflow/tensorboard

add_images will deteriorate image quality

Open
#6,209 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
7.2k
Forks
1.7k
Avg merge
4d 22h
Merged PRs (30d)
1

Description

Issue description

import math
import numpy as np
from matplotlib import pyplot as plt
from torch import Tensor
from torchvision.transforms.functional import pad

from compressai.zoo import image_models
from torch.utils.tensorboard.writer import SummaryWriter
from torchvision.transforms import ToTensor
from PIL import Image

def patch_images(
    samples: Tensor, ratio: int = 1, S: int = 64
) -> tuple[Tensor, list[int]]:
    h, w = samples.shape[-2:]
    h_pad = S * math.ceil(h / S) - h
    w_pad = S * math.ceil(w / S) - w
    l_pad, u_pad = w_pad // 2, h_pad // 2
    r_pad, d_pad = w_pad - l_pad, h_pad - u_pad
    pad_param = [l_pad, u_pad, r_pad, d_pad]
    samples = pad(samples, pad_param)
    crop_param = [ratio * u_pad, ratio * l_pad, ratio * h, ratio * w]
    return samples, crop_param


def convert_images(inputs) -> np.ndarray:
    if isinstance(inputs, np.ndarray):
        img = inputs
    else:
        from torchvision.utils import make_grid

        img = (
            make_grid(inputs.clip(0, 1))
            .permute(1, 2, 0)
            .detach()
            .cpu()
            .float()
            .numpy()
        )
    return img

def plot(inputs: "Tensor | np.ndarray") -> None:
    r"""Plot tensors or array. Clip to $[0, 1]$ by default.

    :param inputs:
    :type inputs: Tensor | np.ndarray
    :rtype: None
    """
    img = convert_images(inputs)
    plt.imshow(img)
    plt.show()

model = image_models["mbt2018"](8, pretrained=True)
img = Image.open("a.png")
images = ToTensor()(img).unsqueeze(0)[:, :3]
padded, params = patch_images(images)
outputs = model(padded)["x_hat"]
plot(outputs)

with SummaryWriter("/output/logs") as writer:
    writer.add_images("a", outputs)

In matplotlib, a.png is displayed normally:

screen-2023-02-25-00-18-06

In tensorbaord, a.png is displayed weirdly:

screen-2023-02-25-00-18-50

a.png is here:

a.png.zip

Environment information (required)

Please run diagnose_tensorboard.py (link below) in the same
environment from which you normally run TensorFlow/TensorBoard, and
paste the output here:

Diagnostics
Diagnostics output 
--- check: autoidentify
INFO: diagnose_tensorboard.py version 516a2f9433ba4f9c3a4fdb0f89735870eda054a1

--- check: general
INFO: sys.version_info: sys.version_info(major=3, minor=10, micro=9, releaselevel='final', serial=0)
INFO: os.name: posix
INFO: os.uname(): posix.uname_result(sysname='Linux', nodename='desktop', release='6.1.12-arch1-1', version='#1 SMP PREEMPT_DYNAMIC Tue, 14 Feb 2023 22:08:08 +0000', machine='x86_64')
INFO: sys.getwindowsversion(): N/A

--- check: package_management
INFO: has conda-meta: False
INFO: $VIRTUAL_ENV: None

--- check: installed_packages
INFO: installed: tensorboard==2.11.0
WARNING: no installation among: ['tensorflow', 'tensorflow-gpu', 'tf-nightly', 'tf-nightly-2.0-preview', 'tf-nightly-gpu', 'tf-nightly-gpu-2.0-preview']
WARNING: no installation among: ['tensorflow-estimator', 'tensorflow-estimator-2.0-preview', 'tf-estimator-nightly']
INFO: installed: tensorboard-data-server==0.7.0a0

--- check: tensorboard_python_version
INFO: tensorboard.version.VERSION: '2.11.0'

--- check: tensorflow_python_version
Traceback (most recent call last):
  File "/dev/shm/./diagnose_tensorboard.py", line 528, in main
    suggestions.extend(check())
  File "/dev/shm/./diagnose_tensorboard.py", line 81, in wrapper
    result = fn()
  File "/dev/shm/./diagnose_tensorboard.py", line 284, in tensorflow_python_version
    import tensorflow as tf
ModuleNotFoundError: No module named 'tensorflow'

--- check: tensorboard_data_server_version
INFO: data server binary: '/usr/lib/python3.10/site-packages/tensorboard_data_server/bin/server'
INFO: data server binary version: b'rustboard 0.7.0-alpha.0'

--- check: tensorboard_binary_path
INFO: which tensorboard: b'/usr/bin/tensorboard\n'

--- check: addrinfos
socket.has_ipv6 = True
socket.AF_UNSPEC = <AddressFamily.AF_UNSPEC: 0>
socket.SOCK_STREAM = <SocketKind.SOCK_STREAM: 1>
socket.AI_ADDRCONFIG = <AddressInfo.AI_ADDRCONFIG: 32>
socket.AI_PASSIVE = <AddressInfo.AI_PASSIVE: 1>
Loopback flags: <AddressInfo.AI_ADDRCONFIG: 32>
Loopback infos: [(<AddressFamily.AF_INET6: 10>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('::1', 0, 0, 0)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 0))]
Wildcard flags: <AddressInfo.AI_PASSIVE: 1>
Wildcard infos: [(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('0.0.0.0', 0)), (<AddressFamily.AF_INET6: 10>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('::', 0, 0, 0))]

--- check: readable_fqdn
INFO: socket.getfqdn(): 'localhost'

--- check: stat_tensorboardinfo
INFO: directory: /tmp/.tensorboard-info
INFO: os.stat(...): os.stat_result(st_mode=16895, st_ino=11056, st_dev=32, st_nlink=2, st_uid=1000, st_gid=1000, st_size=60, st_atime=1677246960, st_mtime=1677253463, st_ctime=1677253463)
INFO: mode: 0o40777

--- check: source_trees_without_genfiles
INFO: tensorboard_roots (1): ['/usr/lib/python3.10/site-packages']; bad_roots (0): []

--- check: full_pip_freeze
INFO: pip freeze --all:
absl-py==1.3.0
agate==1.6.3
agate-dbf==0.2.2
agate-excel==0.2.5
agate-sql==0.5.8
aiohttp==3.8.3
aiosignal==1.3.1
alabaster==0.7.13
albumentations==1.3.0
ansi2html==1.8.0
ansicolor==0.3.2
anyio==3.6.2
appdirs==1.4.4
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
argopt==0.7.1
aria2p==0.11.3
arrow==1.2.3
asciimatics==1.14.0
asciinema==2.2.0
asgiref==3.5.2
asttokens==2.2.1
async-generator==1.10
async-timeout==4.0.2
attrs==22.2.0
autocommand==2.2.2
Babel==2.11.0
backcall==0.2.0
beautifulsoup4==4.11.2
black==23.1.0
bleach==6.0.0
blessed==1.20.0
boolean.py==4.0
Brotli==1.0.9
btrfsutil==6.1.3
build==0.10.0
CacheControl==0.12.11
cachetools==5.3.0
cachy==0.3.0
cattrs==22.2.0
Cerberus==1.3.4
certifi==2022.12.7
cffi==1.15.1
cfgv==3.3.1
chardet==5.1.0
charset-normalizer==2.1.1
cleo==2.0.1
click==8.1.3
cmake-annotate==0.6.13
cmake-format==0.6.13
cmake-lint==0.6.13
cmake-parse==0.6.13
cmakelang==0.6.13
cmakelint==1.4.2
codecov==2.1.12
colorama==0.4.6
comm==0.1.2
compressai==1.2.1
contourpy==1.0.7
coverage==6.5.0
crashtest==0.4.1
cryptography==39.0.1
cssselect==1.2.0
csvkit==1.0.7
cupshelpers==1.0
cupy==11.5.0
cycler==0.11.0
Cython==0.29.33
datasets==1.17.0
dbfread==2.0.7
debugpy==1.6.6
decorator==5.1.1
defusedxml==0.7.1
deprecation==2.1.0
dill==0.3.6
distlib==0.3.6
Django==4.1.7
dnspython==2.2.1
docker-pycreds==0.4.0
docopt==0.6.2
docopts==0.6.1+fix
docstring-to-markdown==0.11
docutils==0.19
dominate==2.7.0
doq==0.9.1
dulwich==0.21.3
dunamai==1.15.0
einops==0.6.0
elementpath==3.0.2
emoji-fzf==0.7.0
entrypoints==0.4
et-xmlfile==1.1.0
eventlet==0.33.2
exceptiongroup==1.1.0
execnet==1.9.0
executing==1.2.0
fastjsonschema==2.16.2
fastrlock==0.8.1
filelock==3.8.2
flexmock==0.11.3
fonttools==4.38.0
frozenlist==1.3.3
fsspec==2022.11.0
future==0.18.2
fuzzywuzzy==0.18.0
gdown==4.6.3
gevent==22.10.2
gitdb==4.0.9
gitlint-core==0.17.0
GitPython==3.1.29
gmpy2==2.1.5
google-auth==2.14.1
google-auth-oauthlib==0.7.1
gpustat==1.0.0
greenlet==2.0.2
grpcio==1.51.1
gunicorn==20.1.0
h5py==3.7.0
help2man==0.0.1
html5lib==1.1
huggingface-hub==0.12.1
HyFetch==1.4.7
hypothesis==6.59.0
identify==2.5.18
idna==3.4
imageio==2.25.0
imagesize==1.4.1
imgaug==0.4.0
importlib-metadata==5.0.0
inflect==6.0.2
iniconfig==2.0.0
installer==0.6.0
iopath==0.1.9
ipykernel==6.21.2
ipython==8.10.0
ipython-genutils==0.2.0
isodate==0.6.1
isort==5.12.0
jaraco.classes==3.2.3
jaraco.context==4.3.0
jaraco.functools==3.5.2
jaraco.text==3.11.1
jarowinkler==1.2.3
jedi==0.18.2
jedi-language-server==0.40.0
jeepney==0.8.0
Jinja2==3.1.2
joblib==1.2.0
-e git+https://github.com/Freed-Wu/jpegai@d7d0c0e8e00f51c15807047b5343d7e8549247e2#egg=jpegai
json5==0.9.12
jsonpatch==1.32
jsonpointer==2.3
jsonschema==4.17.3
jupyter-events==0.6.3
jupyter_client==8.0.3
jupyter_core==5.2.0
jupyter_packaging==0.12.3
jupyter_server==2.3.0
jupyterlab==3.6.1
jupyterlab-pygments==0.2.2
jupyterlab_server==2.19.0
kaggle==1.5.12
keyring==23.11.0
keyring-pass==0.7.1
kiwisolver==1.4.4
langdetect==1.0.9
lark==1.1.5
latexify-py==0.2.0
lazy_loader==0.1
leather==0.3.4
lensfun==0.3.3
Levenshtein==0.20.9
license-expression==30.1.1.dev0+g1548127.d20230117
lit==15.0.7.dev0
lmdb==1.4.0
lockfile==0.12.2
loguru==0.6.0
lpips==0.1.4
lsprotocol==2022.0.0a10
lxml==4.9.2
Markdown==3.4.1
markdown-it-py==2.1.0
MarkupSafe==2.1.2
matplotlib==3.6.2
matplotlib-inline==0.1.6
mdformat==0.7.16
mdit-py-plugins==0.3.1
mdurl==0.1.2
mistune==2.0.5
mock==3.0.5
monotonic==1.6
more-itertools==9.0.0
mpmath==1.2.1
msgpack==1.0.4
multidict==6.0.3
multiprocess==0.70.14
munch==2.5.0
mypy-extensions==1.0.0
myst-parser==0.18.1
namcap==3.3.1
nbclient==0.7.2
nbconvert==7.2.9
nbformat==5.7.3
nest-asyncio==1.5.6
netease-musicbox==0.3.1
netsnmp-python==1.0a1
networkx==2.8.8
nodeenv==1.7.0
nose==1.3.7
notebook_shim==0.2.2
numpy==1.24.2
nvidia-ml-py==11.525.84
oauthlib==3.2.2
olefile==0.46
OpenCC==1.1.5
opencv-python==4.7.0.68
openpyxl==3.1.0
opt-einsum==3.3.0
ordered-set==4.1.0
packaging==23.0
pandas==1.5.3
pandocfilters==1.5.0
parsedatetime==2.6
parso==0.8.3
pathspec==0.11.0
pathtools==0.1.2
pbr==5.11.1
pdm-pep517==1.1.2
pexpect==4.8.0
pickleshare==0.7.5
Pillow==9.4.0
pip==23.0.1
pip-cache==0.0.6
pip-tools==6.12.2
pix2tex==0.0.29
pkginfo==1.9.6
platformdirs==2.6.2
plotext==5.2.8
pluggy==1.0.0
ply==3.11
poetry==1.3.2
poetry-core==1.5.0
poetry-plugin-export==1.2.0
pooch==1.6.0
portalocker==2.7.0
powerline-status==2.8.3
pre-commit==2.20.0
prometheus-client==0.15.0
promise==2.3
prompt-toolkit==3.0.36
protobuf==4.21.12
psutil==5.9.4
ptpython==3.0.21
ptyprocess==0.7.0
pudb==2022.1.3
pure-eval==0.2.2
py-notifier==0.3.2
pyalpm==0.10.6
pyaml==21.10.1
pyarrow==10.0.1
pyasn1==0.4.8
pyasn1-modules==0.2.8
pybind11==2.10.3
pycairo==1.23.0
pycparser==2.21
pycryptodomex==3.12.0
pycups==2.0.1
pycurl==7.45.2
pydantic==1.10.4
pydocstyle==6.1.1
pyelftools==0.29
pyfiglet==0.8.post1
pygls==1.0.1
Pygments==2.14.0
PyGObject==3.42.2
PyICU==2.10.2
pynvim==0.4.3
pynvml==11.4.1
PyOpenGL==3.1.6
pyOpenSSL==23.0.0
pyparsing==3.0.9
pyperclip==1.8.2
pyproject_hooks==1.0.0
PyQt3D==5.15.6
PyQt5==5.15.9
PyQt5-sip==12.11.1
PyQtChart==5.15.6
PyQtDataVisualization==5.15.5
PyQtNetworkAuth==5.15.5
PyQtPurchasing==5.15.5
PyQtWebEngine==5.15.6
pyrsistent==0.19.3
PySocks==1.7.1
PyStarDict @ file:///home/wzy/.cache/yay/python-pystardict/src/PyStarDict-0.8-py2.py3-none-any.whl
pytest==7.2.1
pytest-cov==4.0.0
pytest-pudb==0.7.0
pytest-xdist==3.2.0
python-dateutil==2.8.2
python-json-logger==2.0.5
python-slugify==8.0.0
python-wakatime==0.0.3
pytimeparse==1.1.8
pytorch-msssim==0.2.1
pytz==2022.7
PyWavelets==1.4.1
pyxdg==0.28
PyYAML==6.0
pyzmq==24.0.1
qudida==0.0.4
rapidfuzz==2.13.7
readme-renderer==37.3
regex==2022.10.31
repl-python-codestats==0.0.3
repl-python-wakatime==0.0.5
requests==2.28.1
requests-cache==0.9.6
requests-oauthlib==1.3.1
requests-toolbelt==0.10.1
requests-unixsocket==0.3.0
rfc3986==2.0.0
rich==13.3.1
rsa==4.9
sacremoses==0.0.35
SciencePlots==2.0.1
scikit-image==0.20.0rc6
scikit-learn==1.2.0
scipy==1.10.1
sebaubuntu-libs==1.3.1
SecretStorage==3.3.3
Send2Trash==1.8.0
sentry-sdk==1.11.0
setproctitle==1.3.2
setuptools==67.3.0
setuptools-scm==7.1.0
sh==1.14.3
Shapely==1.8.5.post1
shellingham==1.5.0.post1
shortuuid==1.0.11
shtab==1.5.8
six==1.16.0
smmap==5.0.0
sniffio==1.3.0
snowballstemmer==2.2.0
sortedcontainers==2.4.0
soupsieve==2.4
Sphinx==5.3.0
sphinx-rtd-theme==1.1.1
sphinxcontrib-applehelp==1.0.4
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-eval==0.0.3
sphinxcontrib-htmlhelp==2.0.1
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.5
SQLAlchemy==1.4.44
sqlparse==0.4.3
stack-data==0.6.2
super-image @ https://test-files.pythonhosted.org/packages/42/c4/8b9cca01f45246017791440e70cef1ca52614daae317faaa49295d233c24/super_image-0.1.7-py3-none-any.whl
sympy==1.11.1
tabulate==0.9.0
TBB==0.2
team==1.0
tensorboard==2.11.0
tensorboard-data-server==0.7.0a0
tensorboard-plugin-wit==1.8.1
tensorboardX==2.5.1
termcolor==2.2.0
terminado==0.17.1
testpath==0.6.0
text-unidecode==1.3
threadpoolctl==3.1.0
tifffile==2023.2.3
timm==0.6.12
tinycss2==1.2.1
tokenizers==0.13.2
toml==0.10.2
tomli==2.0.1
tomli_w==1.0.0
tomlkit==0.11.6
torch==1.13.1
torch-fidelity==0.3.0
-e git+https://github.com/Harry24k/adversarial-attacks-pytorch@320d15dd7b2234b25a389f0fee6ef23b02c46fb7#egg=torchattacks
torchdata==0.5.1
torchfile==0.1.0
torchinfo==1.7.1
torchmetrics==0.11.1
torchvision==0.14.1a0
tornado==6.2
tqdm==4.64.1
traitlets==5.9.0
transformers==4.26.1
translate-shell==0.0.11
trash-cli==0.22.10.20
trove-classifiers==2023.2.9
twine==4.0.2
typeguard==2.13.3
typing_extensions==4.5.0
uc-micro-py==1.0.1
url-normalize==1.4.3
urllib3==1.26.12
urwid==2.1.2
urwid-readline==0.13
uvloop==0.17.0
validate-pyproject==0.12.1
validators==0.20.0
vim-vint==0.4a4.dev14+ge120918
vimdoc==0.6.0
virtualenv==20.17.1
visdom==0.2.4
visidata==2.11
wandb==0.13.5
wcwidth==0.2.5
webencodings==0.5.1
websocket-client==1.5.1
Werkzeug==2.2.3
wheel==0.38.4
wordcloud==1.8.1
wxPython==4.2.0
x-transformers==1.2.3
xlrd==2.0.1
xmlschema==2.2.1
xxhash==3.2.0
yamllint==1.29.0
yapf==0.32.0
yarl==1.8.2
you-get==0.4.1650
youtube-dl==2021.12.17
zipp==3.14.0
zope.event==4.6
zope.interface==5.5.2
Next steps

No action items identified. Please copy ALL of the above output,
including the lines containing only backticks, into your GitHub issue
or comment. Be sure to redact any sensitive information.

For browser-related issues, please additionally specify:

  • Browser type and version (e.g., Chrome 64.0.3282.140):
    firefox 110.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the provided example with TensorBoard 2.11.0, the mbt2018 model output, and a.png; compare matplotlib's plot with SummaryWriter.add_images in TensorBoard. Trace the add_images logging and rendering path to determine where image quality changes. Done means TensorBoard preserves the expected image appearance.

Written by the indexing model from the issue text.

Assessment

Tech stack
matplotlib, numpy, python, pytorch
Domain
data-visualization
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.