ManimCommunity / ManimCommunity/manim

OpenGLRenderer.get_texture_id sometimes returns wrong tid

Open
#3,368 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

issue:bug opengl
Dominant language
Python
Stars
40.9k
Forks
3.1k
Avg merge
3d 12h
Merged PRs (30d)
25

Description

## Description of bug / unexpected behavior

I have posted this issue on [Discord #helpForum](https://discordapp.com/channels/581738731934056449/1150099499335819295/1150099499335819295) a couple of days ago. I also posted the video showing the poorly rendered video. After analyzing this a bit further, I think it is worth to file an issue for it.

I have created a 3D scene showing a torus with textured surface. For each frame of the animation I replace the torus with a new one having a slightly different texture. The resulting video flickers and the animation seems to frequently jump back and forth.

## Expected behavior

The animation should create a smooth, flicker free video, in this case a winding line around the torus starting at the red dot towards the blue dot.

### Analysis
I have modified `OpenGLRenderer.get_texture_id` with some `print()` statements to see, what is going on.

```py
def get_texture_id(self, path):
if repr(path) not in self.path_to_texture_id:
tid = len(self.path_to_texture_id)
texture = self.context.texture(
size=path.size,
components=len(path.getbands()),
data=path.tobytes(),
)
texture.repeat_x = False
texture.repeat_y = False
texture.filter = (moderngl.NEAREST, moderngl.NEAREST)
texture.swizzle = "RRR1" if path.mode == "L" else "RGBA"
texture.use(location=tid)
self.path_to_texture_id[repr(path)] = tid

texture_id = self.path_to_texture_id[repr(path)]
print("OpenGLRenderer.get_texture_id", repr(path), path.file_name, "->", texture_id)
if self.texture_id > texture_id:
print(f">>> OpenGLRenderer.get_texture_id wants to return {texture_id}, but it should return {self.cnt-1} instead!")
self.texture_id = texture_id
return texture_id
```

I also modified `OpenGLTexturedSurface.get_image_from_file` to make `file_name` available.
```py
def get_image_from_file(
self,
image_name: str | Path,
image_mode: str,
):
image_file = get_full_raster_image_path(image_name)
image = Image.open(image_file).convert(image_mode)
image.file_name = image_name
return image

```
Below you can see the output for rendering frame `#184`. You can see that the key into `self.path_to_texture_id` is somehow flawed. The only discriminator between the images is the reported address of the `Image` object. However an image might be subject for garbage collection, and therefore the object address is probably not a good hash for the image.

For frame `#184` the method should return `tid=184`.
```
#184 OpenGLRenderer.render scene:
OpenGLMobject.get_shader_wrapper OpenGLFrameTexturedSurface
ShaderWrapper texture_path: /home/mk/dev/manim/ecc/gist/assets/MainScene0184.png
OpenGlRenderer.render_mobject mobject: OpenGLFrameTexturedSurface shader_wrapper_list: []
OpenGLRenderer.get_texture_id /home/mk/dev/manim/ecc/gist/assets/MainScene0184.png -> 128
>>> OpenGLRenderer.get_texture_id wants to return 128, but it should return 184 instead!
OpenGLRenderer.get_texture_id /home/mk/dev/manim/ecc/gist/assets/MainScene0184.png -> 154
Mesh.render self.shader:
OpenGLMobject.get_shader_wrapper OpenGLFrameTexturedSurface
ShaderWrapper texture_path: /home/mk/dev/manim/ecc/gist/assets/MainScene0185.png
```

After replacing `repr(path)` with `hash(path.file_name)` the video renders nicely as expected. However I don't think, that this is an appropriate fix.

If an `OpenGLMobject` gets removed (e.g. via `Scene.replace`) from the scene, I assume the textures should also be [released](https://moderngl.readthedocs.io/en/latest/topics/gc.html).

Also `OpenGLRenderer.path_to_texture_id` should not be constantly growing. Some housekeeping might be required.

## How to reproduce the issue

In order to run this code you need images `assets/MainScene0000.png` ... `assets/MainScene0195.png`. Then render the script with low quality, i.e. 15 frames/second, resulting in 13s video output.

Code for reproducing the problem

```py
from manim import *
from manim.opengl import *
import os
from pathlib import Path

config.renderer = "opengl"

def texture(frame):
return (
Path(os.path.realpath(__file__)).parent
/ "assets"
/ ("MainScene%04d.png" % frame)
)

class OpenGLTorus(OpenGLSurface):
def func(self, u, v):
P = np.array([np.cos(u), np.sin(u), 0])
return (self.R - self.r * np.cos(v)) * P - self.r * np.sin(v) * OUT

def __init__(self, **kwargs):
self.R = 4
self.r = 1

super().__init__(
uv_func=self.func,
u_range=(0, TAU),
v_range=(0, TAU),
resolution=(101, 101),
**kwargs,
)

def create_torus_surface(torus, frame):
return OpenGLTexturedSurface(torus, texture(frame)).scale(0.8).set_opacity(0.7)

class AnimateTorusTexture(ThreeDScene):
def construct(self):
config.disable_caching = True

torus = OpenGLTorus()
frame = 0
torus_surface = create_torus_surface(torus, frame)

def replace_torus(dt):
nonlocal torus_surface, frame
new_surface = create_torus_surface(torus, frame)
self.replace(torus_surface, new_surface)
torus_surface = new_surface
frame += 1

trk = ValueTracker()
self.add(torus_surface)
self.add_updater(replace_torus)
self.play(trk.animate.set_value(1), run_time=13)
self.remove_updater(replace_torus)

```

## Additional media files

Images/GIFs

## Logs
Terminal output

```
PASTE HERE OR PROVIDE LINK TO https://pastebin.com/ OR SIMILAR
```

## System specifications

System Details

- OS: Arch Linux
- Kernel: x86_64 Linux 6.5.3-arch1-1
- RAM: 32G
- Python version: Python 3.9.18 in .venv
- Installed modules (provide output from `pip list`):
```
Package Version
------------------------------ ----------
absl-py 1.4.0
accelerate 0.22.0
aiohttp 3.8.5
aiosignal 1.3.1
anyascii 0.3.2
appdirs 1.4.4
async-timeout 4.0.3
attrs 23.1.0
audioread 3.0.0
azure-cognitiveservices-speech 1.31.0
Babel 2.12.1
bangla 0.0.2
blinker 1.6.2
bnnumerizer 0.0.2
bnunicodenormalizer 0.1.1
boltons 23.0.0
cachetools 5.3.1
certifi 2023.7.22
cffi 1.15.1
charset-normalizer 3.2.0
clean-fid 0.1.35
click 8.1.7
click-default-group 1.2.4
clip-anytorch 2.5.2
cloup 0.13.1
cmake 3.27.2
colour 0.1.5
contourpy 1.1.0
coqpit 0.0.17
cycler 0.11.0
Cython 0.29.30
dateparser 1.1.8
decorator 5.1.1
deepl 1.15.0
docker-pycreds 0.4.0
docopt 0.6.2
einops 0.6.1
encodec 0.1.1
evdev 1.6.1
ffmpeg-python 0.2.0
filelock 3.12.3
Flask 2.3.3
fonttools 4.42.1
frozenlist 1.4.0
fsspec 2023.9.0
ftfy 6.1.1
future 0.18.3
g2pkk 0.1.2
gitdb 4.0.10
GitPython 3.1.34
glcontext 2.4.0
google-auth 2.22.0
google-auth-oauthlib 1.0.0
grpcio 1.57.0
gruut 2.2.3
gruut-ipa 0.13.0
gruut-lang-de 2.0.0
gruut-lang-en 2.0.0
gruut-lang-es 2.0.0
gruut-lang-fr 2.0.2
gTTS 2.3.2
huggingface-hub 0.16.4
idna 3.4
imageio 2.31.3
importlib-metadata 6.8.0
importlib-resources 6.0.1
inflect 5.6.0
isosurfaces 0.1.0
itsdangerous 2.1.2
jamo 0.4.1
jieba 0.42.1
Jinja2 3.1.2
joblib 1.3.2
jsonlines 1.2.0
jsonmerge 1.9.2
jsonschema 4.19.0
jsonschema-specifications 2023.7.1
k-diffusion 0.0.16
kiwisolver 1.4.5
kornia 0.7.0
lazy_loader 0.3
librosa 0.10.0
lit 16.0.6
llvmlite 0.40.1
manim 0.17.3
manim-voiceover 0.3.4
ManimPango 0.4.3
mapbox-earcut 1.0.1
Markdown 3.4.4
markdown-it-py 3.0.0
MarkupSafe 2.1.3
matplotlib 3.7.2
mdurl 0.1.2
moderngl 5.8.2
moderngl-window 2.4.4
more-itertools 10.1.0
mpmath 1.3.0
msgpack 1.0.5
multidict 6.0.4
multipledispatch 1.0.0
mutagen 1.47.0
networkx 2.8.8
nltk 3.8.1
num2words 0.5.12
numba 0.57.0
numpy 1.22.0
nvidia-cublas-cu11 11.10.3.66
nvidia-cuda-cupti-cu11 11.7.101
nvidia-cuda-nvrtc-cu11 11.7.99
nvidia-cuda-runtime-cu11 11.7.99
nvidia-cudnn-cu11 8.5.0.96
nvidia-cufft-cu11 10.9.0.58
nvidia-curand-cu11 10.2.10.91
nvidia-cusolver-cu11 11.4.0.1
nvidia-cusparse-cu11 11.7.4.91
nvidia-nccl-cu11 2.14.3
nvidia-nvtx-cu11 11.7.91
oauthlib 3.2.2
openai-whisper 20230314
packaging 23.1
pandas 2.0.3
pathtools 0.1.2
Pillow 9.5.0
pip 23.2.1
platformdirs 3.10.0
pooch 1.7.0
protobuf 4.24.2
psutil 5.9.5
pyasn1 0.5.0
pyasn1-modules 0.3.0
PyAudio 0.2.13
pycairo 1.24.0
pycparser 2.21
pydub 0.25.1
pyglet 2.0.9
Pygments 2.16.1
pynndescent 0.5.10
pynput 1.7.6
pyparsing 3.0.9
pypinyin 0.49.0
pyrr 0.10.3
pysbd 0.3.4
python-crfsuite 0.9.9
python-dateutil 2.8.2
python-dotenv 0.21.1
python-slugify 8.0.1
python-xlib 0.33
pyttsx3 2.90
pytz 2023.3
PyWavelets 1.4.1
PyYAML 6.0.1
referencing 0.30.2
regex 2023.8.8
requests 2.31.0
requests-oauthlib 1.3.1
resize-right 0.0.2
rich 13.5.2
rpds-py 0.10.0
rsa 4.9
safetensors 0.3.3
scikit-image 0.21.0
scikit-learn 1.3.0
scipy 1.11.2
screeninfo 0.8.1
sentry-sdk 1.30.0
setproctitle 1.3.2
setuptools 59.8.0
six 1.16.0
skia-pathops 0.7.4
smmap 5.0.0
soundfile 0.12.1
sox 1.4.1
soxr 0.3.6
srt 3.5.3
stable-ts 2.9.0
svgelements 1.9.6
sympy 1.12
tensorboard 2.14.0
tensorboard-data-server 0.7.1
text-unidecode 1.3
threadpoolctl 3.2.0
tifffile 2023.8.30
tiktoken 0.3.1
tokenizers 0.13.3
torch 2.0.1
torchaudio 2.0.2
torchdiffeq 0.2.3
torchsde 0.2.5
torchvision 0.15.2
tqdm 4.66.1
trainer 0.0.31
trampoline 0.1.2
transformers 4.32.1
triton 2.0.0
TTS 0.16.5
typing_extensions 4.7.1
tzdata 2023.3
tzlocal 5.0.1
umap-learn 0.5.1
urllib3 1.26.16
wandb 0.15.9
watchdog 2.3.1
wcwidth 0.2.6
Werkzeug 2.3.7
wheel 0.41.0
yarl 1.9.2
zipp 3.16.2
```

LaTeX details

+ LaTeX distribution (e.g. TeX Live 2020):
+ Installed LaTeX packages:

FFMPEG

Output of `ffmpeg -version`:

```
ffmpeg version n6.0 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 13.2.1 (GCC) 20230801
configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libjxl --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-opencl --enable-opengl --enable-shared --enable-version3 --enable-vulkan
libavutil 58. 2.100 / 58. 2.100
libavcodec 60. 3.100 / 60. 3.100
libavformat 60. 3.100 / 60. 3.100
libavdevice 60. 1.100 / 60. 1.100
libavfilter 9. 3.100 / 9. 3.100
libswscale 7. 1.100 / 7. 1.100
libswresample 4. 10.100 / 4. 10.100
libpostproc 57. 1.100 / 57. 1.100
```

## Additional comments

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with OpenGLRenderer.get_texture_id and OpenGLTexturedSurface.get_image_from_file, then run the provided AnimateTorusTexture reproduction with the numbered PNG assets. Trace how image identity maps to texture IDs during Scene.replace and verify that successive frames receive the correct IDs. Done means the animation renders without flicker or stale texture reuse, with texture resources no longer growing unnecessarily.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-graphics
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.