ManimCommunity / ManimCommunity/manim
OpenGLRenderer.get_texture_id sometimes returns wrong tid
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 40.9k
- Forks
- 3.1k
- Avg merge
- 3d 12h
- Merged PRs (30d)
- 25
Description
## Description of bug / unexpected behavior
I have posted this issue on [Discord #helpForum](https://discordapp.com/channels/581738731934056449/1150099499335819295/1150099499335819295) a couple of days ago. I also posted the video showing the poorly rendered video. After analyzing this a bit further, I think it is worth to file an issue for it.
I have created a 3D scene showing a torus with textured surface. For each frame of the animation I replace the torus with a new one having a slightly different texture. The resulting video flickers and the animation seems to frequently jump back and forth.
## Expected behavior
The animation should create a smooth, flicker free video, in this case a winding line around the torus starting at the red dot towards the blue dot.
### Analysis
I have modified `OpenGLRenderer.get_texture_id` with some `print()` statements to see, what is going on.
```py
def get_texture_id(self, path):
if repr(path) not in self.path_to_texture_id:
tid = len(self.path_to_texture_id)
texture = self.context.texture(
size=path.size,
components=len(path.getbands()),
data=path.tobytes(),
)
texture.repeat_x = False
texture.repeat_y = False
texture.filter = (moderngl.NEAREST, moderngl.NEAREST)
texture.swizzle = "RRR1" if path.mode == "L" else "RGBA"
texture.use(location=tid)
self.path_to_texture_id[repr(path)] = tid
texture_id = self.path_to_texture_id[repr(path)]
print("OpenGLRenderer.get_texture_id", repr(path), path.file_name, "->", texture_id)
if self.texture_id > texture_id:
print(f">>> OpenGLRenderer.get_texture_id wants to return {texture_id}, but it should return {self.cnt-1} instead!")
self.texture_id = texture_id
return texture_id
```
I also modified `OpenGLTexturedSurface.get_image_from_file` to make `file_name` available.
```py
def get_image_from_file(
self,
image_name: str | Path,
image_mode: str,
):
image_file = get_full_raster_image_path(image_name)
image = Image.open(image_file).convert(image_mode)
image.file_name = image_name
return image
```
Below you can see the output for rendering frame `#184`. You can see that the key into `self.path_to_texture_id` is somehow flawed. The only discriminator between the images is the reported address of the `Image` object. However an image might be subject for garbage collection, and therefore the object address is probably not a good hash for the image.
For frame `#184` the method should return `tid=184`.
```
#184 OpenGLRenderer.render scene:
OpenGLMobject.get_shader_wrapper OpenGLFrameTexturedSurface
ShaderWrapper texture_path: /home/mk/dev/manim/ecc/gist/assets/MainScene0184.png
OpenGlRenderer.render_mobject mobject: OpenGLFrameTexturedSurface shader_wrapper_list: []
OpenGLRenderer.get_texture_id /home/mk/dev/manim/ecc/gist/assets/MainScene0184.png -> 128
>>> OpenGLRenderer.get_texture_id wants to return 128, but it should return 184 instead!
OpenGLRenderer.get_texture_id /home/mk/dev/manim/ecc/gist/assets/MainScene0184.png -> 154
Mesh.render self.shader:
OpenGLMobject.get_shader_wrapper OpenGLFrameTexturedSurface
ShaderWrapper texture_path: /home/mk/dev/manim/ecc/gist/assets/MainScene0185.png
```
After replacing `repr(path)` with `hash(path.file_name)` the video renders nicely as expected. However I don't think, that this is an appropriate fix.
If an `OpenGLMobject` gets removed (e.g. via `Scene.replace`) from the scene, I assume the textures should also be [released](https://moderngl.readthedocs.io/en/latest/topics/gc.html).
Also `OpenGLRenderer.path_to_texture_id` should not be constantly growing. Some housekeeping might be required.
## How to reproduce the issue
In order to run this code you need images `assets/MainScene0000.png` ... `assets/MainScene0195.png`. Then render the script with low quality, i.e. 15 frames/second, resulting in 13s video output.
Code for reproducing the problem
```py
from manim import *
from manim.opengl import *
import os
from pathlib import Path
config.renderer = "opengl"
def texture(frame):
return (
Path(os.path.realpath(__file__)).parent
/ "assets"
/ ("MainScene%04d.png" % frame)
)
class OpenGLTorus(OpenGLSurface):
def func(self, u, v):
P = np.array([np.cos(u), np.sin(u), 0])
return (self.R - self.r * np.cos(v)) * P - self.r * np.sin(v) * OUT
def __init__(self, **kwargs):
self.R = 4
self.r = 1
super().__init__(
uv_func=self.func,
u_range=(0, TAU),
v_range=(0, TAU),
resolution=(101, 101),
**kwargs,
)
def create_torus_surface(torus, frame):
return OpenGLTexturedSurface(torus, texture(frame)).scale(0.8).set_opacity(0.7)
class AnimateTorusTexture(ThreeDScene):
def construct(self):
config.disable_caching = True
torus = OpenGLTorus()
frame = 0
torus_surface = create_torus_surface(torus, frame)
def replace_torus(dt):
nonlocal torus_surface, frame
new_surface = create_torus_surface(torus, frame)
self.replace(torus_surface, new_surface)
torus_surface = new_surface
frame += 1
trk = ValueTracker()
self.add(torus_surface)
self.add_updater(replace_torus)
self.play(trk.animate.set_value(1), run_time=13)
self.remove_updater(replace_torus)
```
## Additional media files
Images/GIFs
## Logs
Terminal output
```
PASTE HERE OR PROVIDE LINK TO https://pastebin.com/ OR SIMILAR
```
## System specifications
System Details
- OS: Arch Linux
- Kernel: x86_64 Linux 6.5.3-arch1-1
- RAM: 32G
- Python version: Python 3.9.18 in .venv
- Installed modules (provide output from `pip list`):
```
Package Version
------------------------------ ----------
absl-py 1.4.0
accelerate 0.22.0
aiohttp 3.8.5
aiosignal 1.3.1
anyascii 0.3.2
appdirs 1.4.4
async-timeout 4.0.3
attrs 23.1.0
audioread 3.0.0
azure-cognitiveservices-speech 1.31.0
Babel 2.12.1
bangla 0.0.2
blinker 1.6.2
bnnumerizer 0.0.2
bnunicodenormalizer 0.1.1
boltons 23.0.0
cachetools 5.3.1
certifi 2023.7.22
cffi 1.15.1
charset-normalizer 3.2.0
clean-fid 0.1.35
click 8.1.7
click-default-group 1.2.4
clip-anytorch 2.5.2
cloup 0.13.1
cmake 3.27.2
colour 0.1.5
contourpy 1.1.0
coqpit 0.0.17
cycler 0.11.0
Cython 0.29.30
dateparser 1.1.8
decorator 5.1.1
deepl 1.15.0
docker-pycreds 0.4.0
docopt 0.6.2
einops 0.6.1
encodec 0.1.1
evdev 1.6.1
ffmpeg-python 0.2.0
filelock 3.12.3
Flask 2.3.3
fonttools 4.42.1
frozenlist 1.4.0
fsspec 2023.9.0
ftfy 6.1.1
future 0.18.3
g2pkk 0.1.2
gitdb 4.0.10
GitPython 3.1.34
glcontext 2.4.0
google-auth 2.22.0
google-auth-oauthlib 1.0.0
grpcio 1.57.0
gruut 2.2.3
gruut-ipa 0.13.0
gruut-lang-de 2.0.0
gruut-lang-en 2.0.0
gruut-lang-es 2.0.0
gruut-lang-fr 2.0.2
gTTS 2.3.2
huggingface-hub 0.16.4
idna 3.4
imageio 2.31.3
importlib-metadata 6.8.0
importlib-resources 6.0.1
inflect 5.6.0
isosurfaces 0.1.0
itsdangerous 2.1.2
jamo 0.4.1
jieba 0.42.1
Jinja2 3.1.2
joblib 1.3.2
jsonlines 1.2.0
jsonmerge 1.9.2
jsonschema 4.19.0
jsonschema-specifications 2023.7.1
k-diffusion 0.0.16
kiwisolver 1.4.5
kornia 0.7.0
lazy_loader 0.3
librosa 0.10.0
lit 16.0.6
llvmlite 0.40.1
manim 0.17.3
manim-voiceover 0.3.4
ManimPango 0.4.3
mapbox-earcut 1.0.1
Markdown 3.4.4
markdown-it-py 3.0.0
MarkupSafe 2.1.3
matplotlib 3.7.2
mdurl 0.1.2
moderngl 5.8.2
moderngl-window 2.4.4
more-itertools 10.1.0
mpmath 1.3.0
msgpack 1.0.5
multidict 6.0.4
multipledispatch 1.0.0
mutagen 1.47.0
networkx 2.8.8
nltk 3.8.1
num2words 0.5.12
numba 0.57.0
numpy 1.22.0
nvidia-cublas-cu11 11.10.3.66
nvidia-cuda-cupti-cu11 11.7.101
nvidia-cuda-nvrtc-cu11 11.7.99
nvidia-cuda-runtime-cu11 11.7.99
nvidia-cudnn-cu11 8.5.0.96
nvidia-cufft-cu11 10.9.0.58
nvidia-curand-cu11 10.2.10.91
nvidia-cusolver-cu11 11.4.0.1
nvidia-cusparse-cu11 11.7.4.91
nvidia-nccl-cu11 2.14.3
nvidia-nvtx-cu11 11.7.91
oauthlib 3.2.2
openai-whisper 20230314
packaging 23.1
pandas 2.0.3
pathtools 0.1.2
Pillow 9.5.0
pip 23.2.1
platformdirs 3.10.0
pooch 1.7.0
protobuf 4.24.2
psutil 5.9.5
pyasn1 0.5.0
pyasn1-modules 0.3.0
PyAudio 0.2.13
pycairo 1.24.0
pycparser 2.21
pydub 0.25.1
pyglet 2.0.9
Pygments 2.16.1
pynndescent 0.5.10
pynput 1.7.6
pyparsing 3.0.9
pypinyin 0.49.0
pyrr 0.10.3
pysbd 0.3.4
python-crfsuite 0.9.9
python-dateutil 2.8.2
python-dotenv 0.21.1
python-slugify 8.0.1
python-xlib 0.33
pyttsx3 2.90
pytz 2023.3
PyWavelets 1.4.1
PyYAML 6.0.1
referencing 0.30.2
regex 2023.8.8
requests 2.31.0
requests-oauthlib 1.3.1
resize-right 0.0.2
rich 13.5.2
rpds-py 0.10.0
rsa 4.9
safetensors 0.3.3
scikit-image 0.21.0
scikit-learn 1.3.0
scipy 1.11.2
screeninfo 0.8.1
sentry-sdk 1.30.0
setproctitle 1.3.2
setuptools 59.8.0
six 1.16.0
skia-pathops 0.7.4
smmap 5.0.0
soundfile 0.12.1
sox 1.4.1
soxr 0.3.6
srt 3.5.3
stable-ts 2.9.0
svgelements 1.9.6
sympy 1.12
tensorboard 2.14.0
tensorboard-data-server 0.7.1
text-unidecode 1.3
threadpoolctl 3.2.0
tifffile 2023.8.30
tiktoken 0.3.1
tokenizers 0.13.3
torch 2.0.1
torchaudio 2.0.2
torchdiffeq 0.2.3
torchsde 0.2.5
torchvision 0.15.2
tqdm 4.66.1
trainer 0.0.31
trampoline 0.1.2
transformers 4.32.1
triton 2.0.0
TTS 0.16.5
typing_extensions 4.7.1
tzdata 2023.3
tzlocal 5.0.1
umap-learn 0.5.1
urllib3 1.26.16
wandb 0.15.9
watchdog 2.3.1
wcwidth 0.2.6
Werkzeug 2.3.7
wheel 0.41.0
yarl 1.9.2
zipp 3.16.2
```
LaTeX details
+ LaTeX distribution (e.g. TeX Live 2020):
+ Installed LaTeX packages:
FFMPEG
Output of `ffmpeg -version`:
```
ffmpeg version n6.0 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 13.2.1 (GCC) 20230801
configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libjxl --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-opencl --enable-opengl --enable-shared --enable-version3 --enable-vulkan
libavutil 58. 2.100 / 58. 2.100
libavcodec 60. 3.100 / 60. 3.100
libavformat 60. 3.100 / 60. 3.100
libavdevice 60. 1.100 / 60. 1.100
libavfilter 9. 3.100 / 9. 3.100
libswscale 7. 1.100 / 7. 1.100
libswresample 4. 10.100 / 4. 10.100
libpostproc 57. 1.100 / 57. 1.100
```
## Additional comments
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with OpenGLRenderer.get_texture_id and OpenGLTexturedSurface.get_image_from_file, then run the provided AnimateTorusTexture reproduction with the numbered PNG assets. Trace how image identity maps to texture IDs during Scene.replace and verify that successive frames receive the correct IDs. Done means the animation renders without flicker or stale texture reuse, with texture resources no longer growing unnecessarily.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-graphics
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100