Bad text rendering quality when using custom fonts

I am so utterly confused by this. Here's my setup: I've customised a couple of the colors (background color). I'm using xelatex (`TEX_USE_CTEX = true`). I've also modified the `ctex_template.tex` to allow for some custom fonts.

Here's the weird part. So you know how manim turns the latex DVI files into SVG? Yeah, I've checked those, and they look hella fine both with the standard latex font (Knuth's *Computer Modern*) and with my fonts.

The video that I get, however, is another story. With the default font it looks fine, but if I use anything other than the default font, it looks horrible :(

I can't really wrap my head around this. Do you have any idea what might be causing it?