Cannot display latex correctly in make_subplots

I am using make_subplots and insert latex math expression but it displays incorrectly.
Here is my sample code:

import plotly.graph_objs as go
from plotly.subplots import make_subplots

fig = make_subplots(rows=1, cols=2)

trace1 = go.Scatter(x=[1, 2, 3], y=[4, 5, 6])
trace2 = go.Scatter(x=[20, 30, 40], y=[50, 60, 70])

fig.add_trace(trace1, row=1, col=1)
fig.add_trace(trace2, row=1, col=2)

fig.update_layout(
    xaxis=dict(title='$x^2$'),
    xaxis2=dict(title='$\\sqrt{x}$')
)

fig.show()

Here is my output:

As you can see, it does not display correctly.

My setup:
OS: Ubuntu 20.04
Python: 3.10.9
Plotly: 5.16.1

Start by running the supplied Python example with Plotly 5.16.1 and compare how the two make_subplots axis titles render. Trace the title-rendering entry point used by make_subplots, then verify that both LaTeX expressions display correctly in the resulting figure.