RNG generators make python abort in python3.11 and tf2.12.1 onwards

### Solution to issue cannot be found in the documentation.

- [X] I checked the documentation.

### Issue

Unable to make random number generators. This happens with conda-forge (python 3.11.3 + tensorflow 2.12.1 and onwards). It works fine with pip (python 3.11.3 + tensorflow 2.12.0 and onwards)

Standalone code to reproduce the issue
```
import tensorflow as tf
g = tf.random.Generator.from_seed(1)
g.normal(shape=[3])
```
Relevant log output
```
Assertion failed: (f == nullptr || dynamic_cast(f) != nullptr), function down_cast, file ./tensorflow/tsl/platform/default/casts.h, line 58.
[1] 18173 abort python
```

### Installed packages

```shell
absl-py 1.4.0 pyhd8ed1ab_0 conda-forge
aiohttp 3.9.1 py311he705e18_0 conda-forge
aiosignal 1.3.1 pyhd8ed1ab_0 conda-forge
anyio 4.1.0 pyhd8ed1ab_0 conda-forge
appnope 0.1.3 pyhd8ed1ab_0 conda-forge
argon2-cffi 23.1.0 pyhd8ed1ab_0 conda-forge
argon2-cffi-bindings 21.2.0 py311h2725bcf_4 conda-forge
arrow 1.3.0 pyhd8ed1ab_0 conda-forge
asttokens 2.4.1 pyhd8ed1ab_0 conda-forge
astunparse 1.6.3 pyhd8ed1ab_0 conda-forge
async-lru 2.0.4 pyhd8ed1ab_0 conda-forge
attrs 23.1.0 pyh71513ae_1 conda-forge
babel 2.13.1 pyhd8ed1ab_0 conda-forge
beautifulsoup4 4.12.2 pyha770c72_0 conda-forge
blas 2.120 mkl conda-forge
blas-devel 3.9.0 20_osx64_mkl conda-forge
bleach 6.1.0 pyhd8ed1ab_0 conda-forge
blinker 1.7.0 pyhd8ed1ab_0 conda-forge
brotli 1.1.0 h0dc2134_1 conda-forge
brotli-bin 1.1.0 h0dc2134_1 conda-forge
brotli-python 1.1.0 py311hdf8f085_1 conda-forge
bzip2 1.0.8 h10d778d_5 conda-forge
c-ares 1.22.1 h10d778d_0 conda-forge
ca-certificates 2023.11.17 h8857fd0_0 conda-forge
cached-property 1.5.2 hd8ed1ab_1 conda-forge
cached_property 1.5.2 pyha770c72_1 conda-forge
cachetools 5.3.2 pyhd8ed1ab_0 conda-forge
certifi 2023.11.17 pyhd8ed1ab_0 conda-forge
cffi 1.16.0 py311hc0b63fd_0 conda-forge
charset-normalizer 3.3.2 pyhd8ed1ab_0 conda-forge
click 8.1.7 unix_pyh707e725_0 conda-forge
cloudpickle 3.0.0 pyhd8ed1ab_0 conda-forge
colorama 0.4.6 pyhd8ed1ab_0 conda-forge
comm 0.1.4 pyhd8ed1ab_0 conda-forge
contourpy 1.2.0 py311h7bea37d_0 conda-forge
cryptography 41.0.7 py311hd51016d_0 conda-forge
cycler 0.12.1 pyhd8ed1ab_0 conda-forge
debugpy 1.8.0 py311hdf8f085_1 conda-forge
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge
dill 0.3.7 pyhd8ed1ab_0 conda-forge
dm-tree 0.1.8 py311h700567c_0 conda-forge
entrypoints 0.4 pyhd8ed1ab_0 conda-forge
etils 1.5.1 pyhd8ed1ab_1 conda-forge
exceptiongroup 1.2.0 pyhd8ed1ab_0 conda-forge
executing 2.0.1 pyhd8ed1ab_0 conda-forge
ffmpeg 4.3 h0a44026_0 pytorch
filelock 3.13.1 pyhd8ed1ab_0 conda-forge
flatbuffers 23.5.26 he965462_1 conda-forge
fonttools 4.45.1 py311he705e18_0 conda-forge
fqdn 1.5.1 pyhd8ed1ab_0 conda-forge
freetype 2.12.1 h60636b9_2 conda-forge
frozenlist 1.4.0 py311h2725bcf_1 conda-forge
gast 0.5.4 pyhd8ed1ab_0 conda-forge
gettext 0.21.1 h8a4c099_0 conda-forge
giflib 5.2.1 hb7f2c08_3 conda-forge
gmp 6.3.0 h93d8f39_0 conda-forge
gmpy2 2.1.2 py311hc5b4402_1 conda-forge
gnutls 3.6.13 h756fd2b_1 conda-forge
google-auth 2.23.4 pyhca7485f_0 conda-forge
google-auth-oauthlib 1.0.0 pyhd8ed1ab_1 conda-forge
google-pasta 0.2.0 pyh8c360ce_0 conda-forge
googleapis-common-protos 1.61.0 pyhd8ed1ab_0 conda-forge
grpcio 1.54.3 py311ha1af104_0 conda-forge
h5py 3.10.0 nompi_py311he02c01d_100 conda-forge
hdf5 1.14.2 nompi_hedada53_100 conda-forge
icu 73.2 hf5e326d_0 conda-forge
idna 3.6 pyhd8ed1ab_0 conda-forge
importlib-metadata 6.8.0 pyha770c72_0 conda-forge
importlib_metadata 6.8.0 hd8ed1ab_0 conda-forge
importlib_resources 6.1.1 pyhd8ed1ab_0 conda-forge
iniconfig 2.0.0 pyhd8ed1ab_0 conda-forge
ipykernel 6.26.0 pyh3cd1d5f_0 conda-forge
ipython 8.18.1 pyh31011fe_1 conda-forge
isoduration 20.11.0 pyhd8ed1ab_0 conda-forge
jax 0.4.19 pyhd8ed1ab_0 conda-forge
jaxlib 0.4.14 cpu_py311hd8ff182_1 conda-forge
jedi 0.19.1 pyhd8ed1ab_0 conda-forge
jinja2 3.1.2 pyhd8ed1ab_1 conda-forge
joblib 1.3.2 pyhd8ed1ab_0 conda-forge
json5 0.9.14 pyhd8ed1ab_0 conda-forge
jsonpointer 2.4 py311h6eed73b_3 conda-forge
jsonschema 4.20.0 pyhd8ed1ab_0 conda-forge
jsonschema-specifications 2023.11.1 pyhd8ed1ab_0 conda-forge
jsonschema-with-format-nongpl 4.20.0 pyhd8ed1ab_0 conda-forge
jupyter-lsp 2.2.1 pyhd8ed1ab_0 conda-forge
jupyter_client 8.6.0 pyhd8ed1ab_0 conda-forge
jupyter_core 5.5.0 py311h6eed73b_0 conda-forge
jupyter_events 0.9.0 pyhd8ed1ab_0 conda-forge
jupyter_server 2.11.1 pyhd8ed1ab_0 conda-forge
jupyter_server_terminals 0.4.4 pyhd8ed1ab_1 conda-forge
jupyterlab 4.0.9 pyhd8ed1ab_0 conda-forge
jupyterlab_pygments 0.3.0 pyhd8ed1ab_0 conda-forge
jupyterlab_server 2.25.2 pyhd8ed1ab_0 conda-forge
keras 2.14.0 pyhd8ed1ab_0 conda-forge
keras-core 0.1.7 pyhd8ed1ab_0 conda-forge
keras-tuner 1.4.6 pyhd8ed1ab_0 conda-forge
kiwisolver 1.4.5 py311h5fe6e05_1 conda-forge
krb5 1.21.2 hb884880_0 conda-forge
kt-legacy 1.0.5 pyhd8ed1ab_0 conda-forge
lame 3.100 hb7f2c08_1003 conda-forge
lcms2 2.15 hd6ba6f3_3 conda-forge
lerc 4.0.0 hb486fe8_0 conda-forge
libabseil 20230125.3 cxx17_h000cb23_0 conda-forge
libaec 1.1.2 he965462_1 conda-forge
libblas 3.9.0 20_osx64_mkl conda-forge
libbrotlicommon 1.1.0 h0dc2134_1 conda-forge
libbrotlidec 1.1.0 h0dc2134_1 conda-forge
libbrotlienc 1.1.0 h0dc2134_1 conda-forge
libcblas 3.9.0 20_osx64_mkl conda-forge
libcurl 8.4.0 h726d00d_0 conda-forge
libcxx 16.0.6 hd57cbcb_0 conda-forge
libdeflate 1.19 ha4e1b8e_0 conda-forge
libedit 3.1.20191231 hed1e85f_2 conda-forge
libev 4.33 haf1e3a3_1 conda-forge
libexpat 2.5.0 hf0c8a7f_1 conda-forge
libffi 3.4.2 h0d85af4_5 conda-forge
libgfortran 5.0.0 13_2_0_h97931a8_1 conda-forge
libgfortran5 13.2.0 h2873a65_1 conda-forge
libgrpc 1.54.3 hfaa49da_0 conda-forge
libiconv 1.17 hac89ed1_0 conda-forge
libjpeg-turbo 3.0.0 h0dc2134_1 conda-forge
liblapack 3.9.0 20_osx64_mkl conda-forge
liblapacke 3.9.0 20_osx64_mkl conda-forge
libnghttp2 1.58.0 h64cf6d3_0 conda-forge
libopenblas 0.3.25 openmp_hfef2a42_0 conda-forge
libpng 1.6.39 ha978bb4_0 conda-forge
libprotobuf 3.21.12 h7d26f99_2 conda-forge
libsodium 1.0.18 hbcb3906_1 conda-forge
libsqlite 3.44.2 h92b6c6a_0 conda-forge
libssh2 1.11.0 hd019ec5_0 conda-forge
libtiff 4.6.0 h684deea_2 conda-forge
libwebp-base 1.3.2 h0dc2134_0 conda-forge
libxcb 1.15 hb7f2c08_0 conda-forge
libzlib 1.2.13 h8a1eda9_5 conda-forge
llvm-openmp 17.0.5 hb6ac08f_0 conda-forge
markdown 3.5.1 pyhd8ed1ab_0 conda-forge
markdown-it-py 3.0.0 pyhd8ed1ab_0 conda-forge
markupsafe 2.1.3 py311h2725bcf_1 conda-forge
matplotlib 3.8.2 py311h6eed73b_0 conda-forge
matplotlib-base 3.8.2 py311hd316c10_0 conda-forge
matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge
mdurl 0.1.0 pyhd8ed1ab_0 conda-forge
mistune 3.0.2 pyhd8ed1ab_0 conda-forge
mkl 2023.2.0 h54c2260_50500 conda-forge
mkl-devel 2023.2.0 h694c41f_50500 conda-forge
mkl-include 2023.2.0 h6bab518_50500 conda-forge
ml_dtypes 0.2.0 py311hab14417_2 conda-forge
mpc 1.3.1 h81bd1dd_0 conda-forge
mpfr 4.2.1 h0c69b56_0 conda-forge
mpmath 1.3.0 pyhd8ed1ab_0 conda-forge
multidict 6.0.4 py311h5547dcb_1 conda-forge
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
namex 0.0.7 pyhd8ed1ab_0 conda-forge
nbclient 0.8.0 pyhd8ed1ab_0 conda-forge
nbconvert-core 7.11.0 pyhd8ed1ab_0 conda-forge
nbformat 5.9.2 pyhd8ed1ab_0 conda-forge
ncurses 6.4 h93d8f39_2 conda-forge
nest-asyncio 1.5.8 pyhd8ed1ab_0 conda-forge
nettle 3.6 hedd7734_0 conda-forge
networkx 3.2.1 pyhd8ed1ab_0 conda-forge
notebook-shim 0.2.3 pyhd8ed1ab_0 conda-forge
numpy 1.26.2 py311h93c810c_0 conda-forge
oauthlib 3.2.2 pyhd8ed1ab_0 conda-forge
openh264 2.1.1 hfd3ada9_0 conda-forge
openjpeg 2.5.0 ha4da562_3 conda-forge
openssl 3.2.0 hd75f5a5_1 conda-forge
opt-einsum 3.3.0 hd8ed1ab_2 conda-forge
opt_einsum 3.3.0 pyhc1e730c_2 conda-forge
overrides 7.4.0 pyhd8ed1ab_0 conda-forge
packaging 23.2 pyhd8ed1ab_0 conda-forge
pandas 2.1.3 py311h1eadf79_0 conda-forge
pandocfilters 1.5.0 pyhd8ed1ab_0 conda-forge
parso 0.8.3 pyhd8ed1ab_0 conda-forge
pexpect 4.8.0 pyh9f0ad1d_2 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pillow 10.1.0 py311hea5c87a_0 conda-forge
pip 23.3.1 pyhd8ed1ab_0 conda-forge
pkgutil-resolve-name 1.3.10 pyhd8ed1ab_1 conda-forge
platformdirs 4.0.0 pyhd8ed1ab_0 conda-forge
plotly 5.18.0 pyhd8ed1ab_0 conda-forge
pluggy 1.3.0 pyhd8ed1ab_0 conda-forge
prometheus_client 0.19.0 pyhd8ed1ab_0 conda-forge
promise 2.3 py311h6eed73b_8 conda-forge
prompt-toolkit 3.0.41 pyha770c72_0 conda-forge
protobuf 4.21.12 py311h814d153_0 conda-forge
psutil 5.9.5 py311h2725bcf_1 conda-forge
pthread-stubs 0.4 hc929b4f_1001 conda-forge
ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge
pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge
pyasn1 0.5.0 pyhd8ed1ab_0 conda-forge
pyasn1-modules 0.3.0 pyhd8ed1ab_0 conda-forge
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pygments 2.17.2 pyhd8ed1ab_0 conda-forge
pyjwt 2.8.0 pyhd8ed1ab_0 conda-forge
pyobjc-core 10.0 py311hf110eff_0 conda-forge
pyobjc-framework-cocoa 10.0 py311hf110eff_1 conda-forge
pyopenssl 23.3.0 pyhd8ed1ab_0 conda-forge
pyparsing 3.1.1 pyhd8ed1ab_0 conda-forge
pysocks 1.7.1 pyha2e5f31_6 conda-forge
pytest 7.4.3 pyhd8ed1ab_0 conda-forge
python 3.11.6 h30d4d87_0_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python-fastjsonschema 2.19.0 pyhd8ed1ab_0 conda-forge
python-flatbuffers 23.5.26 pyhd8ed1ab_0 conda-forge
python-json-logger 2.0.7 pyhd8ed1ab_0 conda-forge
python-tzdata 2023.3 pyhd8ed1ab_0 conda-forge
python_abi 3.11 4_cp311 conda-forge
pytorch 2.1.1 py3.11_0 pytorch
pytz 2023.3.post1 pyhd8ed1ab_0 conda-forge
pyu2f 0.1.5 pyhd8ed1ab_0 conda-forge
pyyaml 6.0.1 py311h2725bcf_1 conda-forge
pyzmq 25.1.1 py311he3804a1_2 conda-forge
re2 2023.03.02 h096449b_0 conda-forge
readline 8.2 h9e318b2_1 conda-forge
referencing 0.31.0 pyhd8ed1ab_0 conda-forge
requests 2.31.0 pyhd8ed1ab_0 conda-forge
requests-oauthlib 1.3.1 pyhd8ed1ab_0 conda-forge
rfc3339-validator 0.1.4 pyhd8ed1ab_0 conda-forge
rfc3986-validator 0.1.1 pyh9f0ad1d_0 conda-forge
rich 13.7.0 pyhd8ed1ab_0 conda-forge
rpds-py 0.13.1 py311h5e0f0e4_0 conda-forge
rsa 4.9 pyhd8ed1ab_0 conda-forge
scikit-learn 1.3.2 py311h66081b9_1 conda-forge
scipy 1.11.4 py311he0bea55_0 conda-forge
send2trash 1.8.2 pyhd1c38e8_0 conda-forge
setuptools 68.2.2 pyhd8ed1ab_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
snappy 1.1.10 h225ccf5_0 conda-forge
sniffio 1.3.0 pyhd8ed1ab_0 conda-forge
soupsieve 2.5 pyhd8ed1ab_1 conda-forge
stack_data 0.6.2 pyhd8ed1ab_0 conda-forge
sympy 1.12 pypyh9d50eac_103 conda-forge
tbb 2021.10.0 h1c7c39f_2 conda-forge
tenacity 8.2.3 pyhd8ed1ab_0 conda-forge
tensorboard 2.14.1 pyhd8ed1ab_0 conda-forge
tensorboard-data-server 0.7.0 py311h892b619_1 conda-forge
tensorflow 2.14.0 cpu_py311hfac9faf_0 conda-forge
tensorflow-base 2.14.0 cpu_py311h3619e24_0 conda-forge
tensorflow-datasets 4.8.3 pyhd8ed1ab_0 conda-forge
tensorflow-estimator 2.14.0 cpu_py311h9be46c4_0 conda-forge
tensorflow-metadata 1.13.1 pyhd8ed1ab_0 conda-forge
tensorflow-probability 0.22.1 pyh4af843d_1 conda-forge
termcolor 2.3.0 pyhd8ed1ab_0 conda-forge
terminado 0.18.0 pyh31c8845_0 conda-forge
threadpoolctl 3.2.0 pyha21a80b_0 conda-forge
tinycss2 1.2.1 pyhd8ed1ab_0 conda-forge
tk 8.6.13 h1abcd95_1 conda-forge
toml 0.10.2 pyhd8ed1ab_0 conda-forge
tomli 2.0.1 pyhd8ed1ab_0 conda-forge
torchaudio 2.1.1 py311_cpu pytorch
torchvision 0.16.1 py311_cpu pytorch
tornado 6.3.3 py311h2725bcf_1 conda-forge
tqdm 4.66.1 pyhd8ed1ab_0 conda-forge
traitlets 5.14.0 pyhd8ed1ab_0 conda-forge
types-python-dateutil 2.8.19.14 pyhd8ed1ab_0 conda-forge
typing-extensions 4.8.0 hd8ed1ab_0 conda-forge
typing_extensions 4.8.0 pyha770c72_0 conda-forge
typing_utils 0.1.0 pyhd8ed1ab_0 conda-forge
tzdata 2023c h71feb2d_0 conda-forge
uri-template 1.3.0 pyhd8ed1ab_0 conda-forge
urllib3 2.1.0 pyhd8ed1ab_0 conda-forge
wcwidth 0.2.12 pyhd8ed1ab_0 conda-forge
webcolors 1.13 pyhd8ed1ab_0 conda-forge
webencodings 0.5.1 pyhd8ed1ab_2 conda-forge
websocket-client 1.6.4 pyhd8ed1ab_0 conda-forge
werkzeug 3.0.1 pyhd8ed1ab_0 conda-forge
wheel 0.42.0 pyhd8ed1ab_0 conda-forge
wrapt 1.14.1 py311h5547dcb_1 conda-forge
xorg-libxau 1.0.11 h0dc2134_0 conda-forge
xorg-libxdmcp 1.1.3 h35c211d_0 conda-forge
xz 5.2.6 h775f41a_0 conda-forge
yaml 0.2.5 h0d85af4_2 conda-forge
yarl 1.9.3 py311he705e18_0 conda-forge
zeromq 4.3.5 h93d8f39_0 conda-forge
zipp 3.17.0 pyhd8ed1ab_0 conda-forge
zlib 1.2.13 h8a1eda9_5 conda-forge
zstd 1.5.5 h829000d_0 conda-forge
```

### Environment info

```shell
environment : new (active)
env location : /Users/zheng/micromamba/envs/new
user config files : /Users/zheng/.mambarc
populated config files : /Users/zheng/.condarc
libmamba version : 1.4.5
micromamba version : 1.4.5
curl version : libcurl/8.1.2 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.11 nghttp2/1.51.0
libarchive version : libarchive 3.5.3 zlib/1.2.11 liblzma/5.0.5 bz2lib/1.0.8
virtual packages : __unix=0=0
__osx=13.6.1=0
__archspec=1=x86_64
channels : https://conda.anaconda.org/conda-forge/osx-64
https://conda.anaconda.org/conda-forge/noarch
https://repo.anaconda.com/pkgs/main/osx-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/osx-64
https://repo.anaconda.com/pkgs/r/noarch
base environment : /Users/zheng/micromamba
platform : osx-64
```