EveryVoiceTTS / EveryVoiceTTS/EveryVoice
Empty directory created "preprocessed/YourDataSet" when training FP model
- Dominant language
- Python
- Stars
- 45
- Forks
- 4
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 14
Description
### Bug description
An empty directory will be created when training a FP model. `preprocessed/YourDataSet` with this message below printed in the logs.
`2025-04-23 09:55:15.736 | INFO | everyvoice.config.validation_helpers:directory_path_must_exist:89 - Directory at preprocessed/YourDataSet does not exist. Creating...`
This does not happen when training a vocoder.
### How to reproduce the bug
Normal FP training ...
`everyvoice train text-to-spec --devices 1 --nodes 1 config/everyvoice-text-to-spec.yaml`
### Error messages and logs
`2025-04-23 09:55:15.736 | INFO | everyvoice.config.validation_helpers:directory_path_must_exist:89 - Directory at preprocessed/YourDataSet does not exist. Creating...`
### Environment
EveryVoice Diagnostic information
EveryVoice version: 0.3.0
Python version: 3.10.17 | packaged by conda-forge | (main, Apr 10 2025, 22:19:12) [GCC 13.3.0]
System: Linux 5.15.0-136-generic #147-Ubuntu SMP Sat Mar 15 15:53:30 UTC 2025 x86_64
*torch* modules installed using pip:
pytorch-lightning==2.5.1
torch==2.3.1+cu118
torchaudio==2.3.1+cu118
torchinfo==1.8.0
torchmetrics==1.7.1
Other modules installed using pip:
absl-py==2.2.2
aiofiles==24.1.0
aiohappyeyeballs==2.6.1
aiohttp==3.11.18
aiosignal==1.3.2
annotated-types==0.7.0
anyio==4.9.0
anytree==2.13.0
arrow==1.3.0
async-timeout==5.0.1
attrs==25.3.0
audioread==3.0.1
black==24.10.0
certifi==2025.1.31
cffi==1.17.1
cfgv==3.4.0
charset-normalizer==3.4.1
chevron==0.14.0
click==8.1.8
clipdetect==0.1.4
coloredlogs==15.0.1
contourpy==1.3.2
coverage==7.8.0
cycler==0.12.1
Cython==3.0.12
decorator==5.2.1
deepdiff==8.4.2
distlib==0.3.9
dnspython==2.7.0
docopt==0.6.2
editdistance==0.8.1
einops==0.5.0
email_validator==2.2.0
et_xmlfile==2.0.0
-e git+https://github.com/roedoejet/EveryVoice.git@74069db1a9223319321b4340b1e0d1587c1e9883#egg=EveryVoice
exceptiongroup==1.2.2
fastapi==0.115.12
ffmpy==0.5.0
filelock==3.18.0
flake8==7.2.0
fonttools==4.57.0
frozenlist==1.6.0
fsspec==2025.3.2
g2p==2.2.1
gitlint-core==0.19.1
gradio==5.25.2
gradio_client==1.8.0
grapheme==0.6.0
groovy==0.1.2
grpcio==1.71.0
h11==0.14.0
httpcore==1.0.8
httpx==0.28.1
huggingface-hub==0.30.2
humanfriendly==10.0
identify==2.6.10
idna==3.10
iniconfig==2.1.0
ipatok==0.4.2
isort==6.0.1
Jinja2==3.1.6
joblib==1.4.2
jsonschema==4.23.0
jsonschema-specifications==2025.4.1
kiwisolver==1.4.8
librosa==0.9.2
lightning==2.5.1
lightning-utilities==0.14.3
lit==18.1.8
llvmlite==0.44.0
loguru==0.6.0
lxml==5.4.0
Markdown==3.8
markdown-it-py==3.0.0
MarkupSafe==3.0.2
matplotlib==3.9.4
mccabe==0.7.0
mdurl==0.1.2
merge-args==0.1.5
mpmath==1.3.0
multidict==6.4.3
munkres==1.1.4
mypy==1.15.0
mypy_extensions==1.1.0
networkx==3.4.2
nltk==3.9.1
nodeenv==1.9.1
numba==0.61.2
numpy==1.26.4
nvidia-cublas-cu11==11.11.3.6
nvidia-cuda-cupti-cu11==11.8.87
nvidia-cuda-nvrtc-cu11==11.8.89
nvidia-cuda-runtime-cu11==11.8.89
nvidia-cudnn-cu11==8.7.0.84
nvidia-cufft-cu11==10.9.0.58
nvidia-curand-cu11==10.3.0.86
nvidia-cusolver-cu11==11.4.1.48
nvidia-cusparse-cu11==11.7.5.86
nvidia-nccl-cu11==2.20.5
nvidia-nvtx-cu11==11.8.86
openpyxl==3.1.5
ordered-set==4.1.0
orderly-set==5.4.0
orjson==3.10.16
packaging==24.2
pandas==2.2.3
panphon==0.20.0
pathspec==0.12.1
pep440==0.1.2
pillow==11.2.1
platformdirs==4.3.7
pluggy==1.5.0
pooch==1.8.2
pre_commit==4.2.0
prompt_toolkit==3.0.51
pronunciation-dictionary==0.0.6
propcache==0.3.1
protobuf==4.25.6
pycodestyle==2.13.0
pycountry==22.3.5
pycparser==2.22
pydantic==2.7.4
pydantic_core==2.18.4
pydub==0.25.1
pyflakes==3.3.2
Pygments==2.19.1
pympi-ling==1.70.2
pyparsing==3.2.3
pysdtw==0.0.5
pytest==7.4.4
python-dateutil==2.9.0.post0
python-multipart==0.0.20
python-slugify==5.0.0
pytz==2025.2
pyworld==0.3.4
PyYAML==6.0.2
questionary==1.10.0
readalongs==1.2.1
referencing==0.36.2
regex==2024.11.6
requests==2.32.3
resampy==0.4.3
rich==14.0.0
rpds-py==0.24.0
ruff==0.11.6
safehttpx==0.1.6
scikit-learn==1.6.1
scipy==1.15.2
semantic-version==2.10.0
sh==2.2.2
shellingham==1.5.4
simple-term-menu==1.5.2
six==1.17.0
sniffio==1.3.1
soundfile==0.13.1
soundswallower==0.6.5
starlette==0.46.2
sympy==1.13.3
tabulate==0.9.0
tensorboard==2.19.0
tensorboard-data-server==0.7.2
termcolor==2.3.0
text-unidecode==1.3
TextGrid==1.6.1
textgrid-tools==0.0.8
threadpoolctl==3.6.0
tomli==2.2.1
tomlkit==0.13.2
tqdm==4.67.1
triton==2.3.1
typer==0.15.2
types-python-dateutil==2.9.0.20241206
types-PyYAML==6.0.12.20250402
types-requests==2.32.0.20250328
types-setuptools==79.0.0.20250422
types-tabulate==0.9.0.0
types-tqdm==4.67.0.20250417
typing_extensions==4.13.2
tzdata==2025.2
unicodecsv==0.14.1
urllib3==2.4.0
uvicorn==0.34.2
virtualenv==20.30.0
wcwidth==0.2.13
websockets==15.0.1
webvtt-py==0.4.2
Werkzeug==3.1.3
yarl==1.20.0
yaspin==3.1.0
Modules installed using conda:
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
alsa-lib 1.2.14 hb9d3cd8_0 conda-forge
aom 3.9.1 hac33072_0 conda-forge
attr 2.5.1 h166bdaf_1 conda-forge
bzip2 1.0.8 h4bc722e_7 conda-forge
ca-certificates 2025.1.31 hbd8a1cb_1 conda-forge
cairo 1.18.4 h3394656_0 conda-forge
dav1d 1.2.1 hd590300_0 conda-forge
dbus 1.13.6 h5008d03_3 conda-forge
expat 2.7.0 h5888daf_0 conda-forge
ffmpeg 7.1.1 gpl_h0b79d52_704 conda-forge
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 h77eed37_3 conda-forge
fontconfig 2.15.0 h7e30c49_1 conda-forge
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
freetype 2.13.3 ha770c72_1 conda-forge
fribidi 1.0.10 h36c2ea0_0 conda-forge
gdk-pixbuf 2.42.12 hb9ae30d_0 conda-forge
gettext 0.23.1 h5888daf_0 conda-forge
gettext-tools 0.23.1 h5888daf_0 conda-forge
gmp 6.3.0 hac33072_2 conda-forge
graphite2 1.3.13 h59595ed_1003 conda-forge
harfbuzz 11.1.0 h3beb420_0 conda-forge
icu 75.1 he02047a_0 conda-forge
jack 1.9.22 h7c63dc7_2 conda-forge
lame 3.100 h166bdaf_1003 conda-forge
ld_impl_linux-64 2.43 h712a8e2_4 conda-forge
lerc 4.0.0 h0aef613_1 conda-forge
level-zero 1.21.9 h84d6215_0 conda-forge
libabseil 20250127.1 cxx17_hbbce691_0 conda-forge
libasprintf 0.23.1 h8e693c7_0 conda-forge
libasprintf-devel 0.23.1 h8e693c7_0 conda-forge
libass 0.17.3 h52826cd_2 conda-forge
libcap 2.75 h39aace5_0 conda-forge
libdb 6.2.32 h9c3ff4c_0 conda-forge
libdeflate 1.23 h86f0d12_0 conda-forge
libdrm 2.4.124 hb9d3cd8_0 conda-forge
libegl 1.7.0 ha4b6fd6_2 conda-forge
libexpat 2.7.0 h5888daf_0 conda-forge
libffi 3.4.6 h2dba641_1 conda-forge
libflac 1.4.3 h59595ed_0 conda-forge
libfreetype 2.13.3 ha770c72_1 conda-forge
libfreetype6 2.13.3 h48d6fc4_1 conda-forge
libgcc 14.2.0 h767d61c_2 conda-forge
libgcc-ng 14.2.0 h69a702a_2 conda-forge
libgcrypt-lib 1.11.0 hb9d3cd8_2 conda-forge
libgettextpo 0.23.1 h5888daf_0 conda-forge
libgettextpo-devel 0.23.1 h5888daf_0 conda-forge
libgl 1.7.0 ha4b6fd6_2 conda-forge
libglib 2.84.1 h2ff4ddf_0 conda-forge
libglvnd 1.7.0 ha4b6fd6_2 conda-forge
libglx 1.7.0 ha4b6fd6_2 conda-forge
libgomp 14.2.0 h767d61c_2 conda-forge
libgpg-error 1.54 hbd13f7d_0 conda-forge
libhwloc 2.11.2 default_h0d58e46_1001 conda-forge
libiconv 1.18 h4ce23a2_1 conda-forge
libjpeg-turbo 3.1.0 hb9d3cd8_0 conda-forge
liblzma 5.8.1 hb9d3cd8_0 conda-forge
liblzma-devel 5.8.1 hb9d3cd8_0 conda-forge
libnsl 2.0.1 hd590300_0 conda-forge
libogg 1.3.5 h4ab18f5_0 conda-forge
libopenvino 2025.0.0 hdc3f47d_3 conda-forge
libopenvino-auto-batch-plugin 2025.0.0 h4d9b6c2_3 conda-forge
libopenvino-auto-plugin 2025.0.0 h4d9b6c2_3 conda-forge
libopenvino-hetero-plugin 2025.0.0 h981d57b_3 conda-forge
libopenvino-intel-cpu-plugin 2025.0.0 hdc3f47d_3 conda-forge
libopenvino-intel-gpu-plugin 2025.0.0 hdc3f47d_3 conda-forge
libopenvino-intel-npu-plugin 2025.0.0 hdc3f47d_3 conda-forge
libopenvino-ir-frontend 2025.0.0 h981d57b_3 conda-forge
libopenvino-onnx-frontend 2025.0.0 h0e684df_3 conda-forge
libopenvino-paddle-frontend 2025.0.0 h0e684df_3 conda-forge
libopenvino-pytorch-frontend 2025.0.0 h5888daf_3 conda-forge
libopenvino-tensorflow-frontend 2025.0.0 h684f15b_3 conda-forge
libopenvino-tensorflow-lite-frontend 2025.0.0 h5888daf_3 conda-forge
libopus 1.5.2 hd0c01bc_0 conda-forge
libpciaccess 0.18 hd590300_0 conda-forge
libpng 1.6.47 h943b412_0 conda-forge
libprotobuf 5.29.3 h501fc15_1 conda-forge
librsvg 2.58.4 he92a37e_3 conda-forge
libsndfile 1.2.2 hc60ed4a_1 conda-forge
libsqlite 3.49.1 hee588c1_2 conda-forge
libstdcxx 14.2.0 h8f9b012_2 conda-forge
libstdcxx-ng 14.2.0 h4852527_2 conda-forge
libsystemd0 257.4 h4e0b6ca_1 conda-forge
libtiff 4.7.0 hd9ff511_4 conda-forge
libudev1 257.4 hbe16f8c_1 conda-forge
libunwind 1.6.2 h9c3ff4c_0 conda-forge
liburing 2.9 h84d6215_0 conda-forge
libusb 1.0.28 hb9d3cd8_0 conda-forge
libuuid 2.38.1 h0b41bf4_0 conda-forge
libva 2.22.0 h4f16b4b_2 conda-forge
libvorbis 1.3.7 h9c3ff4c_0 conda-forge
libvpx 1.14.1 hac33072_0 conda-forge
libwebp-base 1.5.0 h851e524_0 conda-forge
libxcb 1.17.0 h8a09558_0 conda-forge
libxcrypt 4.4.36 hd590300_1 conda-forge
libxkbcommon 1.8.1 hc4a0caf_0 conda-forge
libxml2 2.13.7 h4bc477f_1 conda-forge
libzlib 1.3.1 hb9d3cd8_2 conda-forge
lz4-c 1.10.0 h5888daf_1 conda-forge
mad 0.15.1b h9c3ff4c_1 conda-forge
mpg123 1.32.9 hc50e24c_0 conda-forge
ncurses 6.5 h2d0b736_3 conda-forge
ocl-icd 2.3.3 hb9d3cd8_0 conda-forge
opencl-headers 2024.10.24 h5888daf_0 conda-forge
openh264 2.6.0 hc22cd8d_0 conda-forge
openssl 3.5.0 h7b32b05_0 conda-forge
pango 1.56.3 h9ac818e_1 conda-forge
pcre2 10.44 hba22ea6_2 conda-forge
pip 25.0.1 pyh8b19718_0 conda-forge
pixman 0.44.2 h29eaf8c_0 conda-forge
pthread-stubs 0.4 hb9d3cd8_1002 conda-forge
pugixml 1.15 h3f63f65_0 conda-forge
pulseaudio-client 17.0 hac146a9_1 conda-forge
python 3.10.17 hd6af730_0_cpython conda-forge
readline 8.2 h8c095d6_2 conda-forge
sdl2 2.32.54 h9b8e6db_0 conda-forge
sdl3 3.2.10 h3083f51_0 conda-forge
setuptools 79.0.0 pyhff2d567_0 conda-forge
snappy 1.2.1 h8bd8927_1 conda-forge
sox 14.4.2 h59a48fd_1020 conda-forge
svt-av1 3.0.2 h5888daf_0 conda-forge
tbb 2022.1.0 h4ce085d_0 conda-forge
tk 8.6.13 noxft_h4845f30_101 conda-forge
wayland 1.23.1 h3e06ad9_0 conda-forge
wayland-protocols 1.43 hd8ed1ab_0 conda-forge
wheel 0.45.1 pyhd8ed1ab_1 conda-forge
x264 1!164.3095 h166bdaf_2 conda-forge
x265 3.5 h924138e_3 conda-forge
xkeyboard-config 2.43 hb9d3cd8_0 conda-forge
xorg-libice 1.1.2 hb9d3cd8_0 conda-forge
xorg-libsm 1.2.6 he73a12e_0 conda-forge
xorg-libx11 1.8.12 h4f16b4b_0 conda-forge
xorg-libxau 1.0.12 hb9d3cd8_0 conda-forge
xorg-libxcursor 1.2.3 hb9d3cd8_0 conda-forge
xorg-libxdmcp 1.1.5 hb9d3cd8_0 conda-forge
xorg-libxext 1.3.6 hb9d3cd8_0 conda-forge
xorg-libxfixes 6.0.1 hb9d3cd8_0 conda-forge
xorg-libxrender 0.9.12 hb9d3cd8_0 conda-forge
xorg-libxscrnsaver 1.2.4 hb9d3cd8_0 conda-forge
zlib 1.3.1 hb9d3cd8_2 conda-forge
zstd 1.5.7 hb8e6e7a_2 conda-forge
### More info
Comment from Sam ;-)
This is a haunting feature of pydantic
Contributor guide
Research direction
Start with the everyvoice.config.validation_helpers:directory_path_must_exist log location and the config/everyvoice-text-to-spec.yaml entry point, then reproduce the issue with the provided everyvoice train text-to-spec command. Done means normal FP training no longer creates the empty preprocessed/YourDataSet directory or logs its creation, while vocoder training remains unaffected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100