magenta / magenta/ddsp

how to update sample_rate of ddsp_run

Open
#355 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
3.4k
Forks
384
PR merge metrics
No merged PRs in 30d

Description

I am using the DDSP train_autoencoder colab notebook here:
https://colab.research.google.com/github/magenta/ddsp/blob/master/ddsp/colab/demos/train_autoencoder.ipynb

I want to train the autoencoder to use 48kHz audio, not 16kHz (default).

To do so, I have made the following changes to sections of the notebook ( --sample_rate=48000)

```
!ddsp_prepare_tfrecord \
--input_audio_filepatterns=$AUDIO_FILEPATTERN \
--output_tfrecord_path=$TRAIN_TFRECORD \
--num_shards=10 \
--sample_rate=48000\
--alsologtostderr
```

Save dataset statistics for timbre transfer

```
from ddsp.colab import colab_utils
import ddsp.training

data_provider = ddsp.training.data.TFRecordProvider(TRAIN_TFRECORD_FILEPATTERN,
sample_rate=48000)
dataset = data_provider.get_dataset(shuffle=False)
PICKLE_FILE_PATH = os.path.join(SAVE_DIR, 'dataset_statistics.pkl')

```

and then for training:

!ddsp_run \
```
--mode=train \
--alsologtostderr \
--save_dir="$SAVE_DIR" \
--gin_file=models/solo_instrument.gin \
--gin_file=datasets/tfrecord.gin \
--gin_param="TFRecordProvider.file_pattern='$TRAIN_TFRECORD_FILEPATTERN'" \
--gin_param="batch_size=16" \
--gin_param="sample_rate=48000" \
--gin_param="train_util.train.num_steps=30000" \
--gin_param="train_util.train.steps_per_save=300" \
--gin_param="trainers.Trainer.checkpoints_to_keep=10"
```

The result is below...I cannot figure out why it doesn't work.
Can someone please tell me what I am doing wrong? Thanks very much in advance!

2021-05-26 01:50:25.950745: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
WARNING:root:Argument whitelist is deprecated. Please use allowlist.
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow_probability/python/internal/variadic_reduce.py:115: calling function (from tensorflow.python.eager.def_function) with experimental_compile is deprecated and will be removed in a future version.
Instructions for updating:
experimental_compile is deprecated, use jit_compile instead
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow_probability/python/internal/variadic_reduce.py:115: calling function (from tensorflow.python.eager.def_function) with experimental_compile is deprecated and will be removed in a future version.
Instructions for updating:
experimental_compile is deprecated, use jit_compile instead
I0526 01:50:29.118762 139803541604224 ddsp_run.py:176] Restore Dir: /content/drive/MyDrive/drum_sample_solo/SHORT-ddsp-solo-instrument
I0526 01:50:29.119161 139803541604224 ddsp_run.py:177] Save Dir: /content/drive/MyDrive/drum_sample_solo/SHORT-ddsp-solo-instrument
I0526 01:50:29.121088 139803541604224 resource_reader.py:50] system_path_file_exists:optimization/base.gin
E0526 01:50:29.121450 139803541604224 resource_reader.py:55] Path not found: optimization/base.gin
I0526 01:50:29.125206 139803541604224 resource_reader.py:50] system_path_file_exists:eval/basic.gin
E0526 01:50:29.125523 139803541604224 resource_reader.py:55] Path not found: eval/basic.gin
I0526 01:50:29.127723 139803541604224 resource_reader.py:50] system_path_file_exists:models/solo_instrument.gin
E0526 01:50:29.127967 139803541604224 resource_reader.py:55] Path not found: models/solo_instrument.gin
I0526 01:50:29.128289 139803541604224 resource_reader.py:50] system_path_file_exists:models/ae.gin
E0526 01:50:29.128505 139803541604224 resource_reader.py:55] Path not found: models/ae.gin
I0526 01:50:29.135938 139803541604224 resource_reader.py:50] system_path_file_exists:datasets/tfrecord.gin
E0526 01:50:29.136190 139803541604224 resource_reader.py:55] Path not found: datasets/tfrecord.gin
I0526 01:50:29.136540 139803541604224 resource_reader.py:50] system_path_file_exists:datasets/base.gin
E0526 01:50:29.136765 139803541604224 resource_reader.py:55] Path not found: datasets/base.gin
I0526 01:50:29.169140 139803541604224 train_util.py:78] Defaulting to MirroredStrategy
2021-05-26 01:50:29.170495: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcuda.so.1
2021-05-26 01:50:29.179300: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-05-26 01:50:29.179911: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties:
pciBusID: 0000:00:04.0 name: Tesla V100-SXM2-16GB computeCapability: 7.0
coreClock: 1.53GHz coreCount: 80 deviceMemorySize: 15.78GiB deviceMemoryBandwidth: 836.37GiB/s
2021-05-26 01:50:29.179946: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
2021-05-26 01:50:29.182908: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublas.so.11
2021-05-26 01:50:29.182982: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublasLt.so.11
2021-05-26 01:50:29.184593: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcufft.so.10
2021-05-26 01:50:29.184967: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcurand.so.10
2021-05-26 01:50:29.186709: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcusolver.so.10
2021-05-26 01:50:29.187344: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcusparse.so.11
2021-05-26 01:50:29.187547: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudnn.so.8
2021-05-26 01:50:29.187663: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-05-26 01:50:29.188345: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-05-26 01:50:29.188900: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
2021-05-26 01:50:29.189253: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX512F
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-05-26 01:50:29.189600: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-05-26 01:50:29.190186: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties:
pciBusID: 0000:00:04.0 name: Tesla V100-SXM2-16GB computeCapability: 7.0
coreClock: 1.53GHz coreCount: 80 deviceMemorySize: 15.78GiB deviceMemoryBandwidth: 836.37GiB/s
2021-05-26 01:50:29.190261: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-05-26 01:50:29.190840: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-05-26 01:50:29.191472: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
2021-05-26 01:50:29.191521: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
2021-05-26 01:50:29.701107: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-05-26 01:50:29.701162: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264] 0
2021-05-26 01:50:29.701171: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0: N
2021-05-26 01:50:29.701381: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-05-26 01:50:29.702050: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-05-26 01:50:29.702700: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-05-26 01:50:29.703268: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:39] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.
2021-05-26 01:50:29.703314: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 13787 MB memory) -> physical GPU (device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:04.0, compute capability: 7.0)
WARNING:tensorflow:Collective ops is not configured at program startup. Some performance features may not be enabled.
W0526 01:50:29.705162 139803541604224 mirrored_strategy.py:379] Collective ops is not configured at program startup. Some performance features may not be enabled.
INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0',)
I0526 01:50:29.708120 139803541604224 mirrored_strategy.py:369] Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0',)
2021-05-26 01:50:30.023877: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:176] None of the MLIR Optimization Passes are enabled (registered 2)
2021-05-26 01:50:30.024463: I tensorflow/core/platform/profile_utils/cpu_utils.cc:114] CPU Frequency: 2000179999 Hz
2021-05-26 01:50:30.091103: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at example_parsing_ops.cc:94 : Invalid argument: Key: audio. Can't parse serialized Example.
2021-05-26 01:50:30.091209: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at example_parsing_ops.cc:94 : Invalid argument: Key: audio. Can't parse serialized Example.
2021-05-26 01:50:30.091287: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at example_parsing_ops.cc:94 : Invalid argument: Key: audio. Can't parse serialized Example.
2021-05-26 01:50:30.091416: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at example_parsing_ops.cc:94 : Invalid argument: Key: audio. Can't parse serialized Example.
2021-05-26 01:50:30.091468: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at example_parsing_ops.cc:94 : Invalid argument: Key: audio. Can't parse serialized Example.
2021-05-26 01:50:30.091860: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at example_parsing_ops.cc:94 : Invalid argument: Key: audio. Can't parse serialized Example.
2021-05-26 01:50:30.092173: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at example_parsing_ops.cc:94 : Invalid argument: Key: audio. Can't parse serialized Example.
2021-05-26 01:50:30.092210: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at example_parsing_ops.cc:94 : Invalid argument: Key: audio. Can't parse serialized Example.
Traceback (most recent call last):
File "/usr/local/bin/ddsp_run", line 8, in
sys.exit(console_entry_point())
File "/usr/local/lib/python3.7/dist-packages/ddsp/training/ddsp_run.py", line 224, in console_entry_point
app.run(main)
File "/usr/local/lib/python3.7/dist-packages/absl/app.py", line 303, in run
_run_main(main, args)
File "/usr/local/lib/python3.7/dist-packages/absl/app.py", line 251, in _run_main
sys.exit(main(argv))
File "/usr/local/lib/python3.7/dist-packages/ddsp/training/ddsp_run.py", line 199, in main
report_loss_to_hypertune=FLAGS.hypertune)
File "/usr/local/lib/python3.7/dist-packages/gin/config.py", line 1069, in gin_wrapper
utils.augment_exception_message_and_reraise(e, err_str)
File "/usr/local/lib/python3.7/dist-packages/gin/utils.py", line 41, in augment_exception_message_and_reraise
raise proxy.with_traceback(exception.__traceback__) from None
File "/usr/local/lib/python3.7/dist-packages/gin/config.py", line 1046, in gin_wrapper
return fn(*new_args, **new_kwargs)
File "/usr/local/lib/python3.7/dist-packages/ddsp/training/train_util.py", line 198, in train
trainer.build(next(dataset_iter))
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/distribute/input_lib.py", line 686, in __next__
return self.get_next()
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/distribute/input_lib.py", line 717, in get_next
2021-05-26 01:50:30.093915: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at example_parsing_ops.cc:94 : Invalid argument: Key: audio. Can't parse serialized Example.
self._iterators[i].get_next_as_list_static_shapes(new_name))
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/distribute/input_lib.py", line 1927, in get_next_as_list_static_shapes
return self._format_data_list_with_options(self._iterator.get_next())
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/data/ops/multi_device_iterator_ops.py", line 585, in get_next
result.append(self._device_iterators[i].get_next())
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/data/ops/iterator_ops.py", line 814, in get_next
return self._next_internal()
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/data/ops/iterator_ops.py", line 747, in _next_internal
output_shapes=self._flat_output_shapes)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gen_dataset_ops.py", line 2728, in iterator_get_next
_ops.raise_from_not_ok_status(e, name)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 6897, in raise_from_not_ok_status
six.raise_from(core._status_to_exception(e.code, message), None)
File "", line 3, in raise_from
tensorflow.python.framework.errors_impl.InvalidArgumentError: Key: audio. Can't parse serialized Example.
[[{{node ParseSingleExample/ParseExample/ParseExampleV2}}]]
[[MultiDeviceIteratorGetNextFromShard]]
[[RemoteCall]] [Op:IteratorGetNext]
In call to configurable 'train' ()

colab_utils.save_dataset_statistics(data_provider, PICKLE_FILE_PATH, batch_size=1)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the train_autoencoder Colab notebook and compare the ddsp_prepare_tfrecord, TFRecordProvider, and ddsp_run settings shown in the issue. Investigate the reported “Key: audio. Can't parse serialized Example” failure; done means the 48 kHz TFRecord dataset is accepted by ddsp_run and training starts without that parsing error.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, python, tensorflow
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.