基于MindSpore实现WaveNet语音合成应用案例优化

[superjingshen]

任务描述

基于 MindSpore 2.7.1 版本 + 动态图 + mint 接口实现 WaveNet 语音合成应用案例的整改。

修改内容

1. 新增 train_wavenet_audio_generation.ipynb：包含模型训练+推理的完整动态图实现流程。
2. 删除旧版 wavenet.ipynb。
3. 更新 README.md 中的案例列表。

关联 Issue

Fixes #ID9HTU

[moyu026]

运行报错：

RuntimeError Traceback (most recent call last)
Cell In[6], line 23
21 print("dataset loading ...")
22 dataset = create_dataset(dataset_file=dataset_file, wave_location=wave_location, batch_size=batch_size, classes=classes, output_length=output_length, receptive_feild=model.receptive_field, sampling_rate=16000, mono=True)
---> 23 print("dataset loaded.\n\tdataset size: {:d}\n\tbatch size: {:d}".format(dataset.get_dataset_size(), dataset.get_batch_size()))
25 # 开始训练
26 for t in range(epochs):

File /usr/local/python3.10.14/lib/python3.10/site-packages/mindspore/dataset/engine/iterators.py:74, in _cleanup_the_iterators_if_created..wrapper(self, *args, **kwargs)
70 @wraps(method)
71 def wrapper(self, *args, **kwargs):
72 original_iterators = deepcopy(ITERATORS_LIST)
---> 74 result = method(self, *args, **kwargs)
76 # it is used to attribute function like: dataset_size / output_shapes / output_types and
77 # it is a GeneratorDataset with two stage pipeline. The first pipeline will create a new iterator
78 # which need to be released after dataset_size / output_shapes / output_types end.
79 # 1. find the iterators which are started by dataset_size / output_shapes / output_types with two stage pipeline
80 iterators_to_be_released = []

File /usr/local/python3.10.14/lib/python3.10/site-packages/mindspore/dataset/engine/datasets.py:1854, in Dataset.get_dataset_size(self)
1833 """
1834 Return the number of batches in an epoch.
1835
(...)
1851 66
1852 """
1853 if self.dataset_size is None:
-> 1854 runtime_getter = self.__init_size_getter()
1855 self.dataset_size = runtime_getter[0].GetDatasetSize(False)
1856 if self.dataset_size == 0:

File /usr/local/python3.10.14/lib/python3.10/site-packages/mindspore/dataset/engine/datasets.py:1713, in Dataset.__init_size_getter(self)
1709 def __init_size_getter(self):
1710 """
1711 Get pipeline information.
1712 """
-> 1713 ir_tree, api_tree = self.create_ir_tree()
1715 runtime_context = cde.PythonRuntimeContext()
1716 runtime_context.Init()

File /usr/local/python3.10.14/lib/python3.10/site-packages/mindspore/dataset/engine/datasets.py:342, in Dataset.create_ir_tree(self, getter_mode)
340 global _OP_NAME
341 _OP_NAME = Dataset._get_operator_id(dataset)
--> 342 ir_tree = dataset.parse_tree(getter_mode)
343 self.parent = parent
344 _init_device_info()

File /usr/local/python3.10.14/lib/python3.10/site-packages/mindspore/dataset/engine/datasets.py:372, in Dataset.parse_tree(self, getter_mode)
370 if len(self.parent) > 1:
371 raise ValueError("The data pipeline is not a tree (i.e., one node has 2 consumers)")
--> 372 ir_children = [d.parse_tree(getter_mode) for d in self.children]
373 # Bootstrap can only be performed on a copy of the original dataset node.
374 # Bootstrap on original dataset node will make all iterators share the same process pool
375 self.pre_parse(getter_mode)

File /usr/local/python3.10.14/lib/python3.10/site-packages/mindspore/dataset/engine/datasets.py:372, in (.0)
370 if len(self.parent) > 1:
371 raise ValueError("The data pipeline is not a tree (i.e., one node has 2 consumers)")
--> 372 ir_children = [d.parse_tree(getter_mode) for d in self.children]
373 # Bootstrap can only be performed on a copy of the original dataset node.
374 # Bootstrap on original dataset node will make all iterators share the same process pool
375 self.pre_parse(getter_mode)

File /usr/local/python3.10.14/lib/python3.10/site-packages/mindspore/dataset/engine/datasets.py:377, in Dataset.parse_tree(self, getter_mode)
375 self.pre_parse(getter_mode)
376 self.iterator_bootstrap()
--> 377 ir_node = self.parse(ir_children)
378 ir_node = self.post_parse(ir_node)
379 return ir_node

File /usr/local/python3.10.14/lib/python3.10/site-packages/mindspore/dataset/engine/datasets_user_defined.py:1096, in GeneratorDataset.parse(self, children)
1094 self.prepare_multiprocessing()
1095 if self.schema is None:
-> 1096 return cde.GeneratorNode(self.prepared_source, self.column_names, self.column_types, self.source_len,
1097 self.sampler, self.num_parallel_workers, self.sample_fn, self.has_batch_sampler)
1098 schema = self.schema
1099 if isinstance(schema, Schema):

RuntimeError: Syntax error.

---

• Dataset Pipeline Error Message:

---

[ERROR] GeneratorNode: data row of input source must not be 0, got: 0

---

• C++ Call Stack: (For framework developers)

---

mindspore/ccsrc/minddata/dataset/data_source/operation/generator_node.cc(134).

[superjingshen]

老师，您好，

我又在云平台用一个epoch测试了一次，没有问题，有没有可能是dataset/audio.wav 暂时没被读取到。产生了错误了缓存文件，可以把之前产生的dataset.npz、pred_head.npz、pred_head.wav、train_audio.wav删除了，然后重新运行嘛,mindspore是2.7.1，然后按照群里说的用的新的mindhf

[moyu026]

使用MindSpore 2.7.1 版本可以运行

[moyu026]

请参考这个模板再修改下格式，https://github.com/BQBBLZ/applications/blob/master/ai_x/train_resnet_classification.ipynb

[superjingshen]

老师您好，已修改格式，请查收

[superjingshen]

老师，运行结果已按要求删除，辛苦查收

[superjingshen]

好的，老师，已经修改了数据处理介绍的那块的位置

[xing-yiren]

1. 其实不用把audio下的README翻译成中文的，但辛苦了，写得很好，我看看考虑后面可以保留一份中文的README
2. 环境准备这里，现在框架默认走动态图，所以这行代码其实是不需要写的

3. 模型训练这里，`ops.depend`接口是旧的写法了，现在可以直接`optimizer(grads)`

其他的没有了，很棒，完成得挺好的

[superjingshen]

嗯呢，已经修改好了。

[xing-yiren]

@moyu026 登金辛苦这个新改动后的版本再验证下运行是否有问题，其他的我这边没问题了，辛苦，很棒

[moyu026]

修改后的版本可以运行

[xing-yiren]

okie啦，写的挺好的，赞，我这边没什么问题了，辛苦 @wang-hua-2019 也把把关