IS2020 Online Supplement

Authors

Zhijun Liu
sorcerer~at~sjtu.edu.cn

Kuan Chen
azrealkuan~at~sjtu.edu.cn

Kai Yu
kai.yu~at~sjtu.edu.cn

This jupyter notebook demonstrates the importance of phase in speech.
The Chinese Standard Mandarin Speech Copus is used for the demo.
All generated samples can be found on the google drive.
The analysis of computational complexity can be found in this jupyter notebook.
Further training details can be found in this jupyter notebook.

Copy Synthesis Demos

NHV(cGAN) is the model described in the paper and used in evaluation. NHV(GAN) was trained with the same discriminator used in b-NSF-adv. They have slightly different loss in audio quality.

DDSP(Sinusoids + Noise) uses the same network structure and inputs as NHV models. The output of the networks are replaced with `harmonic distributions` and `noise filter FFT amplitudes` as described in the DDSP paper. The F0 is encoded with 80 dimensional one-hot vectors in log-scale. This model was not used in evaluations.

DDSP(S+N, cGAN) is the same as DDSP(S+N). Except that it is trained with cGAN. The GAN structure and hyper-parameters are the same as that used in NHV(cGAN).

程长庚#2，生在#1清王朝#1嘉庆#1年间#3，安徽的#1潜山#1小县#4。
Original	NHV(GAN)	NHV(cGAN)

Parallel WaveGAN	NSF-adv	MoL WaveNet

hn-sinc-NSF	NHV-noadv	DDSP(Sinusoids + Noise)

DDSP(S+N, cGAN)

不管#1哈大爷#1去哪#3，它都#1一步#1不离地#1跟着#4。
Original	NHV(GAN)	NHV(cGAN)

Parallel WaveGAN	NSF-adv	MoL WaveNet

hn-sinc-NSF	NHV-noadv	DDSP(Sinusoids + Noise)

DDSP(S+N, cGAN)

再说#3，关于#1利益#1上的事#2俺俩#1都#1不好#1开口#4。
Original	NHV(GAN)	NHV(cGAN)

Parallel WaveGAN	NSF-adv	MoL WaveNet

hn-sinc-NSF	NHV-noadv	DDSP(Sinusoids + Noise)

DDSP(S+N, cGAN)

路上#1关卡#1很多#3，为了#1方便#1撤离#3，只好#1轻装#1前进#4。
Original	NHV(GAN)	NHV(cGAN)

Parallel WaveGAN	NSF-adv	MoL WaveNet

hn-sinc-NSF	NHV-noadv	DDSP(Sinusoids + Noise)

DDSP(S+N, cGAN)

直至#1公元前#1一万#1一千年#2，它又#1再次#1出现#4。
Original	NHV(GAN)	NHV(cGAN)

Parallel WaveGAN	NSF-adv	MoL WaveNet

hn-sinc-NSF	NHV-noadv	DDSP(Sinusoids + Noise)

DDSP(S+N, cGAN)

从五#1到七#3，前后#2也就是#1六个#1月的#1时间#4。
Original	NHV(GAN)	NHV(cGAN)

Parallel WaveGAN	NSF-adv	MoL WaveNet

hn-sinc-NSF	NHV-noadv	DDSP(Sinusoids + Noise)

DDSP(S+N, cGAN)

漫天的#1红霞#2使劲给#1两人#1增添#1气氛#4。
Original	NHV(GAN)	NHV(cGAN)

Parallel WaveGAN	NSF-adv	MoL WaveNet

hn-sinc-NSF	NHV-noadv	DDSP(Sinusoids + Noise)

DDSP(S+N, cGAN)