Back to Main Page

GAN Improvement

Here we present the improvement between without and with GAN in the proposed model. Without GAN, the model will suffer from "over-smoothing" problem, where the model generates uniform and smoothed harmonic distribution and noise magnitudes over the whole note. See below for an intuitive evaluation.

Ground-truth Harmonic Distribution

Violin to Guitar

Predicted Harmonic Distribution without GAN

Violin to Guitar

With the introduction of adversarial training, the proposed model overcomes the over-smoothing problem potentially caused by one-to-many mapping.

Predicted Harmonic Distribution with GAN

Violin to Guitar

One can also hear the over smoothing problem from predicted samples:

Ground-truth With GAN Without GAN

Ground-truth

Violin to Guitar

Without GAN

Violin to Guitar

With GAN

Violin to Guitar

Similar effects can also be seen in noise magnitudes:

Ground-truth Noise Magnitudes

Violin to Guitar

Predicted Noise Magnitudes without GAN

Violin to Guitar

Predicted Noise Magnitudes with GAN

Violin to Guitar