MelGAN is an exciting development in audio waveform generation using a GAN setup. It is a fully convolutional feed-forward network that takes a mel-spectrogram as input and outputs raw waveform.
What is Mel-spectrogram?
A mel-spectrogram represents the frequency content of a signal at different points in time. In other words, it is a visual representation of sound that shows how much energy is present in a particular frequency band at a particular time. The y-axis of a mel-spectrogram represents frequency, while the x-axis represents time. The color of a pixel at a given time and frequency corresponds to the energy present in that frequency band at that point in time.
How does MelGAN work?
MelGAN uses a stack of transposed convolution layers to upsample the mel-spectrogram. Each of these layers is followed by a stack of residual blocks with dilated convolutions. This helps to increase the temporal resolution of the input sequence to match the output waveform.
One of the key differences between MelGAN and other GANs is that it does not use a global noise vector as its input. Instead, it uses the mel-spectrogram as its input.
Dealing with Checkerboard Artifacts
In audio waveform generation, a common problem is the presence of 'checkerboard artifacts' in the output. These artifacts occur when the generator produces high-frequency oscillations in the waveform that are not present in the original signal. This results in a distorted sound that is unpleasant to the ear.
To address this issue, MelGAN uses kernel-size as a multiple of stride instead of PhaseShuffle.
Normalization and Discrimination
Weight normalization is used in MelGAN for normalization. This technique helps to improve the stability and convergence of the generator during training.
In terms of discrimination, MelGAN uses a window-based discriminator similar to a PatchGAN. This type of discriminator evaluates the discriminator score across different regions of the generated waveform.
MelGAN is an impressive development in the field of audio waveform generation. Its ability to generate high-quality audio signals has significant implications for many areas of sound processing, such as speech recognition, music production, and audio compression.
