For many GAN beginners in practice, they may wonder why the Loss of GAN can’t go down all the time. When does GAN converge? In fact, as a well-trained GAN, its Loss cannot be reduced. To measure whether GAN is well trained, only human eyes can see whether the quality of the generated picture is good. However, for the problem that there is no good evaluation of convergence index, many scholars have done some studies. WGAN mentioned in the following paper proposed a new Loss design method, which better solved the problem that it is difficult to judge convergence. Let’s analyze why GAN Loss can’t go down. For the discriminator, GAN’s Loss is as follows:
As you can see, the generator and the discriminator have opposite purposes, that is, the two generator networks and the discriminator network work against each other, one against the other. Loss cannot go all the way down to a convergent state. For generators, Loss drops quickly, most likely because the discriminator is so weak that the generator can easily “fool” the discriminator. For the discriminator, the rapid decline of Loss means that the discriminator is very strong. If the discriminator is very strong, it means that the image generated by the generator is not true enough, which makes it easy for the discriminator to discriminate, leading to the rapid decline of Loss. That is, whether it’s a discriminator or a generator. Loss is not a good or bad generator. In a good GAN network, the GAN Loss tends to fluctuate constantly. It can be a little frustrating to see here, as if the only way to judge whether the model converges is by the quality of the images generated. In fact, the WGAN discussed in the following paper puts forward a new measurement method of Loss, which enables us to judge whether the model converges by certain means.