Exploring GAN: Understanding the Basics and Latest Techniques.由浅入深理解GAN是什么？常见的GAN结构。最新GAN方法

Introduction

GAN (Generative Adversarial Networks) is a type of neural network architecture that consists of two parts: a generator and a discriminator.

GAN（生成对抗网络）是一种神经网络架构，由两部分组成：一个生成器和一个鉴别器。

The generator creates new data samples that are similar to the training data, while the discriminator tries to distinguish between the generated samples and the real samples from the training data.

生成器创建与训练数据相似的新数据样本，而鉴别器则试图区分生成的样本和训练数据中的真实样本。

During training, the generator generates new samples and the discriminator classifies them as either real or fake. The discriminator's feedback is then used to update both the generator's and discriminator's parameters.

在训练过程中，生成器产生新的样本，鉴别器将它们分类为真实或虚假的样本。然后，鉴别器的反馈被用来更新生成器和鉴别器的参数。

As the training progresses, the generator learns to generate more realistic samples that can fool the discriminator, while the discriminator learns to distinguish between the real and fake samples more accurately.

随着训练的进行，生成器学会生成更真实的样本，从而骗过判别器，而判别器则学会更准确地区分真假样本。

In other words, the generator and the discriminator are engaged in a two-player minimax game, where the generator tries to produce samples that can fool the discriminator, and the discriminator tries to correctly identify the fake samples.

换句话说，生成器和鉴别器正在进行一场双人的最小化游戏，生成器试图生成可以欺骗鉴别器的样本，而鉴别器则试图正确识别假样本。

The end goal of the training process is for the generator to be able to generate realistic data samples that are indistinguishable from the real data, as judged by the discriminator.

训练过程的最终目标是让生成器能够生成与真实数据无法区分的真实数据样本，这一点由鉴别器来判断。

生成器结构Structure of Generator

1. Generators based on fully connected layers: the simplest generator structure, consisting of multiple fully connected layers, can generate simple image and sequence data.

   基于全连接层的生成器：最简单的生成器结构，由多个全连接层组成，可以生成简单的图像和序列数据。

2. Generators based on convolutional neural networks: Convolutional neural networks are an effective tool for processing image data, so many generators use a convolutional neural network structure, such as DCGAN, which consists of multiple convolutional and deconvolutional layers for converting random noise into a realistic image.

   基于卷积神经网络的生成器：卷积神经网络是处理图像数据的有效工具，因此很多生成器采用了卷积神经网络结构，例如 DCGAN，该结构包括多个卷积层和反卷积层，用于将随机噪声转化为逼真的图像。

3. Recurrent neural network-based generators: Suitable for generating sequential data, recurrent neural networks can process time-series data and make use of contextual information in their generation, such as the Seq2Seq model and Variational Autoencoder.

   基于循环神经网络的生成器：适用于生成序列数据，循环神经网络可以处理时间序列数据，并在生成时利用上下文信息，例如 Seq2Seq 模型和 Variational Autoencoder。

4. Transfer matrix-based generators: Transfer matrices are a special type of neural network that can handle the generation of images, sequences and graphs, such as PixelCNN and PixelRNN.

   基于转移矩阵的生成器：转移矩阵是一种特殊的神经网络，可以处理图像、序列和图的生成，例如 PixelCNN 和 PixelRNN。

5. Self-encoder based generators: Self-encoders can encode input data into a smaller dimensional representation and decode it back into the original data, and can therefore be used to generate data. For example, DCGAN in VAE and GAN is an improvement from self-encoders.

   基于自编码器的生成器：自编码器可以将输入数据编码为较小的维度表示，并将其解码回原始数据，因此可以用于生成数据。例如 VAE 和 GAN 中的 DCGAN 就是由自编码器改进而来的。

判别器结构Structure of Discriminator

The common structure of the discriminator is largely similar to that of the generator. In a specific task, the generator and discriminator usually adopt similar neural network structures, but they often have different architectures and parameters. Generally speaking, the generator's architecture needs to generate realistic data in a random noise space, while the discriminator's architecture needs to distinguish between the generated data and the real data, which requires them to have different functions and performance. However, sometimes the same network structure can be used for both the generator and the discriminator, as in DCGAN.

判别器的常见结构和生成器大体相同，在一个特定的工作中，生成器和判别器通常采用相似的神经网络结构，但它们往往具有不同的架构和参数。一般来说，生成器的架构需要在随机噪声空间中生成逼真的数据，而判别器的架构需要将生成的数据与真实数据区分开，这就要求他们具有不同的功能和表现。但有时候也会使用相同的网络结构。如DCGAN。

GAN与NeRF

In recent work, NeRF has shown excellent performance in view synthesis. Therefore, in EG3D, the neural rendering method of NeRF is also used as part of the generator.

在最近的工作中，NeRF表现出很好的视图合成效果。因此在EG3D中也讲NeRF的神经渲染方法用做生成器的一部分。