论文笔记：unsupervised representation learning with deep convolutional generative adversarial networks
1. previous work [generative adversarial nets]
paper link: http://arxiv.org/pdf/1406.2661v1.pdf
torch implementation: https://github.com/soumith/dcgan.torch
a. motivation: supervised network requires huge labeled data ==> generative net generates samples and helps to build unsupervised models based on CNN (Markov Chain no longer needed) ==> Adversarial nets (GANs)
b. architecture:
Generative net (G(z)):
input Z: noise vector; output: fake image G(z) and be fed to Discriminator D(x)
Discriminative net D(x):
a CNN that classifies true image X as 1 and fake image G(z) as 0
c. Training algorithm:
i. for K steps: train D to maximize (G keeps still), estimating probability by fitting a gaussian parzen window
where x(i) is true image; z(i) is noise vector, D(x) is Prob(x is true image)
ii. D keeps still, train G to maximize
iii. repeat step 1, 2 (G can't be trained too much without updating D, otherwise G will collapse too many z to same x)
2. Changes in this paper (GANs unstable: generates incomprehensible images and noise)
a. use strided convolutions instead of any spatial pooling ==> allowing nets to learn downsampling
b. use batch normalization ==> useful to prevent G from collapsing all z into same x
c. use global average pooling instead of FC ==> increase stability but hurt convergence speed
d. use leaky ReLU in D(x) for all layers ==> useful for higher resolution modeling
e. *might be useful: sample deduplication: 3072-128-3072 dropout regularized ReLU autoencoder
3. Performance:
Discriminator:
Generator:
