1D GAN for Sine Wave function
Generative Adversarial Networks includes a generator model which is capable of generating new plausible fake samples that can be considered to be coming from an existing distribution of samples and a discriminator model that would classify the given sample as real or fake. The model weights of discriminator and the generator are updated according to the performance of the discriminator model.
Function selection
The first step is to select a function to model. In this article we would be using
y = sin(x) where x is the input value and y is the output value of the function. We can use numpy library to define the function.
import numpy as np
def getSine(x):
return np.sin(x)An input domain can be defined as real values between 0 and 10 (0.1,0.2…) and calculate the output for each input value in the domain. We could visualize it by plotting the points.
time = np.arange(0, 10, 0.1)# Amplitude of the sine wave is sine of a variable like time.
amplitude = np.sin(time)# Plot a sine wave using time and amplitude obtained for the sine wave.
plt.plot(time, amplitude)
Discriminator model
The discriminator model takes a sample such as a point in the latent space and predicts whether it is real or fake. As the problem is not complex we would use a simple neural network model for defining our discriminator model. The model will have only one hidden layer with 25 nodes and ReLU activation function along with HE weight initialization. The output layer will have 1 node as it is a binary classification problem and would be using sigmoid activation function. For the compilation, The model would minimize the binary cross-entropy loss function. The discriminator is defined as follows:
# build discriminator model
def build_discriminator(n_inputs):
model = Sequential()
model.add(Dense(25,activation='relu',input_shape=(n_inputs,)))
model.add(Dense(1,activation='sigmoid'))
model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])
model.summary() return modelmodel = build_discriminator(2)
Discriminator model is trained with both real and fake samples. Hence we would create functions to output real and fake samples. The generate_real_samples function would return a real sample and a class variable (1 as it is a real sample)
# defining our real samplesdef gen_real_samples(n):
time = randn(n)
amplitude = np.sin(time)
time = time.reshape(n,1)
amplitude = amplitude.reshape(n,1)
X = hstack((time,amplitude))
y = np.ones((n,1))
return X,y
The generate_fake_samples function would return a fakr sample and a class variable (0 as it is a fake sample).
# defining our fake samplesdef gen_fake_samples(n):
time = randn(n)
amplitude = time*2
time = time.reshape(n,1)
amplitude = amplitude.reshape(n,1)
X = hstack((time,amplitude))y = np.zeros((n,1))return X,y
Next, we would train and evaluate the discriminator model. For each epoch of training, we would generate half batch of real samples and a half batch of fake samples and train the model on both using the train_on_batch function. We can check the model accuracy for both real and fake samples.
# train discriminator modeldef train_discriminator(model,n_epochs,n_batches=128):
half_batches = int(n_batches/2)
for i in range(n_epochs):
X_real,y_real = gen_real_samples(half_batches)
model.train_on_batch(X_real,y_real) X_fake,y_fake = gen_fake_samples(half_batches)
model.train_on_batch(X_fake,y_fake) lossreal,acc_real = model.evaluate(X_real,y_real) lossfake,acc_fake = model.evaluate(X_fake,y_fake) print(i,acc_real,acc_fake)
The discriminator model is trained for 300 epochs.
train_discriminator(model,300)
The model is identifying the real samples with an accuracy of 100% where as around 85% for fake samples.
Generator model
The generator model takes an input from the latent space and generate a new sample. In our case the sample would be a vector of 2 elements (x and sin(x)). The latent space can be defined and becomes meaningful once the generator model updates the points in the space during the training. Once the training is completed, the points in the latent space would correspond to points in the output space. A standard Gaussian distribution has a mean of zero and a standard deviation of one can be used to generate new inputs.
The generator model would be simple like discriminator model and would not have any complex neural networks. The model will have only one hidden layer with 25 nodes and ReLU activation function along with HE weight initialization. The output layer will have 2 nodes and would be using linear activation function as we want to output a vector of real values. The generator model is not fit directly hence it is not compiled directly.
# generator modeldef build_generator(latent_dim,n=2):
model = Sequential()
model.add(Dense(25,activation='relu',input_dim=latent_dim))
model.add(Dense(n,activation='sigmoid')) model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy']) model.summary() return model
We can define points in the latent space by using the randn function which generates random number from a standard gaussian distribution. We then reshape it into samples with n rows and five (our latent dimension in this article) elements per row.
def generate_latent_space(latent_dim,n):
X = randn(latent_dim*n)
X = X.reshape(n,latent_dim)
return XGenerator model then generates new points using the inputs from the above method. The method generate_fake_samples implements it.
def generate_fake_samples(model,latent_dim,n):
X_input = generate_latent_space(latent_dim,n)
X = model.predict(X_input)
y = np.zeros((n,1)) return X,y
Setting the dimension for the latent space as 5 and generating some samples using the generator model.
latent_dim = 5model = build_generator(latent_dim)X,y = generate_fake_samples(model,latent_dim,100)
We can plot the generated samples which are random as the generator model is not trained yet.
plt.scatter(X[:,0],X[:,1])plt.show()
Generator model training
When the discriminator model is relatively good at detecting fake samples then the generator model is updated more whereas when the discriminator model is poor at detecting fake samples, the weights of the generator model is updated less. A new GAN model can be defined which takes a random input, generate samples and provide it as input to the discriminator model and output of the discriminator model is used to update the weight of the generator model.
The discriminator can be trained on real and fake samples in a standalone manner whereas the generator gets updated according to the performance of the discriminator model on detecting fake samples. Therefore the discriminator which is part of the new GAN model is marked as not trainable. Also the generated samples from the generator model are marked as real so the discriminator would detect it as real or fake and give a probability using which during backpropagation process the weights of the generator model is updated to minimize the loss.
# building our gan modeldef build_gan(generator,discriminator):
discriminator.trainable = False
model = Sequential()
model.add(generator)
model.add(discriminator) model.compile(loss='binary_crossentropy',optimizer='adam') return model
Model training would include first updating the discriminator model with real and fake samples, then the generator model in the composite model is updated.
# training our gan modeldef train_gan(gen_model,disc_model,gan_model,latent_dim,n_epochs=100000,n_batch_size=128): half_batch = int(n_batch_size/2) for i in range(n_epochs):
X_real,y_real = gen_real_samples(half_batch)
disc_model.train_on_batch(X_real,y_real)
X_fake,y_fake = generate_fake_samples(gen_model,latent_dim,half_batch)
disc_model.train_on_batch(X_fake,y_fake) # input for gan model
X_gan = generate_latent_space(latent_dim,n_batch_size)
y_gan = np.ones((n_batch_size,1))
gan_model.train_on_batch(X_gan,y_gan) if(i%10000 == 0):
summarize_perf(generator,discriminator,latent_dim,100,i)
GAN Performance Evaluation
We can evaluate the performance of our GAN model by first using the generated samples from the generator model and inspecting them relative to real samples.
def summarize_perf(generator,discriminator,latent_dim,n,epoch): X_real,y_real = gen_real_samples(n)
lossreal,acc_real = discriminator.evaluate(X_real,y_real) X_fake,y_fake = generate_fake_samples(generator,latent_dim,n)
lossfake,acc_fake = discriminator.evaluate(X_fake,y_fake) print(epoch,acc_real,acc_fake) plt.scatter(X_fake[:,0],X_fake[:,1],color='red') plt.scatter(X_real[:,0],X_real[:,1],color='blue') plt.show()
Training our GAN model for around 100,000 epochs and checking the performance of the model after every 10,000 epoch.
# size of the latent space
latent_dim = 5# create a discriminator
discriminator = build_discriminator(2)#create a generator
generator = build_generator(latent_dim)#create the gan
gan_model = build_gan(generator,discriminator)#train the gan model
train_gan(generator,discriminator,gan_model,latent_dim)
The blue points represent the real samples and the red points represents the fake samples generated by our generator model.
The model seems to generate plausible samples in the domain [0,1] adhering to the sin(x) relationship. Further improvements can be done by making the discriminator as well generator model more complex and having a more constrained domain space.
Resources
