Tensorflow로 10분만에 만드는 auto-encoder

(http://curiousily.com/data-science/2017/02/02/what-to-do-when-data-is-missing-part-2.html)

Autoencoder는 데이터의 입력 값만 주어진 상태로 학습하는 unsupervised learning의 일종이다. 

그 구성은 input data에 대한 encoder, 그리고 decoder로 구성되어 있다.

최종적으로 신경망은 output data가 input data와 유사하도록 학습을 수행한다.

구조 만큼이나 tensorflow로 만드는 것도 아주 간단하다.

1) 필요한 library import 

학습 데이터로는 역시 만만한 MNIST로 하자 

from __future__ import division, print_function, absolute_import
 
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
 
%matplotlib inline
 
# Import MNIST data
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("/tmp/data/", one_hot=True)

2) Hyper parameter

learning_rate = 0.01
num_steps = 30000
batch_size = 300
 
display_step = 1000
examples_to_show = 10
 
num_hidden_1 = 256
num_hidden_2 = 128
num_input = 784

2개의 hidden layer를 가지는 network로 학습을 할 예정이다.

MNIST는 28 * 28개의 손글씨 데이터이기 때문에, input layer는 784개의 input을 받는다.

3) Network initialize

X = tf.placeholder("float", [None, num_input])
 
weights = {
    'encoder_h1': tf.Variable(tf.random_normal([num_input, num_hidden_1])),
    'encoder_h2': tf.Variable(tf.random_normal([num_hidden_1, num_hidden_2])),
    'decoder_h1': tf.Variable(tf.random_normal([num_hidden_2, num_hidden_1])),
    'decoder_h2': tf.Variable(tf.random_normal([num_hidden_1, num_input])),
}
biases = {
    'encoder_b1': tf.Variable(tf.random_normal([num_hidden_1])),
    'encoder_b2': tf.Variable(tf.random_normal([num_hidden_2])),
    'decoder_b1': tf.Variable(tf.random_normal([num_hidden_1])),
    'decoder_b2': tf.Variable(tf.random_normal([num_input])),
}

여기서 X는 input image다.

4) Encoder 만들기

def encoder(x):
    layer_1 = tf.nn.sigmoid(tf.add(tf.matmul(x, weights['encoder_h1']), biases['encoder_b1']))
    layer_2 = tf.nn.sigmoid(tf.add(tf.matmul(layer_1, weights['encoder_h2']), biases['encoder_b2']))
    return layer_2

Sigmoid activation으로 된 layer를 생성했다.

5) Decoder 만들기

def decoder(x):
    layer_1 = tf.nn.sigmoid(tf.add(tf.matmul(x, weights['decoder_h1']), biases['decoder_b1']))
    layer_2 = tf.nn.sigmoid(tf.add(tf.matmul(layer_1, weights['decoder_h2']), biases['decoder_b2']))
    return layer_2

6) Model 생성

encoder_op = encoder(X)
decoder_op = decoder(encoder_op)
 
y_pred = decoder_op
y_true = X
 
loss = tf.reduce_mean(tf.pow(y_true - y_pred, 2))
optimizer = tf.train.RMSPropOptimizer(learning_rate).minimize(loss)
 
init = tf.global_variables_initializer()

MNIST 원본 image를 가지는 encoder와 decoder를 생성해준다.

Loss는 predicted image와 실제 input image의 차이를 이용해 구하고, 해당 loss를 최소화 하도록 RMSPropOptimizer를 이용했다.

7) Train & test!!

with tf.Session() as sess:
 
    sess.run(init)
 
    # Training
    for i in range(1, num_steps+1):
        batch_x, _ = mnist.train.next_batch(batch_size)
        feed_dict_batch = {X, batch_x}
        _, l = sess.run([optimizer, loss], feed_dict=feed_dict_batch)
        if i % display_step == 0 or i == 1:
            print('Step %i: Minibatch Loss: %f' % (i, l))
 
    # Testing
    n = 4
    canvas_orig = np.empty((28 * n, 28 * n))
    canvas_recon = np.empty((28 * n, 28 * n))
    for i in range(n):
        batch_x, _ = mnist.test.next_batch(n)
        g = sess.run(decoder_op, feed_dict={X: batch_x})
 
        for j in range(n):
            canvas_orig[i * 28:(i + 1) * 28, j * 28:(j + 1) * 28] = \ batch_x_noisy[j].reshape([28, 28])
        for j in range(n):
            canvas_recon[i * 28:(i + 1) * 28, j * 28:(j + 1) * 28] = \ g[j].reshape([28, 28])
 
    print("Original Images")
    plt.figure(figsize=(n, n))
    plt.imshow(canvas_orig, origin="upper", cmap="gray")
    plt.show()
 
    print("Reconstructed Images")
    plt.figure(figsize=(n, n))
    plt.imshow(canvas_recon, origin="upper", cmap="gray")
    plt.show()

Extracting /tmp/data/train-images-idx3-ubyte.gz
Extracting /tmp/data/train-labels-idx1-ubyte.gz
Extracting /tmp/data/t10k-images-idx3-ubyte.gz
Extracting /tmp/data/t10k-labels-idx1-ubyte.gz
Step 1: Minibatch Loss: 0.469423
Step 1000: Minibatch Loss: 0.134342
Step 2000: Minibatch Loss: 0.114325
Step 3000: Minibatch Loss: 0.105298
Step 4000: Minibatch Loss: 0.099428
Step 5000: Minibatch Loss: 0.096570
Step 6000: Minibatch Loss: 0.094484
Step 7000: Minibatch Loss: 0.092524
Step 8000: Minibatch Loss: 0.090487
Step 9000: Minibatch Loss: 0.086699
Step 10000: Minibatch Loss: 0.084158
Step 11000: Minibatch Loss: 0.083585
Step 12000: Minibatch Loss: 0.083244
Step 13000: Minibatch Loss: 0.080697
Step 14000: Minibatch Loss: 0.081696
Step 15000: Minibatch Loss: 0.079015
Step 16000: Minibatch Loss: 0.077121
Step 17000: Minibatch Loss: 0.076649
Step 18000: Minibatch Loss: 0.074011
Step 19000: Minibatch Loss: 0.073590
Step 20000: Minibatch Loss: 0.072406
Step 21000: Minibatch Loss: 0.072786
Step 22000: Minibatch Loss: 0.072392
Step 23000: Minibatch Loss: 0.071469
Step 24000: Minibatch Loss: 0.068318
Step 25000: Minibatch Loss: 0.066830
Step 26000: Minibatch Loss: 0.066004
Step 27000: Minibatch Loss: 0.068503
Step 28000: Minibatch Loss: 0.063444
Step 29000: Minibatch Loss: 0.063924
Step 30000: Minibatch Loss: 0.064071

음.. 이쁘진 않다 ^^;;

8) Denoising

앞서 autoencder의 output을 input image로 맞춰주었는데, 이번엔 noisy data를 input으로 넣고 output을 denoising된 image로 넣어보자.

noise_level = 0.6
x_noisy = tf.placeholder("float", [None, num_input])
 
batch_x_noisy = batch_x + noise_level * np.random.normal(loc=0.0, scale=1.0, size=batch_x.shape)
 

Encode에 들어가는 image를 batch_x_noisy로 바꿔주고, decode로 나오는 predicted data를 원본 image로 바꿔버리면 된다.

결과는?

이정도면 나쁘지 않은 것 같다! ^^;;