Brtdku

I am learning how to train a keras neural network on the MNIST dataset. However, when I run this code, I get only 10% accuracy after 10 epochs of training. This means that the neural network is predicting only one class, since there are 10 classes. I am sure it is a bug in data preparation rather than a problem with the network architecture, because I got the architecture off of a tutorial (medium tutorial). Any idea why the model is not training?

My code:

from skimage import io

import numpy as np

from numpy import array

from PIL import Image

import csv

import random

from keras.preprocessing.image import ImageDataGenerator

import pandas as pd

from keras.utils import multi_gpu_model

import tensorflow as tf

train_datagen = ImageDataGenerator()

train_generator = train_datagen.flow_from_directory(

    directory="./trainingSet",

    class_mode="categorical",

    target_size=(50, 50),

    color_mode="rgb",

    batch_size=1,

    shuffle=True,

    seed=42

)

print(str(train_generator.class_indices) + " class indices")

from keras.models import Sequential

from keras.layers.core import Dense, Dropout, Activation, Flatten

from keras.layers.convolutional import Conv2D

from keras.layers.pooling import MaxPooling2D, GlobalAveragePooling2D

from keras.optimizers import SGD

from keras import backend as K

from keras.layers import Input

from keras.models import Model

import keras

from keras.layers.normalization import BatchNormalization



K.clear_session()

K.set_image_dim_ordering('tf')

reg = keras.regularizers.l1_l2(1e-5, 0.0)

def conv_layer(channels, kernel_size, input):

    output = Conv2D(channels, kernel_size, padding='same',kernel_regularizer=reg)(input)

    output = BatchNormalization()(output)

    output = Activation('relu')(output)

    output = Dropout(0)(output)

    return output



model = Sequential()

model.add(Conv2D(28, kernel_size=(3,3), input_shape=(50, 50, 3)))

model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Flatten()) # Flattening the 2D arrays for fully connected layers

model.add(Dense(128, activation=tf.nn.relu))

model.add(Dropout(0.2))

model.add(Dense(10, activation=tf.nn.softmax))





from keras.optimizers import Adam

import tensorflow as tf



model.compile(loss='categorical_crossentropy',

              optimizer='adam',

              metrics=['accuracy'])

from keras.callbacks import ModelCheckpoint



epochs = 10



checkpoint = ModelCheckpoint('mnist.h5', save_best_only=True)



STEP_SIZE_TRAIN=train_generator.n/train_generator.batch_size

model.fit_generator(generator=train_generator,

                    steps_per_epoch=STEP_SIZE_TRAIN,

                    epochs=epochs,

                    callbacks=[checkpoint]

)

The output I am getting is as follows:

Using TensorFlow backend.

Found 42000 images belonging to 10 classes.

{'0': 0, '1': 1, '2': 2, '3': 3, '4': 4, '5': 5, '6': 6, '7': 7, '8': 8, '9': 9} class indices

Epoch 1/10

42000/42000 [==============================] - 174s 4ms/step - loss: 14.4503 - acc: 0.1035

/home/ec2-user/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/keras/callbacks.py:434: RuntimeWarning: Can save best model only with val_loss available, skipping.

  'skipping.' % (self.monitor), RuntimeWarning)

Epoch 2/10

42000/42000 [==============================] - 169s 4ms/step - loss: 14.4487 - acc: 0.1036

Epoch 3/10

42000/42000 [==============================] - 169s 4ms/step - loss: 14.4483 - acc: 0.1036

Epoch 4/10

42000/42000 [==============================] - 168s 4ms/step - loss: 14.4483 - acc: 0.1036

Epoch 5/10

42000/42000 [==============================] - 169s 4ms/step - loss: 14.4483 - acc: 0.1036

Epoch 6/10

42000/42000 [==============================] - 168s 4ms/step - loss: 14.4483 - acc: 0.1036

Epoch 7/10

42000/42000 [==============================] - 168s 4ms/step - loss: 14.4483 - acc: 0.1036

Epoch 8/10

42000/42000 [==============================] - 168s 4ms/step - loss: 14.4483 - acc: 0.1036

Epoch 9/10

42000/42000 [==============================] - 168s 4ms/step - loss: 14.4480 - acc: 0.1036

Epoch 10/10

 5444/42000 [==>...........................] - ETA: 2:26 - loss: 14.3979 - acc: 0.1067

The trainingSet directory contains a folder for each 1-9 digit with the images inside the folders. I am training on an AWS EC2 p3.2xlarge instance with the Amazon Deep Learning Linux AMI.

Here is the list of some weird points that I see :

• Not rescaling your images -> ImageDataGenerator(rescale=1/255)
  


• Batch Size of 1 (You may want to increase that)


• MNIST is grayscale pictures , therefore color_mode should be "grayscale".

(Also you have several unused part in your code, that you may want to delete from the question)

Adding two more point in answer of @abcdaire,

1. 
   mnist has image size of (28,28), you have assigned it wrong.


2. 
   Binarization is another method, which can be used. It also make network to learn fast. It can be done like this.

`

imges_dataset = imges_dataset/255.0

imges_dataset = np.where(imges_dataset>0.5,1,0)