Image Classification tests scripts with Keras and tensorflow

This work was based on the following Keras blog Post : https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html

The purpose was to evaluate image Classification with a small data set, and compare it with an SVM approach. However, and unlike the blog post, I wanted to evaluate this approcah on a multi class problem.

I won't share the business case of this work, only the models that were used, and some scripts made to evaluate image Augmentation impact on those.

Feel free to take those as a first step, and improove them by trying and error. I am no data scientist at all, the purpose there was just to experiment and see the tools in action 🙈 🙉 🙊

TensorFlow Environment installation

You will find complete installation instructions there : https://www.tensorflow.org/install/install_mac

As advised, I used the following steps to build my virtualEnv installation on a mac :

Start a terminal (a shell). You'll perform all subsequent steps in this shell.
Install pip and Virtualenv :

 $ sudo easy_install pip
 $ pip install --upgrade virtualenv

Create a Virtualenv environment :

 $ virtualenv --system-site-packages -p python3 targetDirectory

where targetDirectory identifies the top of the Virtualenv tree.

Activate the Virtualenv environment :

$ cd targetDirectory
$ source ./bin/activate      # If using bash, sh, ksh, or zsh

The preceding source command should change your prompt to the following:

 (targetDirectory)$

Ensure pip 8.1 is installed:

 (targetDirectory)$ easy_install -U pip

Issue the following command to install TensorFlow and all the packages that TensorFlow requires into the active Virtualenv environment:

 (targetDirectory)$ pip3 install --upgrade tensorflow

Keras installation

Inside your virtualenv :

 (targetDirectory)$ pip3 install keras

You will also need pillow for image modification and h5py for saving models for thoses scripts to run :

 (targetDirectory)$ pip3 install Pillow
 (targetDirectory)$ pip3 install h5py

Tensorboard installation

Using Tensorboard was a good way to have a visual comparison between iterations.

Inside your virtualenv :

 (targetDirectory)$ pip3 install tensorboard

Be aware that in order to launch Tensorboard, you to do it from tensoarboard directory directly like this :

$ cd targetDirectory/lib/python3.6/site-packages/tensorboard/
$ tensorboard --logdir ~/[YOUR_LOG_DIR]

Then go to tensorbaord URL with your favorite browser : http://localhost:6006

Scripts description and usage

For all those samples, please ensure that directories exist before launching scripts.

Image augmentation

UnitaryImageAugmentationSample This sript demonstrate a simple image augmentation method from one image
ClassImageAugmentationSample This script use the same approcah, but directory based, so you can make equivalent classes in temr of numbers of sample .

Models

FirstModel
A model that use dropout only on the activation layer, and image augmentation technique

SecondModel
A model that use dropout on each layer, and class weights representation. No augmentation on this test.

Results

The results on my test data, composed of 341 images belonging to 4 classes for training and 87 images belonging to 4 classes for validation, are the following after 50 epochs :

FirstModel (in Blue) : loss: 0.7886 - acc: 0.7267 - val_loss: 0.4058 - val_acc: 0.8800
SecondModel (in Red) : loss: 0.0470 - acc: 0.9867 - val_loss: 1.8659 - val_acc: 0.8200

(Runs took 46mn / 48 mn on a macbook air 1,3 GHz Intel Core i5)

As you can see, the second approach gives better theorical results, but loss is very fuzzy on validation set, meaning that the model is not stable at all from my point of vue.