TensorFlow iOS and Android Apps to Detect Dog Breeds
MIT License
"What Dog?" is a mobile app based on the TensorFlow example classification app with a new model trained to detect dog breeds.
Yes, there's already one from the guys at Microsoft and probably numerous others, but the Microsoft one simply submits an image to a web service.
Unlike theirs, I wanted my app to do offline on-device inference with my trained model.
Plus this serves as an open source example for any other Udacity AIND students who want to test their models outside on RL dogs.
I did this project as part of the Udacity - Artificial Intelligence Nanodegree on CNNs and transfer learning in Keras. You can read more about the project in my dog_app.ipynb Jupyter Notebook.
The model was based on Google's InceptionV3 model and retrained using Transfer Learning. I followed some suggestions for Transfer Learning and Fine Tuning from @dlsandbox on his Github.
After an afternoon of playing around with an AWS p2.xlarge I got the model to an accuracy of 87.2% on the test set.
Once the model was trained and working, I wanted to test it on real dogs outside, so I figured I would need to run it on my phone.
@timanglade Wrote about his experience making the HBO show Silicon Valley's Not Hotdog App and deploying a Keras model to mobile. I also found this great guide by @stratospark on doing a similar Keras to TensorFlow app for Food Detection.
Inspired by the Not Hotdog App and food-101 guides, I decided to use TensorFlow SDK to make "What Dog?".
Unfortunately there was no simple instructions on how to get my model from Keras into a TensorFlow pb (protobuff) file ready for on-device inference.
After speaking with both @timanglade and @stratospark on Twitter, reading lots of TensorFlow GitHub issues and modifying the useful keras_to_tensorflow tool, I managed to get it working.
This should just work if your model is based off the Google InceptionV3 model, or from my Jupyter Notebook, otherwise you might need to make some adjustments.
Inside the ios folder run:
$ pod install
There plenty of bugs, this is barely Beta quality. The model is large and real time inference takes a lot of CPU power. The app struggles to run on my Sony Xperia Z3 Compact. It runs much better with something like an iPhone 6 or Samsung Galaxy 6.
If the app isn't working properly just quit it and open it again. Sometimes the inference gets stuck or confused by certain images.
Pull Requests are welcome.
Thanks to everyone who wrote guides, code, or answered my questions on Twitter. Big thanks to Udacity for the AIND and Google for making TensorFlow!
The icon graphics used for "What Dog?" are from flaticon.com.
Icons made by Freepik from www.flaticon.com is licensed by CC 3.0 BY