An expression training App that helps users mimic their own expressions.
MIT License
Contact: Jun-Yan Zhu (junyanz at cs dot cmu dot edu)
This is the authors' implementation of Expression Training App described in: Mirror Mirror: Crowdsourcing Better Portraits Jun-Yan Zhu, Aseem Agarwala, Alexei A. Efros, Eli Shechtman and Jue Wang In ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2014)
This App is used for training users to mimic their best expressions. The App takes input from a webcam/video and displays the current expression along with its attractiveness and seriousness scores, computed in real-time. The user can specify a target expression to mimic. The App then shows three windows; the current expression, the target expression, and an aligned and a blended cross-fade between the two. The cross-fade oscillates between the target and current expression once per two seconds, so that the subject can examine differences between the two expressions. See Section 6 in the original paper for details.
This software includes two modules: (1) Learning attractive/serious SVM models given expressions and annotations. (2) Expression Training App.
The code can only be used for non-commercial purposes. Please cite our paper if you use our code and data for your research.
An example for learning attractive/serious svm models: ./MirrorMirror -m 0 ./data/test/ ./data/models/
An example for Expression Training App using trained model and video file: ./MirrorMirror -m 1 -i ./data/video.avi ./data/target.png ./data/models/
An example for Expression Training App using cross-subject model and webcam: ./MirrorMirror -m 1 your/target/image/ ./models/svm/
Command line interface: ./MirrorMirror [-v verbose] [-m mode] [-i input_video] [-s save_dir] data_file model_dir
-v verbose: shows details about the runtime output (default = 1)
-m mode: set the training mode (default = 0)
-i input_video: specify the video input; if no video is specified, the program will use a webcam.
-s save_dir: specify the directory where the results will be saved.
data_file: set the directory/path of the input data:
model_dir: set the directory for svm models.
The program displays three windows:
User Interaction: "MirrorMirror" window takes the user's keyboard input.
The training data directory should contain: (1) /imgs/: expression images. (2) /scores/: attractive/serious scores (3) /image.txt: image list. See "./data/test/" for an example.