Neural Network to predict face touch on live feed and warn you, "don't touch the face".
GPL-3.0 License
The code is based(mostly all NN code) on the ultralytics/yolov3 and is freely available for redistribution under the GPL-3.0 license. The original repository is licensed under GPL-3.0, hence the license is GPL-3.0.
The neural network to detect human face touch in an image or video stream using Yolo V3. The idea comes from Keras creator Francois Chollet tweet.
Here is the network prediction on a clip of Slavoj Zizek
Source: https://www.youtube.com/embed/YvJocp4Ovzg
Network output on the first ten second clip of the original video - https://www.youtube.com/embed/n44WsmRiAvY
Prediction on set of 17 images
conda env create -f environment.yml
. This is for Mac OSX.conda activate facetouch
.conda install cython numpy
pip install -r requirements.txt
.If you issue with installation, all you need is the following libraries,
pytorch, torchvision, matplotlib, pycocotools, numpy, opencv-python, opencv-python-headless
.
TODO: Add the dockerfile
gdown https://drive.google.com/uc?id=16lYS4bcIdM2HdmyJBVDOvt3Trx6N3W2R
.python detect.py --cfg cfg/yolov3-1cls.cfg --weights best_dataset_v5.pt --source 0
.python detect.py --cfg cfg/yolov3-1cls.cfg --weights best_dataset_v5.pt --source test_images
.output
directory.gdown https://drive.google.com/uc?id=1ziZ0Bw40KE4pfxezyjq7FMzOntDNNzuu
.unzip dataset_v5.zip
.gdown https://drive.google.com/uc?id=16lYS4bcIdM2HdmyJBVDOvt3Trx6N3W2R
.python train.py --weights yolov3-spp.weights --data dataset_v5/coco1.data --cfg cfg/yolov3-1cls.cfg
.weights
directory.best.pt
is the weights
is used for later inference.To download the images for training, you can run the file face_touch_download.py
like
python face_touch_download.py --keyword "chin leaning" --limit 500 --dest chin_leaning
.
The above command opens the firefox web browser, sends the keyword "chin leaning"
to google, downloads first 500 images to the directory chin_leaning
.
You need to have Firefox Selenium plugin.
To annotate the downloaded images, you can use LabelImg.
Install pip install labelimg
. You can annotate the images like labelimg <images> coco1.names
.
The annotation will be stored in the same directory. Do it for all images directories.
Now there are images and annotation. Create the dataset using following command,
$python dataset.py --directories face_touch --directories chin_leaning --name dataset_v5
.
This will create dataset_v5
directory and zip file with labels
and images
directories along with train and valid split.
$ls -lat dataset_v5
total 128
drwxr-xr-x 59 user staff 1888 Mar 15 22:02 ..
-rw-r--r-- 1 user staff 91 Mar 15 17:24 coco1.data
drwxr-xr-x 9 user staff 288 Mar 15 17:24 .
-rw-r--r-- 1 user staff 10 Mar 15 17:24 coco1.names
-rw-r--r-- 1 user staff 4819 Mar 15 17:24 test.txt
-rw-r--r-- 1 user staff 7051 Mar 15 17:24 val.txt
-rw-r--r-- 1 user staff 36893 Mar 15 17:24 train.txt
drwxr-xr-x 5 user staff 160 Mar 15 17:24 labels
drwxr-xr-x 5 user staff 160 Mar 15 17:24 images
$ls -lat dataset_v5/images/
total 0
drwxr-xr-x 9 user staff 288 Mar 15 17:24 ..
drwxr-xr-x 106 user staff 3392 Mar 15 17:24 test
drwxr-xr-x 5 user staff 160 Mar 15 17:24 .
drwxr-xr-x 158 user staff 5056 Mar 15 17:24 val
drwxr-xr-x 782 user staff 25024 Mar 15 17:24 train
dataset_v5.zip
- https://drive.google.com/drive/u/0/folders/17-rLAQ9GLda7M5mvDiitINqSLBUj_wbm