UMU_5TF078_Paper_Keyboard_Tracker

The use of CNN and ROI for classifying drawn keys

This was the final project of the course 5TF078 Deep Learning - Methods and Applications at Umeå University I took as an elective one while doing my main studies at KTH Royal Institute of Technology. The project was highly appriciahted by the teacher and gave a deeper understanding for implementations and limitations of machine learning and computer vision.

"When using computers, as a rule, three types of input methods are used, keyboard, computer mouse (or trackpad on portable devices), and camera. These three have all been developed over a long time and come in different variants and features. But with today's great advances in AI and hardware, we can take this further without developing new computing accessories. By using the webcams we have and AI models together with existing algorithms for image processing, we can, among other things, replace or extend our keyboards with drawn ones. Adapted for just the right area of use."

The introduction above is just a short section of the report written for this university project. The full report is available in Swedish in the folder "Rapport".

Exploring The Program

If you wish you test the program, just download the folder called "Program", install the libraries in "requirements.txt" and execute "main.py".

Exploring The Model

To explore the model, go to the folder CNN. There you'll find all the necessary data including a "requirements.txt"-file which covers all the required libraries. Not that the data used is downloaded by the code written from a separate FTP-server due to file size limitations.

Note that what's been uploaded so far is just the start of a longer project. No support for finger-tracking as been added so far.