OpenSeeFace

This version introduces a number of manually designed features that can be read out from the OpenSeeData structure under the features field.

This version introduces a RetinaFace model for face detection, making it more reliable.

This version adds eye tracking. The OpenSeeVRMDriver example component also is updated to apply eye tracking data to VRM models.

New in this version:

• The 3D model used internally for head pose and depth estimation will now adapt to the tracked landmarks over time, increasing accuracy and reducing 3D point movement due to head rotation.
• OpenSeeLauncher will now work in projects built using IL2CPP when enabling the usePinvoke flag.
• An option was added to the face tracker to offset face ids by a given value to allow receiving face tracking data from multiple sources.
• The OpenSee component now actually closes its UDP socket. Previously, it could linger around, leading to issues when restarting a program using the component.
• The OpenSeeVRMDriver component has been added to the examples directory. It will apply expressions detected by OpenSeeExpression to a VRM model and, when enabled for a given expression, let the model's eyes blink. In addition to this, it will also use OVRLipSync to apply visemes according to audio input. Other than the main OVRLipSync component, no additional components from that library are required in the scene for this.

New in this version:

• A bug in expression training was fixed that reduced the training accuracy.
• It is now possible to train expression detection models in such a way that they output probabilities, but this is much slower.
• The OpenSeeIKTarget component now can mirror movement and rotation.

This release fixes a number of small bugs and includes an example for how to apply detected expressions to VRM models. In addition to this, the OpenSeeExpression component was changed to have less harsh requirements for training models, making an iterative approach much easier, where you collect a bit of data, test, collect more, test, and so on.

New things:

• MobileNetV3 based models with better accuracy
• SVM based expression detection for Unity
• SVM training and inference for Unity through the SVMModel class using LIBSVM
• --models was reordered to be more easily understood
• Tracking multiple faces keeps track of which face is which
• --visualize was changed. 2 now shows face ids, while 3 shows point numbers

This release was updated to include fixes for rotations and coordinates in Unity.

All models were retrained to fix bugs in the training process. There are also other bug fixes and improvements.

Blink detection and stabilized 3D pose estimation