OpenSeeFace

Changed the gaze tracking model to always run single threaded.

Fixed various issues, made other small improvements and added a new, experimental tracking model with improved wink support.

Fixed some bugs.

Reduced the impact of eye blinks and jaw movement on head pose estimation and fixed various things.

This release contains various fixes and adjustments as well as two new tracking models with different quality and speed tradeoffs.

This release adds support for jaw bone animation to OpenSeeVRMDriver, dynamic port selection support to OpenSeeLauncher and updates the binary build of the face tracker to use onnxruntime 1.5.1, fixing some performance issues caused by the tracker not respecting thread limits.

This release decodes landmarks from the python side again, as somehow using the decoded landmarks from setting inference=True on the models caused some issues

With this release, the landmark models decode landmarks within the ONNX models again. The included binary should run with lower CPU utilization at the same speed when enabling multithreading.

This release fixes a bug with the OpenSeeLauncher and adds support for selecting device capability lines for direct show cameras and a --benchmark option.

The main new feature with this release is a very fast, less accurate thirty point tracking model that can be activated with --model -1.

This release contains bug fixes and mostly small improvements. It also contains an optional, simplified expression detection method.

Further reliability improvements. Landmark decoding has been moved from the ONNX model back into tracker.py in one of the recent releases, but model.py still contains suitable decoding code for exporting your own ONNX model containing the landmark decoding.

Improved reliability and logging.

This release includes various small optimizations that can lead to a 20-25% performance gain under certain conditions.

This release fixes further issues with the new camera code.

This release accumulates a lot of random changes, mainly to the Unity parts. One big change is that escapi is no longer used for webcam reading on Windows and has been replaced with libdshowcamera, which is also used in OBS.

This release introduces a new model for --model 2. Unity components relying on 3D points will now skip frames with very bad 3D fits.

This release contains various improvements, such as improved models for --model 0 and --model 1.

The main new feature of this release is a new default face detection model that is used to lock onto yet untracked faces for tracking. There are also some other small changes and fixes. The chin tracking point is used for fitting the 3D points once more to prevent the solvePnP function from choosing an upside down fit.

New in this version:

• 3D points are now scaled according to the reference model along X and Y axis to make them more consistent. The tip of the nose should always be at (0, 0) and the nose should point straight up.
• OpenSeeVRMDriver can now animate mouth and eyes using the face tracking information.
• OpenSeeVRMDriver supports global hotkeys to override expressions.
• Small fixes and changes.