A lightweight 3D Morphable Face Model library in modern C++
APACHE-2.0 License
Bot releases are hidden (Show)
This release contains quite a few major updates, alongside some smaller "maintenance" updates.
glm::project()
, perspective()
and ortho()
using Eigen (see eos/render/matrix_projection.hpp).GLM_FORCE_UNRESTRICTED_GENTYPE
. Eigen and Ceres work much nicer together. Many improvements, and most cost functions were renamed - check the commit message in d101ca546118ae8736920915c6ac0dead448161f for more details. Also reworked PriorCost
and renamed it to NormCost
(e79da919a14b2449acbf8eff57572238e205ed95), and updated fit-model-ceres.fitting::concat
to contour_correspondence.hpp
(1029f96f8bd8fa66b55ad1f73962b14aa3fc0f85)add_definitions()
for Boost (1de80682ad38f0e5784302e3089b52b6aca5ff39), and used CMake targets whenever available (e.g. Ceres::ceres
).The documentation website is updated to v1.4.0.
The updated release is available on PyPI, with wheels for Python 3.9, 3.10 and 3.11 on Windows (x64), and source installation for other platforms.
Note: It is very likely that the next release will require the C++17 standard, and will no longer compile in C++14 mode.
Published by patrikhuber over 2 years ago
This is a minor maintenance release, but with one important change, warranting the version push to 1.3.0 to not catch anyone off guard: The Eigen version in 3rdparty/
is updated from 3.3.4 to the latest 3.4.0.
Full release notes:
3rdparty/
is updated from 3.3.4 to 3.4.0. The submodule now also uses the official Eigen gitlab.com URL.CITATION.cff
file to make it easier to correctly cite eos__repr__
methods of some Python bindings a bit (9b8b3509e609be026f56d98073c2cb6699c55978)ExpressionModelType
None
with none
in Python to avoid clashing with Python's keyword None
(85d09d255da4d57817e0263a06aec0626ed38fed)eos::core::Mesh
's Python bindings to make it more convenient to create a Mesh
object (b2f24a12123b0e2409c139cc5d991e3b2a326454)Note: It is quite likely that support for C++14 will be dropped in one of the future releases, and a C++17 compliant compiler will become the minimum requirement to use eos.
Published by patrikhuber about 4 years ago
This is a follow-up update pushing the python bindings to PyPI:
Updated pybind11 to their latest master commit, and changed the Visual Studio CMake generator for the Python bindings from VS 2017 to VS 2019 (75ecca4ac84dce5a1a5113704f7f8714060663da).
eos/@patrikhuber is now part of GitHub Sponsors. If you are finding eos valuable, there is now the possibility of becoming a sponsor.
See the v1.2.0 release notes for all the changes and improvements in this release!
The updated release is available on PyPI.
Published by patrikhuber about 4 years ago
This release updates the renderer and texture extraction, with full support for perspective projection (along with orthographic projection), and contains various other smaller improvements and fixes.
New / major improvements:
v2::extract_texture(...)
to support texture coordinates with seams (63243b2)v2::extract_texture(...)
to the eos::render
namespace and made it the new default (08cba28). Calling code needs to be adjusted slightly, but the function's parameters are very similar. This new texture extraction uses the Rasterizer
of the new renderer, and supports both orthographic and perspective projection.extract_texture(...)
function, which only supported affine cameras (8fd1735)SoftwareRenderer::render(...)
to correctly deal with texture maps with seams (c1defba)render(...)
function, moved v2::SoftwareRenderer
to the render
namespace, and added a convenience function for the new renderer that can be called (almost) like the old renderer (12a4120) - calling code needs slight adjustments. The core of the new renderer is the same, but the architecture has been changed to mirror more closely the hardware (or OpenGL) pipeline, with vertex- and fragment shaders. Results should be exactly the same, and they have the same parameters.
(0.5, 0.5, 0.5)
, in the new renderer, they will have default colour black (bd01644). In any case, it does not make sense to try and render a mesh without vertex colours or texture.enum class
ProjectionType
, which specifies whether orthographic or perspective projection should be used.Fixes:
Image
python bindings to account for row-major Image
class (dc0fce8). (The Image
class was changed to row-major in a commit a while ago, but the python bindings were never updated to accommodate this.)mpark::variant
to their latest master (bd590bf). This should fix a few compile errors with certain compilers when compiling eos in C++14 mode.Minor improvements/changes:
v2::extract_texture(...)
to not crash when the projected model is partially outside the image (442497e)clear_buffers()
function to SoftwareRenderer
and Rasterizer
to clear the colour- and depth buffers (1e8a875)project(...)
function to PcaModel
(4694d6f) (see #295 for a potential issue)render/draw_utils.hpp
to render/opencv/
(914fdd5)draw_wireframe(...)
function that doesn't need OpenCV (7303b39) and added python binding for it (f1f83d0)<filename>.isomap.png
output of the example apps to <filename>.texture.png
(174a2c2)Removed:
render::detail::interpolate_black_line(...)
(it was only needed for #4, which was fixed, and also it was for the old texture extraction) (35abed8)estimate_affine_camera(...)
. See the commit message, for people that were still using this. (507eb77)render_affine(...)
, this was used in the old texture extraction and is no longer needed (aee7984).Published by patrikhuber over 5 years ago
This is a minor update that slightly improves the internals of loading .scm
models from CVSSP from the University of Surrey, and also adds load_scm_model(...)
to the Python bindings, so these models can be loaded directly in Python.
Changes/improvements:
eos/morphablemodel/io/cvssp.hpp
, and clarified the documentation slightly [dd4cd62d4724f4368916875d1303f03e98f2f0fb and following]utils/scm-to-cereal
being the only one) can now be built without OpenCV [d465a042a860283ba3f11ce15683bf186a607273]load_scm_model(...)
to the Python bindings (in the namespace eos.morphablemodel.load_scm_model(...)
) [b82b85244d5e5ed55a257d0483d21bcfcef4c9c6]The updated release is also available on PyPI.
Published by patrikhuber over 5 years ago
This release mainly fixes the write_obj(...)
and write_textured_obj(...)
functions to now also write out the texture triangle indices (tti), when they're present in a mesh. Previously for meshes who used a different set of texture triangle indices, this could cause weird artefacts on the top of the head because the tti were not written to the obj. [c25319769f3d103293d62c857a1b330631c90e5f]
Other small changes/improvements:
write_obj(...)
and write_textured_obj(...)
into a separate header, eos/core/write_obj.hpp
, so that including Mesh.hpp
doesn't result in transitively including <fstream>
and <string>
[a040c0ed73b6df426fdfedb7fa7804e28514a879]EOS_
in the eos::core
namespace to reduce possible name collisions [4381fd16b994c4533e9af2fe6cc1f63b5adea48a]write_obj(...)
now also inverts the y axis of the uv coords, to be consistent with write_textured_obj(...)
[7ce291825a916190aedf790b465ec68dcf49373f]setup.py
now checks for CMake 3.10.0 on Windows [3dd69f7727167345f135421a918f2e1ec7c6b5b7]Published by patrikhuber over 5 years ago
This release contains an important fix for models that have separate texture_triangle_indices
, like e.g. the 4DFM.
It also contains a number of other minor improvements and an updated documentation.
Fixes:
texture_triangle_indices
is now correctly passed along to sample_to_mesh(...)
in all MorphableModel
member functions (e598203ddf8ec370cc86b6ab3e1ea8045f6fb705). Previously, get_mean
and the draw_sample
functions (except for the one that takes shape, colour and expression coefficients) would have returned a mesh without the texture triangle indices (if the MorphableModel
contained them).sample_to_mesh(...)
is not triggered in Release and RelWithDebInfo builds, therefore we haven't caught this for a while.Minor improvements & changes:
v1.1.0
Mesh::tti
to the Python bindings (afc80dfb4bdd3160c867e6327be0ec3818da1fb0)eos/render/normals.hpp
header (5b6cd230ffac75c8139bb1e74be8d888eb830004)include/eos/render/utils.hpp
to transforms.hpp
MorphableModels
with odd versions (41fddf6d732df07c5077139b74235b995f7edf4f)ray_triangle_intersect(...)
to its own header (5b574092d562e8628b95edbf4af83a2f8049b5e9)Image
visualiser for Image Watch: Set channel type to RGB(A) for 3 and 4 channel images (622fc35a0e6fd22938237e834c1511c7a749bbf4)In-progress changes:
These are in-progress changes on new rendering/texture extraction stuff in the render::v2
namespace - these work in principle but are not too thoroughly tested or used in eos yet.
v2::extract_texture(...)
and updated all affected code (23d07554a014d7ce1a1fcafe13e4173074175512)v2::SoftwareRenderer
and related code to compile with the latest eos changes and without OpenCVPublished by patrikhuber almost 6 years ago
The API and functionality of eos have been quite stable for a while now and proven themselves robust and useful. This release mainly constitutes the exciting bump to v1.0!
One major recent addition is support for the 4D Face Model, 4DFM, a full-head 3D morphable face model with 36 expressions/action units, from 4dface.io.
The update also includes the most recent version of pybind11, and addition of sample_to_mesh()
to the Python bindings.
Other recent changes & improvements include supporting uv coordinates with seams, support for landmark/vertex definitions in the MorphableModel
class to give certain vertices a fixed semantic meaning, and lots of other things. See the release notes from v0.17.0 and v0.18.0 for more details.
Published by patrikhuber almost 6 years ago
This release adds support for texture maps with seams. When a texture map contains seams, the vertices at the seams are usually duplicated, and then some of the triangle indices are different in uv space from the triangulation of the 3D vertices. This change allows to support the texture coordinates of the full-head 4D Face Model (4DFM).
MorphableModel
file format version was increased to 4
to store an additional member containing the uv triangulation (85cc563afe7e1916fa98d577c5b036cb622c8806)extract_texture
to support uv coordinates with seams (a70ded3d3c7644b54fd97878183d7b5c46323624)Other minor changes:
Please also have a look at the changes in v0.17.0.
Published by patrikhuber almost 6 years ago
This is a release with quite a few nice changes and improvements. Please read through the release notes carefully.
This release was in the devel
branch for a few months, and never actually left the devel branch. The MorphableModel
format version was increased to 3, from adding landmark_definitions
. Since there are quite a few model files with this version in circulation, it was deemed best to make it an "official" model version and release it as v0.17.0.
Major changes & improvements:
"eye.right.outer_corner" <=> 177
from a generic name to vertex ids - and then a second mappings file can be used, that maps from 2D landmark identifiers (like the number 37
from ibug) to these generic names ("eye.right.outer_corner"
). This separates the vertex definition from a particular set of 2D landmarks.ibug_to_sfm.txt
, which map directly from 2D landmark identifiers to 3D vertex indices.MorphableModel
for landmark_definitions
(e995b07821038567284bf02202feb6021289d1e0)vertex_idx
in the fitting depending on whether the model contains landmark definitions (f123e518c3a85bb5deaf9d17e7c11931ede9225a)Image
class to a better and more future-proof design (14b7f6f6d998bb30de3f2047fa7f6af374786f3c)eos::core::Image
(be436f4d99406cfb62addabcc45303a7aa780a66)Fixes:
mpark::variant
compilation error on VS >=15.8; Updated pybind11 submodule to fix errors on VS 15.8 (a45a888bbf29b251f2bd472d13a06208387e6a8b, ecc626946da2cbea42623759ff09dec6ef9eba34)Minor improvements, potentially breaking changes:
get_all_mappings()
to get_mappings()
(9e0f539652e28a8b461bde9a53a202c471b04677)eos/core/image/
and moved Image_opencv_interop.hpp
there (ad06db1b9e85319e8bddbca67b52f248b4dda331)triangle_list
by const reference (a7ede7bfca5cc1929b43d629dc90db7c5c3ae14d)__repr__
methods.initial_rendering_params
parameter from fit_shape_and_pose (ba46285497485c1ab8f0f80722e8c09eb8b26e43)Other minor changes/improvements:
read_obj
a bit to handle a few more cases. Still, don't rely on it to parse every .obj
file, it's not meant to do that. Use your own or assimp
or something like that.And as always, various small improvements to the code style and to the documentation.
Published by patrikhuber over 6 years ago
A few minor updates and fixes:
Fixes:
draw_sample
(a6900a2)R
instead of 3x3 in the Python bindings of ScaledOrthoProjectionParameters
(5863f3b)Updates:
render::Rect
to core
namespaceeos::core::Rect
and eos::core::Landmark<Eigen::Vector2f>
. The Python bindings are now using Landmark
and LandmarkCollection
natively, without having to keep a separate list of landmark identifiers.long_description
for PyPI (link)Published by patrikhuber over 6 years ago
This is a major release, bringing full support for expression PCA models. MorphableModel
s can now either contain no expression model, a set of blendshapes, or a PCA expression model.
The fit_shape_and_pose
supports both as well, and calls either the _linear
or _nnls
solver, depending on whether a model contains blendshapes or an expression PCA model.
The main changes that go along with that:
MorphableModel
file format to version to 2
to include an optional expression model. Version 1
models can still be read.ExpressionModelType
to MorphableModel
to easily query the type of the expression model.Note: The behaviour of MorphableModel::get_mean()
has been changed: It now returns the overall shape and expression mean. (see #188)
Other small additions:
As always, when updating from the GitHub repository, make sure to run git submodule update --init --recursive
.
The binary files are provided for convenience only - it is recommended to build the examples from source, and use the library by cloning the GitHub repository.
Windows binaries are compiled with Windows 10 64bit, Visual Studio 2017. A .whl with python bindings for python 3.6 is available on PyPI.
Linux binaries are compiled with Linux Mint 18.3, based on Ubuntu 16.04, gcc-7. Make sure to follow the instructions for how to use gcc-7
with pip
if using the Python bindings on Linux.
Published by patrikhuber over 6 years ago
This release improves a few crucial things over the last release.
Support for pre-C++17 compilers should be a lot better again - eos compiles again in C++14 mode on VS 2017, and, although not officially tested in CI, likely also compiles again with the likes of g++-5
:
akrzemi1::optional
on all pre-C++17 compilers, not only on Apple platforms.optional
also works correctly on all platforms (pre-C++17 and C++17 compilers); and updated cereal to the latest develop
branch commit.optional
now also works within the python bindings for pre-C++17 and C++17 compilers.eos::cpp17::clamp
, which uses std::clamp
on C++17 compilers, or a fallback otherwise.Matlab bindings:
eos-matlab-fitting
bindings! They now work again in the latest version to use the fitting from Matlab.eos-matlab-render
bindings for now, as they would be more work to update. Having the fitting
bindings back should suffice for most use cases.Other small things:
const
s, a couple documentation updates.Published by patrikhuber over 6 years ago
eos is now fully working on Apple Xcode again and CI-tested with Xcode 9.2.
Main changes:
eos::cpp17::optional
throughout the code base, to support platforms that do not have <optional>
yet (currently only macOS - #ifdef __APPLE__
). It is a namespace alias, and will be resolved to std::optional
on compilers that have <optional>
.eigen-git-mirror
as submodule at tag 3.3.4, instead of the patrikhuber/eigen forkKeyframe
-related code to work again with the latest eos versionPublished by patrikhuber almost 7 years ago
This is an update release with various small improvements throughout and two highlights:
One major highlight is proper (and tested) support for the BFM2009 (and partially the BFM2017).
Another highlight is that the multi_image_fit_devel
branch is finally merged into master, and multi-frame fitting is thus now available in the main version.
The changes include, alongside other minor changes:
multi_image_fit_devel
branch. Thank you very much @PhilippKopp for the contribution!sample_to_mesh
find_if
result for std::end
in get_nearest_contour_correspondences
to avoid continuing in Release builds without any error messageconst
qualifiers where possibleMesh
and Blendshape
)Published by patrikhuber almost 7 years ago
This release contains some major changes, mainly to the software-side of the library, and not to the algorithms.
All in all, these changes make using the library much easier, and they should especially enable installation of the python bindings via pip install eos-py
on any system, given >=cmake-3.8.2 and a recent compiler.
The changes, alongside other minor changes, include:
Eigen::Vector
cv::Mat
(eos/core/Image_opencv_interop.hpp))Published by patrikhuber almost 7 years ago
This is a minor release, mainly adding functionality to build PCA models (include/eos/pca/pca.hpp
).
This will be the last version that supports gcc-5, clang-3.x, and VS2015. The next release will require >=gcc-7, >=clang-5 and >=VS2017U3.
Other minor changes:
const
, improved documentation, updates of submodulesPublished by patrikhuber over 7 years ago
This update brings a major speed-up of the shape and blendshape fitting. Depending on the setup (model resolution used, number of coefficients fitted), the speed-up is around 3x to 50x - typically around 10x. The shape and blendshape fitting now take only around or below 1ms per iteration (previously ~10ms). This was achieved by changing all linear algebra operations from OpenCV to Eigen.
Please see also the release notes of v0.12.0 below for the latest changes and updates, some of them breaking changes. See also the release notes of v0.12.0 for information on the PyPi package (eos-py) and the provided binaries.
Published by patrikhuber over 7 years ago
This is mainly a maintenance release but contains quite a number of improvements. There are some breaking changes in this release and models have to be re-downloaded in the new format.
Major changes:
PcaModel
, MorphableModel
and blendshapes to use Eigen for matrices and vectors instead of OpenCV's cv::Mat
. This change was motivated by the fact that we need Eigen already anyway for solving linear systems. And since Eigen is a header-only library, the core of eos becomes more independent of OpenCV and easier to compile in the near future.PcaModel
constructor to take an orthonormal basis (4662c59c4d9ae233047d1b5130470db1ed6bf894)PcaModel
constructor and are passing it a PCA basis matrix, your code will (probably silently) break: The function now expects an orthonormal basis matrix, whereas before, it expected a rescaled basis matrix. However it makes more sense for the function to take an orthonormal basis matrix, as this is usually the result after computing a PCA on data.Minor changes:
get_*_pca_basis(int)
to get_*_pca_basis_at_point(int)
(354deb3e962dce0e7dab7b20f1d4c5886a06e103)pip install ./eos
, and packages can be created with python setup.py sdist
and python setup.py bdist_wheel
. A setup.cfg.template
exists to set library paths to OpenCV and Boost.sdist
package. An external Eigen can still be used by setting EIGEN3_INCLUDE_DIR
.fit-model-ceres
eos python bindings are now available on PyPI! There's a wheel for python 3.5 on Windows, and a source dist. The source dist needs CMake's find_package to be able to find OpenCV and Boost automatically for it to work (so install it beforehand from a package manager).
If there are issues, you can always clone the repo and do pip install ./eos
locally, after adjusting the paths in setup.cfg
.
As always, make sure to clone the repository with git clone --recursive
, or, if you're updating, run git submodule update --init --recursive
.
Windows binaries are compiled with Windows 8.1 64bit, Visual Studio 2015. A .whl with python bindings for python 3.5 is available on PyPI. The Matlab bindings are created for Matlab R2016b.
Linux binaries are compiled with Linux Mint 18, based on Ubuntu 16.04, gcc-5.4.