HolisticMotionCapture is an application and package that can capture the motion of a person with only a monocular color camera and move the VRM avatar's pose, face, and hands.
APACHE-2.0 License
HolisticMotionCapture is an application that can capture the motion of a person with only a monocular color camera and move the VRM avatar's pose, face, and hands.
Pose, face and hands can be moved simultaneously or individually.
You can also output the rendered video to other applications.
HolisticMotionCapture can be downloaded from release page.
Download installer(HolisticMotionCapture-1.1.0setup.exe
file) from 1.1.0 release page and execute it.
Download installer(HolisticMotionCapture_installer.pkg
file) from 1.1.0 release page and execute it.
C:/Users/<user name>/AppData/LocalLow/IKEP/HolisticMotionCapture/VrmFiles
directory in Windows./Users/<user name>/Library/Application Support/IKEP/HolisticMotionCapture/VrmFiles
directory in Mac OS.New VRM
button.
Input Device
pull down.W
and H
input field.Start/Stop
button for starting or stopping camera capture.C:/Users/<user name>/AppData/LocalLow/IKEP/HolisticMotionCapture/LoadedImages
directory in Windows./Users/<user name>/Library/Application Support/IKEP/HolisticMotionCapture/LoadedImages
directory in Mac OS.None
option from pull down.New image
button..png
) or JPG(.jpg
, .jpeg
) images.
Rendered images can be output as the virtual camera image if Output on/off
toggle is on.
You can show composited image in another applications when you select a camera named HolisticMotionCapture
in another applications.
Rendered images can be output as the Syphon image if Output on/off
toggle is on.
You can receive composited image in another applications compatible with Syphon.
Mode
: You can choose to move your avatar's pose, face, or hands.Reset pose
button: You can reset your avatar pose.Move upper body only
toggle: You can choose to move only the upper body or move the whole body. It is useful when only your upper body is shown in the camera such as video conferencing.Look Camera
toggle: You can choose whether your avatar should always look at the camera or not.HolisticMotionCapture provides a package to move VRM avatars independently of my application.
HolisticMotionCapture package can be installed by adding following sections to your manifest file (Packages/manifest.json
).
To the scopedRegistries
section:
{
"name": "Keijiro",
"url": "https://registry.npmjs.com",
"scopes": [ "jp.keijiro" ]
},
{
"name": "creativeikep",
"url": "https://registry.npmjs.com",
"scopes": [ "jp.ikep" ]
}
To the dependencies
section:
"jp.ikep.holistic-motion-capture": "1.1.0"
Finally, the manifest file looks like below:
{
"scopedRegistries": [
{
"name": "Keijiro",
"url": "https://registry.npmjs.com",
"scopes": [ "jp.keijiro" ]
},
{
"name": "creativeikep",
"url": "https://registry.npmjs.com",
"scopes": [ "jp.ikep" ]
}
],
"dependencies": {
"jp.ikep.holistic-motion-capture": "1.1.0",
...
}
}
using UnityEngine;
using HolisticMotionCapture;
public class Visualizer : MonoBehaviour
{
// Animator of VRM avatar
[SerializeField] Animator avatar;
HolisticMotionCapturePipeline motionCapture;
WebCamTexture webCam;
void Start()
{
// Initialize
motionCapture = new HolisticMotionCapturePipeline(avatar);
webCam = new WebCamTexture("Your webcam name", width, height);
webCam.Play();
}
void Update()
{
// Ability to operate avatars with textures only.
// You can also specify some optional arguments.
motionCapture.AvatarPoseRender(webCam);
}
}
Video for demo was downloaded from here
HolisticMotionCapture package uses the following packages:
"com.vrmc.vrmshaders": "https://github.com/vrm-c/UniVRM.git?path=/Assets/VRMShaders#v0.108.0",
"com.vrmc.gltf": "https://github.com/vrm-c/UniVRM.git?path=/Assets/UniGLTF#v0.108.0",
"com.vrmc.univrm": "https://github.com/vrm-c/UniVRM.git?path=/Assets/VRM#v0.108.0"
HolisticMotionCapture application also uses the following packages:
Copyright (c) 2023 IKEP