This repository has some examples of how to run images or existing video through PoseNet and BodyPix using Tensorflow.js in Node
Clone this repository:
git clone https://github.com/oveddan/posenet-body-pix-node.git
Cd into the repository, and install dependencies:
cd posenet-body-pix-node
npm install
Install node http-server globally, so that you can start a server to serve client side files
npm install http-server -g
Startup a local http-server, so you can view the results:
http-server
Estimating poses on an image:
node estimatePosesOnImage.js
This will estimate poses on an image ./assets/dance.jpg and save the results of the poses in data/dances_poses.json
Now, in the browser, open http://localhost:8080/draw_poses_for_image/
Estimating segmentation on an image, and saving the segmentation as a png:
node estimateSegmentationOnImage.js
Make sure you have ffmpeg installed. Follow the instructions here.
The examples for video in this repository work on a video converted into individual frames. ffmpeg
is the tool we use to convert videos into frames.
To download a video from youtube, use the 4k video downloader
To trim the video:
ffmpeg -i movie.mp4 -ss 00:00:00 -t 00:00:10 -async 1 cut.mp4
The above command takes a video and cuts it to be 10 seconds long.
Then to convert this into individual frames:
ffmpeg -i cut.mp4 ./frames/%04d.png
Which will take the cut video and extract it frame by frame into the folder ./frames
To get the framerate of a video:
ffprobe -v 0 -of csv=p=0 -select_streams v:0 -show_entries stream=r_frame_rate video.mp4
If you have a bunch of images you want to convert into a video:
ffmpeg -r 30 -f image2 -i %04d.png -vcodec libx264 -crf 25 -pix_fmt yuv420p video.mp4
Which takes all images in a folder that end with png, and convert them to a video at frame rate 30.
Run ffmpeg to convert the video cut.mp4 into frames:
ffmpeg -i ./assets/cut.mp4 ./assets/frames/%04d.png
Estimate poses on all these files and save them into a json file:
node estimatePoseOnVideo.js
Estimate segmentations on all these files and save each segmentation as an image:
node estimateSegmenationsOnVideo.js
To view the poses in the video open http://localhost:8080/draw_poses_for_video/
In the node.js scripts, replace the line:
const tfjs = require('@tensorflow/tfjs-node');
with
const tfjs = require('@tensorflow/tfjs-node-gpu');