|Part 1: Overview (First steps)||Part 2: Details (manual procedure, many hints&tricks)|
Motion capture is a great way of quickly creating realistic animations. Unfortunately animating a rig from mocap data is a not a trivial exercise. There are several approaches you can take and efficient methods for retargeting animations from one rig to another is very much an active research topic.
Avastar has some support for retargeting motion from another rig such as one from an imported BVH. This is a work in progress and is somewhat manual at this stage. Good results can be obtained with about 10 minutes work. There is essentially three steps involved:
Add a fresh Avastar to your scene. Optionally hide the meshes and set the bones to stick shapes. This Armature will be the target.
Then import your BVH file. This will create a new armature, the source. It might be orientated wrong and have wrong scale. You can fix the scaling right away in the File Importer’s operator panel.
Make both armatures visible (If they are on different layers, then select both layers). Then scale/move/rotate the source armature (i.e. in object mode) so that it matches the target as closely as possible in size and orientation.
Scan through the source animation and choose the segment you want to copy, and set the start and end frames in the scene for this segment.
Choose a frame to be the reference frame – you want to choose a posture that will be easy to match with the Avastar target. It can be either inside or outside the range you want to copy, it doesn’t matter.
If necessary, you can create a new posture in the source animation by selecting the source armature, switching to pose mode, and removing all rotations (Alt-R) and translations (Alt-G). Sometimes BVH files don’t have a nice T-frame to copy so you can add one this way. Or you even can create any pose yoo like. Just take care that the pose of the source matches the pose of the target on the same(!) frame and select this frame as the reference frame. Hint: Don’t forget to keyframe the entire source after you posed it.
Now transfer the pose. This will later instruct Avastar how to transfer the entire anmation. Hence, match the poses as close as possible for getting best matching results.
In the Motion Transfer pane (Tools panel) set the source and target objects, more options will become available when you have these set. In the bone map, set the bones in the source that correspond to the bones in the target.
You only need to do one half and use the Mirror Copy button, and you can select a bone in the source armature and press the target icon next to a field to automatically fill in the name. We also have added a guessing feature which can identify animations made for OpenSim (and similar online worlds) and Animations made for the Carnegie Mellon animation library.
In all other cases there is a degree of artistry in this mapping and you may need to play around and see the results. Typically the two armatures will not have the same numbers of bones and they will be in different proportions.
You should choose bones that should have the same visual orientation rather than the same relationship within the armatures. Set the reference frame number to the one you chose and everything is ready. Now you can choose to copy across individual poses (Transfer Pose) or the animation segment as a whole (Transfer Motion). You can also choose to simplify the resulting keyframes with various algorithms.
Once you copy across the motion you will probably have to manually tweak things to fix any glitches, foot skating etc.
Tips and Tricks
- If you use curve simplification, aim for at most 80% keyframe reduction. More than this and the animation starts to become noticeably deformed. You can see the reduction achieved reported on the console logs. Start with high values for the tolerances and reduce them, this will add more and more keyframes.
- The tool will not delete keyframes, you have to do this by hand if you need to.
- Switching off the rotation constraints can help, you can always switch them on again later.
- Using motion paths can be really useful for finding places where the animation has problems. Set them up in the armature tab of the property editor, there are also buttons for clearing and recalculating them in the tools panel.
- You can correct problems in a BVH by carefully setting up the reference frame. For instance, if the animation drifts to the left a bit you can turn the Avastar reference to the right a bit and the resulting offset can remome the drift. Similarly you can make the arms hang further out or in to adjust for different body shapes between source and target.
How it Works
The reference frame is the key. This defines a map from the source to the target for each bone involved. The map is concerned only with the orientation of the bone in the global coordinate system so you just need to match up the bones with how they should be visually aligned. Or, to be more accurate, with how the mesh that the bone controls should be aligned. Having this reference frame map means you can transfer the pose from an armature with different proportions, bone rolls, bone orientations, and even structure. It also means that you don’t have to have a zero-rotation posture that’s a T-pose – any pose will do. Through the reference map you can also correct for different proportions and different orientations of bones with a bit of practice. The price to pay is that it’s hard to automate this whole process as there are various artistic choices to be made.
There are two simplification algorithms available, though this may change as we are actively experimenting with different approaches. Generally, simplifying the motion curves is a good thing. The resulting curves should capture all the motion but have substantially fewer keyframes so be much nicer to work with if you want to adjust the motion later. Experiment a bit with the algorithms to find out which is best for you.
- Lowes Local. The rotation of each bone can be thought of as a curve in 5 dimensions (well, you could if you could visualise 5 dimensions). Each point is made up of the frame and the components of a quaternion. Lowes algorithm is applied to this curve to find a subset of keyframes that captures the curve to within a tolerance. The tolerance is just the distance in this 5D space. Note that Lowe’s algorithm doesn’t modify keyframes, it just identifies a subset. Also it retains the extreme points of a curve. The Lowes Local method will do this simplification for each bone independently.
- Lowes Global. As you might guess, this treats all the rotations of the armature as a single curve in a 100+ dimensional space (depends on how many bones you’ve mapped). The Lowes algorithm is applied to this single curve. The resulting keyframes are across all the bones.
Use this for looping animations. For it to work well you need to choose the start and end frames so that the posture is nearly the same. This tool will then replace the start and end posture with their averaged values and blend in the change back through a number of frames.
Blender’s mocap tools
Blender comes with it’s own mocap tools and you can freely mix-n-match with the tools in Avastar. Here are a whole lot of tutorials on Blender’s Mocap tools by BenjyTCook:
- Tutorial 1 Basic Retargeting
- Tutorial 2 Location Retargeting
- Tutorial 3 NLA System and manual Tweaking
- Tutorial 4 Post retarget fixes
- Tutorial 5 Retarget switching and stitching
- Tutorial 6 Path editing
- Tutorial 7 Baking and finishing up
Use imported BVH as a reference
The sure-fire way that will always give you good results is not surprisingly the most tedious. You simply import the BVH animation into a separate layer, scale and position the imported armature to match Avastar’s, then use it as a reference to pose your avatar by hand.
Start by blocking out the motion first at key places starting with the COG, then work out from there.
For example, for a walk animation set up poses at the frames where the foot lands on the ground and at the frames where the feet cross. Start by keyframing the COG at those locations then add the hip bones and torso etc, working outward …
The same technique can be used to copy across the motion from video.
For all these sources please respect the license imposed by the content creators.
- CG Speed – over 2500 files from the Carnegie Mellon University Motion Capture Database. Available in a variety of formats including BVH. There is a separate index though it requires a bit of searching to find clips that match what you are after.
- ACCAD – sample files from the Advanced Computing Center for the Arts and Design.
- centralsource.com – collection of BVH files for personal use by Michael D’Andrea.
- mixamo.com – online animation creation with both commercial and sample BVH files. Neat preview on hover.
- freemotionfiles – Blog with news and various sets of free mocap files.
- motcap.com – various free motion capture sequences with previews.
- www.ipisoft.com – Commercial markerless motion capture system for your desktop making use of MS Kinects or webcams