LipSynching Characters
Synchronizing the lip movements of a character to match the words of a recorded voice track is called lip synching. Lip-synching a character is a lot like choreographing, but on a smaller scale. In fact, motion capture systems are designed for capturing facial and lip movements of an actor in real time.
In the same way that live motion capture is not used unless the project is for a regular, repetitive, or large project, lip synching by hand is preferred by most animators who only require the technique occasionally or who need to lip-synch several characters in short sequences. As a beginning animator, the target audience of this book, chances are your first experiences with lip synching will not be enhanced with a live motion capture system. Don't feel bad. After more than 10 years' of work in the business and just as many years writing about it, I haven't had a chance to try motion capture lip synch either.
After you get the hang of it, lip synching by hand can be fun and you can get about a sentence a day done on a good software program. The technique begins by importing the voice of the character into your animation program as an .mp3 or .wav file (the two most popular audio file formats).
Many animation programs feature the capability to import and scrub through sound using a tool that mimics a nonlinear editing (NLE) program or video cassette recorder (VCR). First, you link the audio track to the animation software's timeline so that when you scrub forward and backward on the timeline, the audio plays along. Then you place the first frame of your character's lip movement to the first frame of the voice track. Now scrub a few frames forward until the first sound is heard. Using the various control tools of the software, you distort the lips of the character to match the sound at a particular frame.
For instance, if the voice is saying, "Boy," and you are on the frame where the B sound begins, you would distort the lips to be pressed together and rolled a bit into the mouth. When you get the lips looking just right, you save this position of the character as a keyframe. Then you scrub forward a few frames until the sound of the "o" is heard. Here you distort the lips to be open, forming the vowel with the jaw dropped. You'd save another key frame here to record this position and scrub forward some more to the "oy," which is a bit tricky because this sound is produced by bringing the tongue upward a bit with the jaw. Record this change with another keyframe and then play back your results. If the animation looks realistic, continue to the next sound and so forth. If the synch doesn't look so good, you need to back up to your previous keyframes and reposition the lips and face features until you get it looking right.
It's always a good idea to work with a mirror in front of you, mounted by the computer monitor, so that you can see your own face and model the computer results to match reality. By doing so, you will also notice other ele-ments—your eyebrows, cheeks, eyes, and forehead—that also play a large part in making a character look real. Don't hesitate to capture these elements as well with their own keyframes. Eventually, with practice, you will achieve a high degree of realism. The difference between a wooden-looking puppet and something that captures the imagination of your audience is one of the true arts of the electronic animator. Superior results in this skill will win you the attention of producers and animation directors and lead to both freelance and full-time work opportunities.
Jack spent a lot of time lip-synching Dr. Jones, the Egyptian, and the Pharaoh. Because Dr. Jones spoke clear English without an accent, he was the most difficult. The audience's eye will forgive a mistake in lip movement when the voice is a thick foreign accent like Farag's or a short sequence like that of the Pharaoh. However, when Dr. Jones says "The power?" the lips better be smacking on the p and moving on the w or belief is lost and the audience senses a drop in quality.
Rendering a Test Pass in SD Resolution
Here's a big secret of professional animators: Hardly anybody renders entire frames in one pass. Let me explain. Suppose you have a complex scene, such as our pyramid interior, which is composed of the walls of the pyramid, the flaming torches mounted on the walls, the characters, and the flames of their hand-held torches.
Which of these elements is going to take the longest time to render in each frame? Right, the torch flames. In order to look realistic, these flames are composed of millions of individual particles, each with its own light, trajectory, and limits. The walls, of course, are the easiest to render. The walls hardly change, except when the camera moves, or when the light of the torches falls on them.
Let's say we're going to do a test rendering of a scene. The walls most likely will come out fine with few corrections required. Most of the problems we are going to find are going to be in the characters. An arm joint, while moving, may expose a bad surface element, resulting in an apparent hole. A lip movement may be out of synch. A bad shadow may fall across the face. Each of these mistakes and others could appear in a test rendering, requiring the animator to go back to the offending frames, diagnose the problem, and then do another test rendering.
If we had to rerender the entire frame—the wall background, the flames, and the characters—every time we did a test render, the time required would be enormous. An animator, making hundreds of dollars a day, would end up spending most of his or her time sitting around waiting for the render to finish, chomping Doritos, and guzzling Pepsi.
Thanks to image-compositing programs, however, the time required for test renderings, in fact, the time required for all renderings, can be drastically reduced. Instead of rendering the entire frame for every scene, the animator breaks the scene down into layers and renders them separately. For instance, in our JVC project, the walls with the torch-lighting effect were rendered in one pass. After minor corrections and rerenders, the sequences of frames representing the wall were stored to a hard drive under the sequential names Scene5Wall00001, Scene5Wall00002, and so on. Obviously, the computer can chug through these frames really quickly.
Incidentally, the computer also calculates wherever the characters and other foreground objects will appear and deletes these areas from the wall rendering. What you get is the wall with black images, representing the foreground objects, moving about.
Next, the animator will render test sequences of the characters, perhaps even breaking these renders down into individual characters if necessary. Here more problems may be found and corrected, but each time the animator will direct the computer to render the character frames, the computer will only have to render the character, not the background or the flames. When the characters are correct, the animator stores these frames on the hard drive under another series of sequential names, say, Scene5Jones00001 and so on.
When all the layers are test rendered, corrected, and rerendered, the animator will then open up an image-compositing program. The leading compositing programs for the desktop artist are Discreet's combustion (www.discreet.com), Adobe After Effects (www.adobe.com), and eyeon's Digital Fusion (www.leitch.com). Each program works a bit differently, but each enables the synchronized, digital marriage of complex, moving image sequences. If the animator has done his or her preparation work properly, each layer in each frame will match up and combine to form the final composite image.
Of course, the animator must compose one final test render of the entire scene to make sure each element falls into the correct place to form a perfect frame. Occasionally, the animator may have to return to one or more layers, fix an error, rerender the sequence, and recomposite the scene. This is all part of the testing and perfecting process, which hopefully improves with each iteration.
At some time in the testing and rerendering process, the producer will decide when the animation is good enough to show to the client for a milestone approval. At this time, the producer is saying to the client, "This is our finished animation, except for mixing the final audio track." Once the client signs off on this version, the producer is free to take the project to completion by adding the final mixed audio and delivering the final master to the client. In reality, however, the version shown to the client for approval may not be the final rendered version.
Just as the process of test rendering is made more efficient by layering and compositing the frames, so is the final rendering. In fact, the final rendering is defined as the test rendering in which no mistakes occur. Sometimes, however, the animator never gets to such a perfect test rendering. Sometimes the clock or the budget runs out, and the best render version is the one that gets released.
We're fond of an old saying, "We provide the level of perfection the client can afford in time and money." Every animator wants a perfect end result, but often animators become obsessed with their work. What was once a perfect test on Friday, after a weekend's reflection, becomes a horribly scarred abomination, screaming for a rework on Monday. In many cases, the client has already expressed his satisfaction, even delight, but the animator goes further. At such times, it is the producer's job to gently direct the animator toward a greater understanding. This is most easily accomplished by assigning the animator a new project.
Post a comment