Independent Study Post-Mortem

For my independent study this semester, I wanted to do something to be able to apply what I learned in the Game arts II class, animation.  Since I was creating a rig, and doing in game animations for the Projects class, I thought it would be good to do an additional project using that character, that may or may not actually go into the game, but be a project that would be the work of an entire semester.  What I decided to do, was to create an escape cinematic that would explain how the main character of the projects game escaped.  For the projects class, I was not set to work on cinematics, and the end was at the minimum just going to be a picture of the character jumping out into Lake Erie.  I also specifically choose this sequence, because it would include many complex animations, not all of which were cycles, like in the game.  This would give me an opportunity to stretch myself and create an entire animated scene, which I had yet to do.  I would have to use the principles of design, and film techniques to set up the shots.

To begin I started with a story board.  On the storyboard I planned out each shot I wanted to make, what type of shot it was to be, ie; medium, close up, etc., and what was to happen in the scene with basic stick figure drawings.  This I reviewed with Craig to get approval that the scene was clear enough, and that it would be a worthy assignment.

 Once that was finished I began to block the set out in maya.  This included creating several set pieces to be used, such as the ventilation fan, the switch, the corrugated tunnel, the dynamic water and the skybox that showed the Monroe Nuclear Power plant in the background.  I then laid out how each shot with a separate camera in Maya, named with the shot number, so they could be switched to and from easily in the panel viewer.  I then Began to block out the movements, and timing along with all the camera angles, so that the entire sequence could be watched in maya.  This was where most of the animation came in.  I had to make basic animations for the male character, as well as the monster, the doors to the room, the fan, the grate in front of the fan, the grate in the vent, and the water flowing.

I actually got very far in maya, even to the point that there was more than just blocked out animations.  I also had the lighting all set up, with fogs, and particle emissions.  I even had set up a dynamic water volume so that I could make a more realistic looking fall into the water. I began the process of rendering, and that is where I hit a snag.  This scene was extremely complicated, and the render times were measuring in the minutes for a single frame.  Even if I could have sent it to a render farm, it would have taken far too long to get the whole thing rendered.  Then came the issue of whether it could even be used in game.  Well, this was fine with me, as I had never specifically had the goal of integrating the sequence into the actual game.  The rest of the team however decided that I needed to get it working in game.  I began to look into the options of what kind of video could be integrated into the Unreal engine, and I game across bink video, which is supposed to be integrated easily into unreal.  The problem though, was that it could only be used as a texture during the actual game, unless you have the pro version of unreal.  To test the possibilities of this working, I created a few death cinematics for the game, none of which made it in, and were mostly to test whether I could get the big cinematic in.

I was eventually able to figure out how to get this working in the game, by setting up a virtual screen room, with an in game camera pointing at it, but the only problem was there was no way to tell the texture to play or not, it could only loop, so for the more linear sequence at the escape needed to be started at any given time, and would not work this same way.  The only way I could get the sequence to work in the game was to create the whole thing within the engine.

I had to begin nearly from scratch.  I had to create a new version of the layout that I had made in maya, with the existing meshes that were in the game.  I had to get the assets like the fan exported and working in game.  I had to create all new cameras and lay out the whole scene again in the unreal editor.  I also had to learn how to use the in game track editing system, Matinee Editor.  As a result of this complication, I decided to slightly simplify the flow of the scene, I cut out the ventilation tube, and had the fan go directly into the water tunnel, and have it go straight out into the water, rather than curving around.  I also decided to have the grate that was over the fan already on the ground, so as to eliminate that additional animation.  It wasn't that I didn't want to do all the animations, but I was now under a time crunch, and I just wanted to see a completed project, even if it wasn't how I had originally envisioned it.

Now came the tedius process of animating through matinee.  For any animation that you want to use, there has to be an additional channel in the slot node in the animation tree for the character.  The animation tree is how the blending between animations is done.  You must create the animation keys in maya, and then export them into the animation set, and create a blend node in the animation tree.  Once all this is done, you can use that animation in matinee.  Over the next few weeks, I got the animations all laid out with the animations that I currently had.  If I had had more time I would have added a few more animations to make the transitions a little better, such as going from roaring to walking, going from walking to slashing, and having a reaction animation for when he hit the fan.  I also would have added the main character hitting the button and jumping into the pipe a little less awkward.  Next is to set up a director track i matinee, where you can animate the camera positions, switch between cameras, and add sound tracks.  

Once all the shots were set up how I wanted, I began to add the sounds, and edit the curves turning sounds, animations and cameras on and off.  It is now finally complete enough to be included in the game.

I have learned a lot from this process.  First I learned to take an idea to a storyboard.  Use that storyboard to layout, setup camera shots, and lighting in maya, and beging blocking out animations, and iterating on feedback.  I also learned about adapting to extenuating circumstances, and figuring out how to still bring your creative vision to life, in some way you can be satisfied with.  I learned many new tool sets, and skills regarding the practical application of animation in game production.  I also got a taste of everything from modeling, texturing, rigging, animating, and editing as a result of my efforts to complete the work for the Independent Study project, that I would not have learned from the projects class alone.  Things I would do differently had I had the chance... I would have looked into the demands of the medium before I got so far down the wrong road, ie; had I known movies were such a problem to integrate, I would have planned from the beginning a much better setup to be created in game, rather than scrambling to get it together after I had to scrap half my work.  At the same time though, I am grateful to have had to go through both, because I learned new things specific to each medium that I wouldn't have learned otherwise.  I consider my time spent on this Independent study project to be very valuable, and the class study to be quite worthwhile. 

Projects Class Post-mortem: My work on ERIE this semester

Since I have done so much this semester :), in order to have some sort of coherency in this paper, I am going to organize my thoughts by when they happened.  From the beginning of the project I had volunteered to do the rigging and animation for everything in the game, and from right off the bat I ended up doing much more.  In the beginning I created a bat.  In order to bring life to the (literally) dead world, I decided to make some non-player characters.  Last semester I grabbed the reigns and figured out how to get a rat modeled, textured, rigged and animated and working inside of unreal.  This semester I made the rat's cousin.  The bat added a new challenge- the wings.  in order to make the wings foldable and movable without having clipping problems the skinning has to be exactly the same on the front and back of the wing.  To do this, the vertices of the wings needed to match up in the front and back, and then when the weight is assigned to the vertex to the corresponding bone, the front and back vertices need to have the exact same weight value.  When I first learned how to skin meshes to bones, I used the paint skin weights tool, which is an interactive painting tool on the mesh that is a grayscale where white is one and black is 0 influence.  That works great for most things, and is an easy way to see where the skin will deform, but in order to get exact numbers I had to learn a new way to weight.  That tool is the component editor.  In that editor, when you choose the vertices, it populates a spreadsheet with the vertex numbers on the left side, and across the top are what bones are influencing those vertices, and what float value they have.  In this way I quickly could see what value the vertex had, and make sure the corresponding vertex on the back side was exactly the same.  That way, no matter how far you bend the wing, the two layers of geometry wont overlap, causing the common 3d flaw of clipping.  The next thing I learned with the bat was about creating normal maps.  The texture on a 3d model is assigned to the surface by way of UV layouts.  The uv's correspond to the vertices, but in the UV layout editor, you can flatten by way of cutting up the model until it is all in one square area.  You can then use photoshop or another similar program to create textures.  They way I like to think of it is like a bearskin rug.  You take a bear you cut it up, and lay it flat.  That is great for making the color texture, but there are many other maps that a 3d model can have in order to give it a more dynamic look.  One of the most important is the normal map.  That map is one that takes a violet color scale to artificially create bump texture, and is much more effective than a grayscale bump map.  The problem with it is you either can make them by creating a high poly version sculpted out the way you want it to look, and then baking it out to project onto the low poly version, or you can use a program like Crazy Bump to take the color texture you made in photoshop, and it will create a bump normal map for you, as well as specular, occlusion, and a few other maps.  When I learned this process, I also went back to the rat model, and fixed a few problems it had with the seams, then I made a brown and black version of the rat texture, and used crazy bump to make a normal map for the rat.  Here are some pictures of the rat and bat to illustrate this concept..

 Bat Animations: Hang to flap, flapping loop, flap to land

Integreation: skeletal mesh, anim set, animtree.

Matinee

Next, since it sounded so much like rat and bat, I decided to go with the cat.  For this one, I wanted to learn how to use mudbox to make the texture and normal map.  For this I created a base low poly mesh for the cat, and then exported it to mudbox, where many subdivisions are added to the mesh, allowing for almost literal sculpting like you would do with clay.  Once that process is done, you can bake out a projection of these folds and bumps onto a low poly mesh as a normal map.  Here is a video and pictures of the process of modeling and texturing the cat.

Integreation: Because of the time constraint to get the main character in, once Ryan finished with that mesh, I had to stop the cat there until I was finished with the higher priority task of rigging skinning and animating the main character which we had decided to fully animate and be able to see through a first person view at any time.  After I was done with that  the monster became another priority, so the cat never received a rig, but it ended up having an unusual but entertaining part of the game.  We made the dead cat with its awful dying sound a hidden pickup collect-able, for which I was responsible for integrating, but I will go into that more when I get to Unreal Kismet, and how I stumbled into that.

Isaac/Clarke/Oliver

Clarke is the pet name I gave to the main character when we were in transition between 1940s German setting to 1960's Michigan.  I worked with Ryan to ensure proper deformations for animation were on the mesh he created, and when the file was finally passed on to me I had to build a complete rig from the ground up.  After laying out the bones, I created controllers to be used to keyframe on for the animation.  I also used the connection editor to transfer custom channel controls to those controllers such as IK switching, and Custom Channels, with Set Driven Keys.  when you create a controller, you can add attributes with a min and max value.  Through the set driven key editor, you select what you want as the driver, in the case of a rig, the controller's custom attribute, and the driven, which in the case of animation, I used the rotation attribute on the bones of say a finger.  Then you set a key at 0, then move the attribute in the channel box to -10 and move the bones in the finger into a curl, and set another key, and with 10 move the finger outstretched and up as far as it should go, and set another key.   Once you do this, the custom attribute can be used as a slider, when you slide it you will see the model moving from a curl to extended and everywhere inbetween.  In this way I created custom controls for foot ball, toe heel and ankle rotation, toe lift, individual finger curls, fist curl, thumb rotations and curl, Jaw grind, mouth open, and a few others.  I also created IK/FK functionality for the arms and legs.  IK standing for inverse kinematics allows you to grab the ankle and the rest of the leg will move with it.  FK for Forward Kinematics allows you to rotate a joint and the children of that joint will follow exactly relative to the parent joint.  Both are usefull to use, and blending between the two allows the most control for animation.  Using both methods I learned while painting the skin weights on the rat, I skinned Clarke until I was satisfied he could do all the animations I needed him to do for the game.  Then, I was off to animation.  For Clarke's animations we were not exactly sure how we were going to integrate him, so I ended up having to make 3 sets of animations for every action.  One with full body movement, one with his right arm hidden, so as to replace with a socketed arm(which didn't end up working) and another set with and without him holding a spray can.  The actions that I was able to get done were:  Idle, Idle Crouching, Turning in place, crouching turning in place, crawling, slide, jump, spraying, walking, running, opening door, closing door, and hitting a button.  Once I had all the animations, I had to hook them all up in the Animation Tree so they could blend together no matter which animation was called upon to play.  Here are some pictures and videos of all this stuff.

 

Fbx, Actor X,  weapon can problems and solutions, socket exploration,  Fixing crawling animation so the camera didn't go so far forward.  Camera sockets, Jamie integration demands and iteration for spray can.  Fixing star of david texture on arm band.  Re exporting the rig with no controls after animations were set.    Creating Materials, anim sets, anim trees: physics blending, blend nodes, slot nodes.  What I would do differently:  use slot nodes to make more dynamic movements, and possibly for controlling the arm better.  

Monster:

Coordinate with greg on rigging preferences.  Fix rig:

Rename all bones, and create an export friendly rig to animate and export with.  Change of plans with one weeks notice to add monster to game.  Create teaser trailer and temporary death cinematic:

Photoshop and monster composites, sound work, video editing.  


Figuring out integration of videos into UDK

Bink video, looping video projected on a virtual screen with camera fixed on it to fake video in unreal...Because of material changes the same cinematic was replaced later with an in game monster looping the same aimation with a camera fixed on it in a black room to fake the effect.: Idea from Arkham asylum deaths, per Chris.  

Animations:

Crawl, walk, butterfly stroke run, swing, pole flip, jump swing, hang to roar, hang banging, tear apart, roar, slash

Integreation:

Skeletal Mesh, Anim sets, animtree, slot nodes, blending.  Physics states for use with ai monster, cryo monster, and Matinee controlled monsters.

Matinee events:  triggered death events in beta, moving triggers, scare events with no death threat.

one in water crawl area, one in blocked off hallway, one with cat eating, one with monster coming at you, me and ryan figure out how to use matinee.  One with monster running accross bars.

Gold:  Original glass and pole scene to just glass smashing.  scare monster in one of the note matinees, two monster scares when walking down tunnels.  One with monter pounding on glass in lab, One with monters coming out of cargo crates: also trick with invisible beam barrier to hold back, setting path collide to false, while player collide to true.  

triggers,

Kismet webs.

Notes: Materials, mesh, matinees, problems with exiting, losing player control, etc. Attempts at swf integration.  Key card triggers, and door dependancies  Compare bools, Int counter, three switch activaion in any order.  Coordination with Robert for help with logic.  Help message integreation.  Cat pickups.  Inventory intgration and coordination with Rob.  Light switching, toggle visible, toggle,   Alarm events, helping to find custom kismet commands that the programmers could make to help me with the kismet limitations.  Ending cinematic:

Fan creation, switch creation, matinee with director camera switching.  Switch from maya to UDK for bink limitations, reusing existing animations and adjusting layout for easier completion.

Project in General

Communication between departments, finding the vision, and plan, and making steps to get there.  All nighters, working with the group in the same room.  Learning from peers, and passing on information.  Finding answers from blogs, forums, and help pages, and just trying stuff until something works.  Finding hack as solutions when development time is tight.  Stepping up and filling in wherever you can makes you learn new things.  I stumbled on matinee and kismet because I wanted to take some load off of ryan, and ended up finding it to be really fun.  Testing extensively back and forth in the game to make sure every variable is accounted for before submitting.  Making sure you cant open a door that you arent supposed to , or a message that displays when it isn't supposed to and fixing all the comparisons to ensure it all runs smoothly.  Trying to figure out checkpoints.  Functionally restarting later in the level, but no trigger states could be saved easily, so we had to abandon.  We should have looked into that sooner, or done separate or streaming levels to make it much more efficient, as well as cut out a lot of the dependancies with who could work on the level at a time.