Independent Study Post-Mortem

For my independent study this semester, I wanted to do something to be able to apply what I learned in the Game arts II class, animation.  Since I was creating a rig, and doing in game animations for the Projects class, I thought it would be good to do an additional project using that character, that may or may not actually go into the game, but be a project that would be the work of an entire semester.  What I decided to do, was to create an escape cinematic that would explain how the main character of the projects game escaped.  For the projects class, I was not set to work on cinematics, and the end was at the minimum just going to be a picture of the character jumping out into Lake Erie.  I also specifically choose this sequence, because it would include many complex animations, not all of which were cycles, like in the game.  This would give me an opportunity to stretch myself and create an entire animated scene, which I had yet to do.  I would have to use the principles of design, and film techniques to set up the shots.

To begin I started with a story board.  On the storyboard I planned out each shot I wanted to make, what type of shot it was to be, ie; medium, close up, etc., and what was to happen in the scene with basic stick figure drawings.  This I reviewed with Craig to get approval that the scene was clear enough, and that it would be a worthy assignment.

 Once that was finished I began to block the set out in maya.  This included creating several set pieces to be used, such as the ventilation fan, the switch, the corrugated tunnel, the dynamic water and the skybox that showed the Monroe Nuclear Power plant in the background.  I then laid out how each shot with a separate camera in Maya, named with the shot number, so they could be switched to and from easily in the panel viewer.  I then Began to block out the movements, and timing along with all the camera angles, so that the entire sequence could be watched in maya.  This was where most of the animation came in.  I had to make basic animations for the male character, as well as the monster, the doors to the room, the fan, the grate in front of the fan, the grate in the vent, and the water flowing.

I actually got very far in maya, even to the point that there was more than just blocked out animations.  I also had the lighting all set up, with fogs, and particle emissions.  I even had set up a dynamic water volume so that I could make a more realistic looking fall into the water. I began the process of rendering, and that is where I hit a snag.  This scene was extremely complicated, and the render times were measuring in the minutes for a single frame.  Even if I could have sent it to a render farm, it would have taken far too long to get the whole thing rendered.  Then came the issue of whether it could even be used in game.  Well, this was fine with me, as I had never specifically had the goal of integrating the sequence into the actual game.  The rest of the team however decided that I needed to get it working in game.  I began to look into the options of what kind of video could be integrated into the Unreal engine, and I game across bink video, which is supposed to be integrated easily into unreal.  The problem though, was that it could only be used as a texture during the actual game, unless you have the pro version of unreal.  To test the possibilities of this working, I created a few death cinematics for the game, none of which made it in, and were mostly to test whether I could get the big cinematic in.

I was eventually able to figure out how to get this working in the game, by setting up a virtual screen room, with an in game camera pointing at it, but the only problem was there was no way to tell the texture to play or not, it could only loop, so for the more linear sequence at the escape needed to be started at any given time, and would not work this same way.  The only way I could get the sequence to work in the game was to create the whole thing within the engine.

I had to begin nearly from scratch.  I had to create a new version of the layout that I had made in maya, with the existing meshes that were in the game.  I had to get the assets like the fan exported and working in game.  I had to create all new cameras and lay out the whole scene again in the unreal editor.  I also had to learn how to use the in game track editing system, Matinee Editor.  As a result of this complication, I decided to slightly simplify the flow of the scene, I cut out the ventilation tube, and had the fan go directly into the water tunnel, and have it go straight out into the water, rather than curving around.  I also decided to have the grate that was over the fan already on the ground, so as to eliminate that additional animation.  It wasn't that I didn't want to do all the animations, but I was now under a time crunch, and I just wanted to see a completed project, even if it wasn't how I had originally envisioned it.

Now came the tedius process of animating through matinee.  For any animation that you want to use, there has to be an additional channel in the slot node in the animation tree for the character.  The animation tree is how the blending between animations is done.  You must create the animation keys in maya, and then export them into the animation set, and create a blend node in the animation tree.  Once all this is done, you can use that animation in matinee.  Over the next few weeks, I got the animations all laid out with the animations that I currently had.  If I had had more time I would have added a few more animations to make the transitions a little better, such as going from roaring to walking, going from walking to slashing, and having a reaction animation for when he hit the fan.  I also would have added the main character hitting the button and jumping into the pipe a little less awkward.  Next is to set up a director track i matinee, where you can animate the camera positions, switch between cameras, and add sound tracks.  

Once all the shots were set up how I wanted, I began to add the sounds, and edit the curves turning sounds, animations and cameras on and off.  It is now finally complete enough to be included in the game.

I have learned a lot from this process.  First I learned to take an idea to a storyboard.  Use that storyboard to layout, setup camera shots, and lighting in maya, and beging blocking out animations, and iterating on feedback.  I also learned about adapting to extenuating circumstances, and figuring out how to still bring your creative vision to life, in some way you can be satisfied with.  I learned many new tool sets, and skills regarding the practical application of animation in game production.  I also got a taste of everything from modeling, texturing, rigging, animating, and editing as a result of my efforts to complete the work for the Independent Study project, that I would not have learned from the projects class alone.  Things I would do differently had I had the chance... I would have looked into the demands of the medium before I got so far down the wrong road, ie; had I known movies were such a problem to integrate, I would have planned from the beginning a much better setup to be created in game, rather than scrambling to get it together after I had to scrap half my work.  At the same time though, I am grateful to have had to go through both, because I learned new things specific to each medium that I wouldn't have learned otherwise.  I consider my time spent on this Independent study project to be very valuable, and the class study to be quite worthwhile. 

Projects Class Post-mortem: My work on ERIE this semester

Since I have done so much this semester :), in order to have some sort of coherency in this paper, I am going to organize my thoughts by when they happened.  From the beginning of the project I had volunteered to do the rigging and animation for everything in the game, and from right off the bat I ended up doing much more.  In the beginning I created a bat.  In order to bring life to the (literally) dead world, I decided to make some non-player characters.  Last semester I grabbed the reigns and figured out how to get a rat modeled, textured, rigged and animated and working inside of unreal.  This semester I made the rat's cousin.  The bat added a new challenge- the wings.  in order to make the wings foldable and movable without having clipping problems the skinning has to be exactly the same on the front and back of the wing.  To do this, the vertices of the wings needed to match up in the front and back, and then when the weight is assigned to the vertex to the corresponding bone, the front and back vertices need to have the exact same weight value.  When I first learned how to skin meshes to bones, I used the paint skin weights tool, which is an interactive painting tool on the mesh that is a grayscale where white is one and black is 0 influence.  That works great for most things, and is an easy way to see where the skin will deform, but in order to get exact numbers I had to learn a new way to weight.  That tool is the component editor.  In that editor, when you choose the vertices, it populates a spreadsheet with the vertex numbers on the left side, and across the top are what bones are influencing those vertices, and what float value they have.  In this way I quickly could see what value the vertex had, and make sure the corresponding vertex on the back side was exactly the same.  That way, no matter how far you bend the wing, the two layers of geometry wont overlap, causing the common 3d flaw of clipping.  The next thing I learned with the bat was about creating normal maps.  The texture on a 3d model is assigned to the surface by way of UV layouts.  The uv's correspond to the vertices, but in the UV layout editor, you can flatten by way of cutting up the model until it is all in one square area.  You can then use photoshop or another similar program to create textures.  They way I like to think of it is like a bearskin rug.  You take a bear you cut it up, and lay it flat.  That is great for making the color texture, but there are many other maps that a 3d model can have in order to give it a more dynamic look.  One of the most important is the normal map.  That map is one that takes a violet color scale to artificially create bump texture, and is much more effective than a grayscale bump map.  The problem with it is you either can make them by creating a high poly version sculpted out the way you want it to look, and then baking it out to project onto the low poly version, or you can use a program like Crazy Bump to take the color texture you made in photoshop, and it will create a bump normal map for you, as well as specular, occlusion, and a few other maps.  When I learned this process, I also went back to the rat model, and fixed a few problems it had with the seams, then I made a brown and black version of the rat texture, and used crazy bump to make a normal map for the rat.  Here are some pictures of the rat and bat to illustrate this concept..

 Bat Animations: Hang to flap, flapping loop, flap to land

Integreation: skeletal mesh, anim set, animtree.

Matinee

Next, since it sounded so much like rat and bat, I decided to go with the cat.  For this one, I wanted to learn how to use mudbox to make the texture and normal map.  For this I created a base low poly mesh for the cat, and then exported it to mudbox, where many subdivisions are added to the mesh, allowing for almost literal sculpting like you would do with clay.  Once that process is done, you can bake out a projection of these folds and bumps onto a low poly mesh as a normal map.  Here is a video and pictures of the process of modeling and texturing the cat.

Integreation: Because of the time constraint to get the main character in, once Ryan finished with that mesh, I had to stop the cat there until I was finished with the higher priority task of rigging skinning and animating the main character which we had decided to fully animate and be able to see through a first person view at any time.  After I was done with that  the monster became another priority, so the cat never received a rig, but it ended up having an unusual but entertaining part of the game.  We made the dead cat with its awful dying sound a hidden pickup collect-able, for which I was responsible for integrating, but I will go into that more when I get to Unreal Kismet, and how I stumbled into that.

Isaac/Clarke/Oliver

Clarke is the pet name I gave to the main character when we were in transition between 1940s German setting to 1960's Michigan.  I worked with Ryan to ensure proper deformations for animation were on the mesh he created, and when the file was finally passed on to me I had to build a complete rig from the ground up.  After laying out the bones, I created controllers to be used to keyframe on for the animation.  I also used the connection editor to transfer custom channel controls to those controllers such as IK switching, and Custom Channels, with Set Driven Keys.  when you create a controller, you can add attributes with a min and max value.  Through the set driven key editor, you select what you want as the driver, in the case of a rig, the controller's custom attribute, and the driven, which in the case of animation, I used the rotation attribute on the bones of say a finger.  Then you set a key at 0, then move the attribute in the channel box to -10 and move the bones in the finger into a curl, and set another key, and with 10 move the finger outstretched and up as far as it should go, and set another key.   Once you do this, the custom attribute can be used as a slider, when you slide it you will see the model moving from a curl to extended and everywhere inbetween.  In this way I created custom controls for foot ball, toe heel and ankle rotation, toe lift, individual finger curls, fist curl, thumb rotations and curl, Jaw grind, mouth open, and a few others.  I also created IK/FK functionality for the arms and legs.  IK standing for inverse kinematics allows you to grab the ankle and the rest of the leg will move with it.  FK for Forward Kinematics allows you to rotate a joint and the children of that joint will follow exactly relative to the parent joint.  Both are usefull to use, and blending between the two allows the most control for animation.  Using both methods I learned while painting the skin weights on the rat, I skinned Clarke until I was satisfied he could do all the animations I needed him to do for the game.  Then, I was off to animation.  For Clarke's animations we were not exactly sure how we were going to integrate him, so I ended up having to make 3 sets of animations for every action.  One with full body movement, one with his right arm hidden, so as to replace with a socketed arm(which didn't end up working) and another set with and without him holding a spray can.  The actions that I was able to get done were:  Idle, Idle Crouching, Turning in place, crouching turning in place, crawling, slide, jump, spraying, walking, running, opening door, closing door, and hitting a button.  Once I had all the animations, I had to hook them all up in the Animation Tree so they could blend together no matter which animation was called upon to play.  Here are some pictures and videos of all this stuff.

 

Fbx, Actor X,  weapon can problems and solutions, socket exploration,  Fixing crawling animation so the camera didn't go so far forward.  Camera sockets, Jamie integration demands and iteration for spray can.  Fixing star of david texture on arm band.  Re exporting the rig with no controls after animations were set.    Creating Materials, anim sets, anim trees: physics blending, blend nodes, slot nodes.  What I would do differently:  use slot nodes to make more dynamic movements, and possibly for controlling the arm better.  

Monster:

Coordinate with greg on rigging preferences.  Fix rig:

Rename all bones, and create an export friendly rig to animate and export with.  Change of plans with one weeks notice to add monster to game.  Create teaser trailer and temporary death cinematic:

Photoshop and monster composites, sound work, video editing.  


Figuring out integration of videos into UDK

Bink video, looping video projected on a virtual screen with camera fixed on it to fake video in unreal...Because of material changes the same cinematic was replaced later with an in game monster looping the same aimation with a camera fixed on it in a black room to fake the effect.: Idea from Arkham asylum deaths, per Chris.  

Animations:

Crawl, walk, butterfly stroke run, swing, pole flip, jump swing, hang to roar, hang banging, tear apart, roar, slash

Integreation:

Skeletal Mesh, Anim sets, animtree, slot nodes, blending.  Physics states for use with ai monster, cryo monster, and Matinee controlled monsters.

Matinee events:  triggered death events in beta, moving triggers, scare events with no death threat.

one in water crawl area, one in blocked off hallway, one with cat eating, one with monster coming at you, me and ryan figure out how to use matinee.  One with monster running accross bars.

Gold:  Original glass and pole scene to just glass smashing.  scare monster in one of the note matinees, two monster scares when walking down tunnels.  One with monter pounding on glass in lab, One with monters coming out of cargo crates: also trick with invisible beam barrier to hold back, setting path collide to false, while player collide to true.  

triggers,

Kismet webs.

Notes: Materials, mesh, matinees, problems with exiting, losing player control, etc. Attempts at swf integration.  Key card triggers, and door dependancies  Compare bools, Int counter, three switch activaion in any order.  Coordination with Robert for help with logic.  Help message integreation.  Cat pickups.  Inventory intgration and coordination with Rob.  Light switching, toggle visible, toggle,   Alarm events, helping to find custom kismet commands that the programmers could make to help me with the kismet limitations.  Ending cinematic:

Fan creation, switch creation, matinee with director camera switching.  Switch from maya to UDK for bink limitations, reusing existing animations and adjusting layout for easier completion.

Project in General

Communication between departments, finding the vision, and plan, and making steps to get there.  All nighters, working with the group in the same room.  Learning from peers, and passing on information.  Finding answers from blogs, forums, and help pages, and just trying stuff until something works.  Finding hack as solutions when development time is tight.  Stepping up and filling in wherever you can makes you learn new things.  I stumbled on matinee and kismet because I wanted to take some load off of ryan, and ended up finding it to be really fun.  Testing extensively back and forth in the game to make sure every variable is accounted for before submitting.  Making sure you cant open a door that you arent supposed to , or a message that displays when it isn't supposed to and fixing all the comparisons to ensure it all runs smoothly.  Trying to figure out checkpoints.  Functionally restarting later in the level, but no trigger states could be saved easily, so we had to abandon.  We should have looked into that sooner, or done separate or streaming levels to make it much more efficient, as well as cut out a lot of the dependancies with who could work on the level at a time.  

Some of my work on ERIE

It has been quite a semester, and I have jumped into Kismet, and Matinee, and learned a ton in the process.  I basically built the whole game's level functionality, as far as pickups, keys, doors, and anything moving in the level.  It was really fun to put together, but it took way too many All-nighters.

Monster animations

So, this last week as a last ditch effort, I had to get the monster into the game for beta.  This meant getting the rig ready and making sure everything got into UDK, as well as animating all the animations I could think of.  Here is a screen capture of the animations I have so far in my AnimTree in UDK

Escape into Erie!

So this is a rough out playblast of the shots and timing with basic movement to get an idea of what the ending cinematic for Erie will be like.  For this I mostly used Ryan's props, but made a few of my own.  I am also doing the sound for this scene as my final project for my sound class.

Erie Teaser

Here is a quick video I put together as a death sequence for the game, and we are using it as our teaser trailer for the game website.

Another Updated Reel

Shot Breakdown

1. Regular Walk Cycle - Animation
2. Leg/Hip Walk Cycle - Animation
3. Rat Idle/Sniff Cycle - Model, UV, Texture, Rig, Animation
4. Rat Walk Cycle 1 - Model, UV, Texture, Rig, Animation
5. Rat Run Cycle- Model, UV, Texture, Rig, Animation
6. Rat Walk Cycle 2 - Model, UV, Texture, Rig, Animation
7. Luxo Jr. in "Attack" -Animation, Sound Design
8. Sneaky Walk Cycle - Animation

Clarke!

So this week has been a nightmare trying to get the new player rig into UDK, but after many many hours I finally got it in and working, so I could start doing the animations.  So far I have a walk while holding a flashlight, a spraycan, and the same again only running, and also a sliding animation for the slides that he will do in the game.  Here is a video of the POV and the Front View of the Slide animation.  Really simple, yet effective for the game I think...

Cat

So I have started working on the third NPC of Forever Hunted, the cat.  For this one, I started playing around in Mudbox to get some cooler Normal map textures.  This was a completely different process than what I used to make the bat, but both were super fun, and I might just have to go back and throw the bat in mudbox to get an even cooler normal map.  Here are some screen shots of what I have so far.

Finishing the Rat and Bat in UDK

So, this week I worked on fixing a few things with the textures in the rat, including making a black and brown fur version of the material, and getting it all imported into udk.  I also made the specular, normal and occlusion maps with crazy bump for both the bat and the rat.  Then I got the last animation done for the bat, which is going from flying to landing back upsidedown and being able to loop through the whole thing smoothly.  I then got everything imported into udk, and made the new bat package!  Take a look!

Bat Animation

So I have finished the animation of the bat going from hanging to flapping, and then a flying cycle.  Here is a video of the two put together.

Gone Batty!

So now that summer is in full swing, the art team is meeting weekly to continue working on Forever Hunted, and I have been working on making additional NPC's.  Now my rat will have a friend in the dark sewers of 1940's Germany.  Here are some shots of the Bat I made today.

Latest Animation:Dialogue Acting

Projects Class Portfolio of Work

Now that the end is fast approaching(at least for this semester)  I thought I would recap everything I have done to contribute to Forever Hunted so far.

To start off with...

In the beginning, there were design brainstorm sessions I had fun with.  One of my ideas was to have a monster similar to an angler fish:

The idea of the angler fish is that it hunts using a lure that glows, while hiding in the darkness.  In the game we would have had something like that, where there is somthing that looks like an item glowing, and you figure you will go pick it up, but then when you do, the rest of it lights up, and goes to chase you.  For sake of everyone, the time it would take to script out a sequence like that was just not feasable for alpha, but everyone agreed it was a pretty cool idea for the game.  It was also my idea that the monster should have a really long tail/ side profile so that when it is running past a hallway, you have a longer chance to catch it out of the corner of your eye, and it also makes it seem bigger, since it can't be super tall in a sewer. 

As far as reality was concearned though, we had to get down to business, and get something working for alpha, so everyone agreed to use gregs raptor model he had made previously while following a modeling tutorial.  It was my job to take the model, create the rig,

skin weight,

 and make a simple animation

to use as a stand-in until we had the new original monster created.  My first few weeks of work I spend on that, as well as figuring out how to get it all into Unreal, which took a lot of trial and error before it worked decent enough.  At first, the skeleton was rotated 90 degrees, and then it was just not centered.

So, I worked on it some more, and decided I didn't want to wait for someone else to check on it to see if it was working, so I decided to open up udk myself and try to figure things out.  It took about a half hour, and I had the raptor successfully walking in UDK!

 Note that there was never a UV map, or texture made for the stand in.

Once I was finished with the raptor, getting it exported and packaged up, I was assigned to work on some other non player characters.  I had this idea that rats were really effective in creating fear without having to be an aggressive character.  I thought of Indiana Jones and the Last Crusade, where they are in the catacombs and the rats all of the sudden are swarming away from where they are trying to go, because they know there is going to be a huge explosion coming.  I thought for the game it would be really awesome to have swarms of rats everywhere, and when the monster would come near, they would flee away from it, so if you saw a bunch of rats coming at you, you would know the monster was not far behind, and to book it out of there.  Since we are trying to go HUDless, I thought this was a great way to alert the player to the monster's position aside from just the sound cues. 

This time I had to actually create the model of the rats,

(not final Model)

 as well as texture,

rig,

 skin and animate.

  After getting the raptor into UDK, it was much easier to get the rats working properly.

After that was all done, I finally got the low poly model that the final monster will be based off, so that I could start making the rig, and a stand in animation to get working for alpha.  With the rig, I created custom controllers, with controls for jaw movement, nostril flare, eye movement, individual automated finger curl sliders, a finger spread controller. A complete foot controller with heel, toe and ball rotation, pole vector locators for knee and elbow positioning.  Tail , spine, and neck controllers, and a clavicle controller.

  It can be positioned into any imaginable pose. 

The skinning I have done for the alpha version is nowhere near complete, because it will just have to be completely redone once the final mesh is finished, and I am not satisfied with the animation, but it will have to do for now.  I was able to help get the textures from Greg into UDK, and running in time as well.  I also began doing a pounding Idle animation, which looked awesome, but for some reason I could not get it to export correctly to UDK, and I eventually crashed Maya, and lost the animation.  Since time was running out, we will not get to see that animation in the alpha...Sorry.

Oh ya.  One more thing.  I am helping Chris with sounds, and have made a test monster scream, but it was not used.  I also recorded myself running around in the studio, and breathing and stuff, and for now, the sprinting in the game uses my heavy breathing that I recorded!  I also recently recorded some running foley so we don't have to use the default run noises.   I also recorded 81 original sound effects specifically to be used in the game.  The library includes metal clanks, stabbings, punches, bone breaks and crushes, walking/running, growls, screams, knifes, pounds, rat noises, slime movement, and ambient water, light, and fans.  They were recorded in the sound room at the U, so they are fairly good quality, and can be edited to our liking.  Here is an example of a soundscape created 100% from the recordings I made:

(Please Listen With Headphones)

This has been a great project to work on, and I feel like I have learned and grown a ton since the beginning of the semester.  I am excited to keep working on it, and to finish it up next semester.