My Self Defense Coach (called Self Defense Training Camp in the US) is a Kinect exclusive game developed by AMA Studios and published by Ubisoft.

{youtube}HWNOCc9rbAw|768|432|1|{/youtube}

How it all started

FightersUncaged

In 2010 I was asked by Yves Grolet, if I was interested in helping finalising the characters lighting and doing the sound effects for Fighters Uncaged, the first game of his new development outfit 'Ama Studios'.

It was the first time since he left Appeal in 2001 (see Outcast II story) that Yves and I had been collaborating on a game. It was a good opportunity for working together again, and I was happy doing sound effects for a change, something I don't have often the opportunity to work on.

After Fighters Uncaged shipped, Ubisoft was interested in doing a sequel, provided AMA Studios could prove they could push the envelope much further and develop a much more advanced and interesting title. They however didn't have the technology to do a full blown realtime 3d title (the environments in Fighters Uncaged were pre-rendered CG) and Yves naturally turned to Yann and I to see if it would make sense to work with our technology, FreshEngine.

After some technology assessment it appears it did make sense, and I started in november 2010 working on a prototype scene called Lost Temple. The scene would be a demonstrator of the studio's ability to create top-of-the-line realtime 3d content for current generation (PS3/Xbox360), so that Ubisoft could decide if they wanted to proceed with the game development or not.

Above: Fighters uncaged used prerendered CG for environments.

Eventually, the game that would later become Fighter Within was greenlit and the production started, only to be slowed down weeks later by an order from Ubisoft asking to start working on yet another game for Kinect, My Self Defense Coach.

The game had to ship in October and we were in March 2011. The studio would have to develop a game from scratch in only 6 month.

My contribution

The team was split in two, but most worked on this new project. They knew they had not to fail this small game or the main project Fighter Within could have been jeopardized.

I was put in charge of training the art team to our FreshEngine pipeline, to write all shaders, and to write lua scripts so setup the scene and special effects. I would also do the terrain textures for the garden area.

I started learning about programmable shaders back in 2006 when we worked on the Playstation3 for our aborded project Strings Attached for SCEE.

As a Kinect game, this project was to be running exclusively on the XBox 360. My experience with the PS3 was a great help though, as the two platforms share a common ground in term of graphics capabilities, and I would use the same CgFX language. There was one thing however that was going to make me learn new skills such as linear and high dynamic range rendering.

LoftShaders

Above: The main room of the game's loft with the generic shader opened in Maya's attribute window.

HDR and linear lighting

TonemapCurves

HDR (High Dynamics Range) rendering has been one of the much anticipated features in realtime rendering for the past few years, but traditionnal 16 bits per component (64 bits total) HDR rendering has been too much of a burden for this generation of platforms, with rendering penalty as high as 100%.

However, the 360 has the ability to render semi-HDR into a 32 bits buffer using 10 bits of RGB and 2 bits for alpha.

Right: various tone map curves are explored to better exploit the semi-HDR mode of the 360. The red square represents the unit value. Input is on X axis, output on Y axis.

This was my first foray into the realm of linear lighting and HDR rendering. In order to tame the HDR lighting, a tone mapping function has to be applied to the rendering so that the brightest lighting values can be remapped to the low dynamics range of a tv or monitor.

This process is called 'tone mapping', and in order to help me devise the right mapping curve I wrote a small lua tool that would display various curve scenarios so that I could more easily anticipate the final result and choose the better one for the 10 bits dynamic range.

Left: My Self Defense Coach required a more cartoonish and lighter graphics style. Custom shaders were developped to accomodate the style.

Characters

For charcaters I had to develop a soft shading technique to give the rendering a slightly cartoonish quality. I also developped a semi-transparent shader for the third person view, so that the player's character would not bloc the view from the opponent.

As all my lighting was setup in tangent space (as of course were the normal maps), I used a hemispherical projection for environment mapping instead of a cube map. When using generic reflections, this is way faster than fetching a cube map and it doesn't require to transform the normal maps into world space to match the cubemap space.

Effects

I wrote some special shaders for effects such as water, ground and mirror reflections. All shaders were written in Cg and previewed in Maya before being exported to our engine. The preview was pretty faithful, except for dynamic textures that could not be rendered in the Maya viewport.

For the ground and mirror reflection, the team modeled a simplified version of the scene that I rendered in a separate render buffer using a mirrored camera (FreshEngine integrates mirror constraints). I then blurred the buffer using a two pass separable gaussian blur (7 taps), and then used the resulting texture as screen projection in the ground shader.

The mirrors on the room's left wall were even simplier as it didn't require the blur pass.

Screenshot04

Above: dynamic reflections are generated for the floor and the left mirror

For the water I used a combination of 4 procedural waves at the vertex level. Each wave can be oriented individually by binding a locator to the corresponding Cg input in the shader (in Maya you can extract a direction vector from the local matrix of any object and connect it to the vector input of a shader).

Wave amplitude, phase, frequency and creasing can be controlled individually, some attributes can even be controlled on a per-vertex basis with vertex color. This creates a nice moving water with actual 3d waves and works well for low to mid frequencies.

On top of that, I used a panning noise normal texture sampled twice at two different frequencies. This brings up the higher frequency wavelets much needed for capturing the nice small reflections of the sun.

WaterShader

For the foam, I used a tileable texture that would pan slowly and fade in and out according to some vertex color value. The alpha vertex was also used for water transparency, simulating the thickness of troubled water.

WaterShaderVertCol

Finally, a generic hemispheric environment texture is used on top of the synthetic sun. Lighting and shadows for static objects are baked into a lightmap.

TerrainAndWaterShader

Terrain

For the textures I used the same aerial terrain photo technique I developped for the Mountain Bike Adrenaline game, along with some extra source material from the excellent CGTexture site.

I used Maya's vertex color painting tools to paint blending weights for the terrain. These vertex colors are feeding the shader and each color channel is used by the shader to control layer blending weight. Each texture layer contains a mask of the texture's relief features that is mixed with the vertex control weight to produce a per-texel weight. This creates a 'feature-aware' blending and avoids each texture blending into another in a smooth, non realistic way.

Of course the mask is being packed into the alpha of the color texture and all is compressed using DXT5 in order to perform a single texture fetch.

Terrain earth Terrain earth mask