E3 2009

[Asher]

Same as MS - just they seem to be doing it much more successfuly.

In some respects, it's a me too -- it's motion control.

But it's in a completely another league. The PS3 solution is another implementation of the Wii Waggle system, which I find to be wholly inadequate.

The MS solution is a "me too" in it's motion control, but without a controller. There's a huge difference.

- It also integrates a mic array for voice recognition and spatial recognition (where is the sound coming from, which speaker is it coming from?)
- It integrates IR projectors which determine with pretty amazing accuracy the depth of all objects in its image (a z-buffer) and two optical HD cameras to judge x and y-axis positioning (and provide reference for the z-buffer)
- It has an onboard chip to do realtime 3D modelling of the scene in question (does not require additional CPU usage to merge the two HD images and apply the z-buffer IR results). It also maps the skeletal points of people on the screen to the 3D scene which it can send to the 360 for processing.

I've been digging around at the Microsoft Research work Molyneaux mentioned they're leveraging for this project. I've only looked at the MS Research "Speech Group" first, but with very interesting results:
Understanding user's intent from speech: http://research.microsoft.com/en-us/...g/default.aspx
Microphone array processing and spatial sound: http://research.microsoft.com/en-us/...g/default.aspx
Language modeling: http://research.microsoft.com/en-us/...g/default.aspx
Multimodal conversational user interface: http://research.microsoft.com/en-us/...u/default.aspx
Speaker identification (who is speaking?): http://research.microsoft.com/en-us/...d/default.aspx
SAPI (Speech API): http://research.microsoft.com/en-us/...i/default.aspx

A lot of this stuff was explicitly developed with an Xbox 360-like environment in mind:

[DrSpike]

It definitely looks in a whole different league I'll grant you. I'm usually sceptical in this area because it's easy to make stuff look cool and innovative but harder to turn it into a compelling game. But I'm genuinely intrigued after the E3 presentations to see what they will come up with.

[Asher]

More Forza 3 media, in game:

http://gamersyde.com/stream_11302_en.html

http://www.gametrailers.com/video/e3-09-forza-motorsport/50406

Forza 2 vs Forza 3...wow:






and a Hands On: http://e3.gamespot.com/story/6211100...ort-3-hands-on

[DarkCloud]

I am a bit skeptical that the new Microsoft controller will really work the way they say it will... how will it handle "noise" in the background of people moving... dogs wandering across the camera... obstacles in the way like drapes or coffee tables or pizza?

Though it does sound intriguing.

[Asher]

I am a bit skeptical that the new Microsoft controller will really work the way they say it will... how will it handle "noise" in the background of people moving... dogs wandering across the camera... obstacles in the way like drapes or coffee tables or pizza?

Though it does sound intriguing.

Pretty basically.

Brief rundown how it works:
There's an IR "gun" which is used to determine depth of objects from the TV. Twin HD cams also capture data of what's going on.

A processor then analyzes the image (it takes 5 frames or 117ms to initially "calibrate" to a scene and locate people in it). It then tracks those people. It analyzes the people and constructs a virtual representation of their skeletal structure and keeps track of them. If a dog walks infront of the screen, it'll not be detected as a human as it's in no way a humanlike form. If another human walks into the field of view, as the enter the field they'd be tagged as another human and not interfere with the recognition of the original. Mind you, if it obstructed the view of the person that'd have an impact.

It doesn't sound like it'll be an issue at all, nor does it look like it'll be an issue judging from the demos shown live.

If there's obstacles in the way, such as when your hands are behind your torso, etc it uses prediction. If you are swinging your arm and it goes out of view behind you, it assumes it'll keep moving adhering to the laws of physics unless it knows otherwise.

I don't see that being an issue either, nothing gamebreaking.

[Asher]

More info on a tech the Natal uses: http://en.wikipedia.org/wiki/Time-of-flight_camera

[Asher]

http://gizmodo.com/5277954/testing-project-natal-we-touched-the-intangible?skyline=true&s=x

Matt Buchanan tested Project Natal today, as did I. Here is his personal take on the technology right alongside mine. We did not share our independent experiences before pasting the text below. Neither of us were allowed to shoot what was happening on screen—hence the crazy pics of our bodily reactions, and that intensely audible racing-game video.

How Natal Works
The test system was an ordinary Xbox 360, connected to small PC and camera that simulates the final Natal rig. There are two cameras—one RGB, for face recognition and display video, and one infrared, for tracking movement and depth. Why infrared? The eye doesn't see infrared light. And when you combine an infrared camera with an infrared emitter (also part of Natal), a room is flooded with a spectrum of invisible light that works in the dark.

Natal also has its own internal processing system handling an unspecified amount of the heavy lifting behind Natal's cleaver image and speech recognition. It breaks the human body into 48 points tracked in real time, and it can sense your whole body in Z space, or depth. In fact, on a heat map that measured depth, my hands appeared hotter than my shoulders—because they were closer.

Natal is so smart, in fact, that, if your room is narrowed by a pair of couches, it can signal to a game to narrow the level. It can see about 15' x 20' of a room, according to project leader Kudo Tsunoda's informal estimation.

Breakout

Matt: My first taste was talking to the father of Project Natal, Kudo Tsunoda and watching as his simple, small hand gestures were mapped perfectly onto the screen. He started up the ballsmacker demo you might have seen in our liveblog, knocking a swarm of balls into wall with every part of his body.

When Kudo gestured to me try it, I jumped right in and immediately started smacking at balls with my hands and feet and knees and arms and head as one ball exploded into many, like a virus, until I was doing sad white ninja jerking and jumping movements. Kudo didn't tell me how to "set it up" or what to do. I just did it. You have to realize, Kudo towers over me. I didn't have to calibrate it to my body size, or stand in a weird way for it to adjust. It just worked. Well, until I broke it at the end—it froze up after a few rounds and had to be rebooted for Mark. Hey, it's an early tech demo, so don't read into it. Until that point, it worked remarkably, incredibly well—better than I expected, honestly. The bright fluorescent lights were turned off and on, and Natal didn't flinch. My real movements translated exactly how I expected them to—the precise position, velocity—90 percent of the time, no matter how ridiculously I moved, and some of the other 10 percent might've just been my own bad timing. But the result is a remarkable sense of control. Immersion.

Mark: Microsoft loaded the 3D Breakout demo we saw at their press conference. I stepped up to a white piece of tape right after Matt, and given that I'm 4 inches taller, Natal needed to account for my larger size.

After about 10 seconds, the blue, ghost-like figure filled in. And he was both taller and bigger-handed than Matt's avatar. Natal noticed that I'm a bigger guy. It made no adjustments for the fact that I'm also better looking.

The first thing I noticed was a slight lag I hadn't intended. It's not horrible, but my avatar moved a hair more slowly than I did. That didn't stop me from reaching up, spiking the imaginary ball at a wall imaginary bricks, and then flailing around to keep up with 2, 3, 4, 5 and more spheres flying at me at once.

My avatar recognized both my pitiful kicks and swipes. And while my avatar never left the ground when I jumped, this turned out to be but an animation limitation within Microsoft's tech demo. My wireframe preview image and heatmap did leave the ground. Besides, this is nitpicking. On the PS2 I played Nike Kinetic, something a bit similar. And I always wanted to be having fun. But on Natal, even in a stuffy windowless room surrounded by Microsoft execs, I was having fun. (Disregard my stern, focused face in these pictures.)

Burnout Revenge
Matt: The Burnout racing-game demo was a little more abstract—in one sense, I almost wished I had a wheel to turn, a pedal to press, because I wanted the feedback. I had trouble getting used to "pressing" the gas, which you do by moving your right foot forward. I threw myself off-balance by taking a ginormous step toward the Frankenstein's lab of demo equipment along the wall (upon which I could see myself represented in infared, covered in boxes like smallpox). But turning my air steering wheel, I felt completely in control. A lot of that was the software—it registered even the smallest pivots of my elbows that sent my forearms right or left—but the way it responded exactly how I expected it to is what made it feel so natural. Which is the real key here. It feels natural.

After I hit full speed on a straightaway, I tried to do a 180. I crashed into a wall and died. Normally, that'd make me bad. But I couldn't stop smiling that I'd held the future of gaming control in my hands—and it was simply air.

Mark: As soon as Matt crashed, I greedily jumped in, asking him if it was OK but not waiting for him to answer. I wanted to play Natal more, and I've played a ton of Burnout.

Burnout showcases a few important points for Microsoft. First, it's a real game that's been on the 360. So Natal doesn't weigh down on the processors so hard that you can't play games. Second, it requires fine motor control.

I raised my hands in the air, mining a steering wheel. I hadn't given the system any time to scan my body after kicking Matt out, but I stepped by foot forward, signaling the gas all the same. The car accelerated. I twisted my arms. The car turned just the right amount.

Microsoft had clearly tweaked the Burnout code a bit, forcing the car to feel a bit more like a powerful sedan than a street illegal beast out of some Fast and Furious sequel. And I'm guessing that Natal's ever so slight control delay was masked by the feeling of a looser-driving steering wheel that we find in more standard cars.

So I floor it, growing confident as I wave through traffic and slowly build speed. I reach maximum velocity, throw my foot back to break, cut the wheel and toss the car into a spin. Yes. This feels right. Just right.

Holy ****.

But Natal can't work this well. It just CAN'T. I need to break it, teach this Microsoft prototype a little humility. What if I stand on my tip toes and steer eight feet in the air?

The car handles fine.

What if I kneel on the ground and steer?

Yup, it still works, save for a moment when my knee shifted and I tricked the machine—a fair mistake, even by my highly ridiculous dork standards.

Closing Thoughts
Matt: Project Natal is the vision of gaming that's danced through people's heads for decades—gaming without the abstraction of controllers, using your body and natural movements—which came more sharply into focus when Nintendo announced the Wii a few years ago. I haven't been quite this blown away by a tech demo in a long time. It looked neat onstage at Microsoft's keynote. Seeing it, feeling it in person, makes me want to believe that this what the future of gaming looks like—no buttons, no joysticks, no wands. The only thing left to get rid of is the screen, and even that'll happen soon enough.

Mark: 2010...or maybe even 2011...is just too long to wait. I want Natal now.

[Asher]

No more Microsoft Points on Xbox Live

[DrSpike]

They're not doing away with them completely - they are expanding options for buying content.

So no more points for me and you, but the younger crowd will still use them.

[DarkCloud]

Asher- Thank you for the explanation. I am still skeptical because I do not know of any other technology that acts similarly, but I am hopeful that it will work, especially since they are at least cognizant of the possible problems.

Should be interesting.

[vee4473]

No comment on the star wras the oldrepublic cinematic? I know cinematics have nothing to do with how good a game will be, but nothing from you guys?

[Asher]

Correct

[vee4473]

gee whiz.

i'm trying to temper my hopes with the knowledge that cinematics mean nothing, now i'm really depressed....

[Asher]

Why?

While there might be a physics engine that applies to the jugs, I doubt that an entire engine was written specifically for the funbags. - Cyclotron - debating the pressing issue of boobies in games.

BTW, Dead or Alive for Xbox had breast physics specifically coded.

[Guynemer]

Boobies