Speaking during the "Biofeedback in Gameplay" talk at GDC, Valve's experimental psychologist Mike Ambinder - they have one - showed off the ways they're using biological measurements to enhance their games. They showed a version of Portal 2 where the player's view is controlled by where he looks in real life, and a version of Left 4 Dead 2 that senses your stress levels and adjusts the game accordingly.
Read on for the details, and video footage of both games.
Valve showed off facial recognition software and 10,000 dollar eye-tracking tech. They used the latter to play a short section of Portal 2, where the player uses their eyes to aim the cursor, and their hands to move around the environment. According to Ambinder, "The eye tracker is used to extract eye's X Y position. This data is updated at 60hz, before being fed back into the game engine."
"We don't know how the player is feeling," said Ambinder. "We'd like to attack that problem. We'd like to figure out player sentiment, and player emotion." He also hinted at future matchmaking profiles based on player's emotional responses: "We don't want to pair passive players with emotional players. It's something we're thinking about it, we think it's worthwhile."
Valve have been experimenting with varied techniques during the research. "We made a heartbeat measuring mouse - a detect heartbeat mouse - but had to ditch it because every time it moved it would introduce artefacts into the readings. We have a mouse that senses skin conductance, it's easy to use, pretty cool.”
Valve's solution is a piece of tech that costs $10 to make, and consists of two metal contacts attached to a strip. They're fixed to your hand by a bandage, and a USB cable sends data about the current passing between them to your PC. It's measuring the electric response of your skin, something which varies with stress.
They've been using biometrics like this to tweak Left 4 Dead 2: they've experimented with an automated AI Director that modified enemy spawns, health and weapon packs based on estimated arousal patterns. According to the surveys post playtest, the use of biofeedback information made Left 4 Dead 2 twice as enjoyable as without. Here's Mike Ambinder's explanation, with one player's stress levels on-screen as they play.
Mike talked us through some of the extraordinary ideas they have planned for this tech. "We can detect when a player is bored," he said, and suggested this as a way of determining if the player is lots, and using in-game prompts to help them out.
"Can we tie health to arousal? If you stay calm, your health rises... imagine a lie detection game: fool your interrogator by remaining calm."
Or an NPC who knows how you're feeling - "Hey Mike, why are you so sad?"
"Watching the arousal patterns of competitive players - just knowing if the competitors were aroused - could be interesting."
Mike said they could also use it to detect when your team-mates are in trouble in a co-op game like Left 4 Dead, by spotting spikes in arousal. In the video above you'll see the clear response when the player's attacked by the tank.
"Or [you could] earn points from eliciting responses from your team-mates," Mike adds.
They've even tried using an emotional tracker during games of DOTA 2, their forthcoming arena-based multiplayer action RPG. "People love it when they do something that makes their opponents suffer," says Mike.
Valve have always been a forward-thinking company, but even for them it's remarkable to see this kind of mad science demonstrated for all. Gabe Newell spoke to us about their plans to use biometrics in games back in September last year. Now we know they're actually going to do it.
But will players go for it? Would you buy a $10 glove that would let you play a version of Left 4 Dead 2 tailored to your emotions? And would you want other players to be able to see that info about you?
