Kinect Died in the Uncanny Valley

If Kinect suffers a slow death because of this week's news, it won't be Kinect's fault.

With Microsoft's announcement that consumers can now purchase Xbox One without a Kinect bundled in every package, it hasn't killed the device's prospects, but it's certainly tempered them. Fantasia is looking pretty great, and I suspect we'll see some other Kinect games at next month's E3.

But the news bums me out. I've always liked Kinect, warts 'n all. Most of Kinect's problems haven't been the result of shoddy technology. It's because designers keep asking the technology to accomplish tasks it's not very good at, and would likely never be very good at. This is partially Microsoft's fault. It planted the wrong seeds into the minds of developers, and only a few realized Microsoft was sending the wrong message as soon as the product was announced.

I was an early champion of the Wii, and the same was true for Kinect. I've spent years playing games with controllers, but the concept of interacting with games on a physical level, echoing a large part of my youth, has always been a tantalizing prospect. In the past 10 years, it seemed like games were heading in that direction.

Let's rewind to the original announcement for Kinect, back when it was called Project Natal in 2009. This was the same year director Steven Spielberg came on stage to tell us how excited he was about gaming.

There are several theoretical uses for Kinect in this video: becoming a kung-fu master, piloting a steering wheel in a race car, swinging a monster's arms around while destroying a city, fully controlling a soccer player, riding a skateboard, hitting an imaginary button in a game show. There's a common thread between these ideas, and it's that Kinect can replicate reality. There's one-to-one interaction between player and technology.

Microsoft's premise argues motion control can replace the controllers that we're used to. The subtext is that controllers, compared to Kinect, are an inferior form of interaction. The company's "you are the controller" message underscored this. In reality, Microsoft had it backwards. Motion control technology, at least as it exists now and for the foreseeable future, is not great at replacing what controllers are good at, but it's fantastic at replicating a form of reality. Only a few developers actually realized this was the true potential behind Kinect.

I'd call this the uncanny valley problem in motion control form.

The uncanny valley is when technology is able to almost mimic reality. You know, like this:

Hold me.

When Project Natal became Kinect in 2010 and Microsoft unveiled the first wave of games to use the hardware, there was a clear standout: Dance Central. It's easy to credit some of the runaway success Kinect experienced (more than 24 million units sold as an accessory--not bad at all) to Harmonix's dancing game. Dance Central wasn't about mimicking reality. It's easy to imagine another developer making a dance game using a camera capable of tracking your skeleton would have your dance moves replicated on-screen. Dance Central smartly took Kinect's limitations at face value and found a way to leverage what it was capable of.

We already have a pretty good idea how that might have turned out:

Compare that to Wii Sports from a few years earlier, an inspiration for Kinect. I didn't love playing tennis or bowling because the motion fidelity in Wii Sports realistically recreated what it was based on, I loved them because in a very video game-y way, it approximated the experience in the pursuit of fun. It felt like tennis and bowling. Wii Sports may have been designed that way due to technological limitations, but it was a better experience. The march of technology can often blind us to the advantages of having rules and barriers.

Johann Sebastian Joust designer Doug Wilson was talking about this on Twitter recently:

Microsoft wanted Kinect to be something more than it was. It overpromised. Developers didn't course correct and take advantage of Kinect's strengths, they kept playing into its flaws. This developed into a narrative that Kinect was flawed. While I won't argue it's perfect, the problems have more to do with how it was used.

One of Kinect's most promising moments on Xbox 360 was Double Fine's Happy Action Theater. You know what Happy Action Theater does? It doesn't give a shit about Kinect's inability to properly track you. Instead, the designers incorporated the fuzzy nature of the technology into the aesthetic, and encouraged players to be subversive through design. Happy Action Theater relishes and indulges in Kinect's quirks.

It should have signaled a new way forward with designing Kinect games on Xbox One. Embrace what the device is, rather than pretending it's something else. Instead, Microsoft decided it would try the same thing all over again. See: Kinect Sports Rivals, which seemingly came and went without anyone taking notice.

At least we got this out of it.

It's not a huge surprise AAA game developers would target realism over and over again. We see that all the time in genres that have nothing to do with motion controls. What's more frustrating is how little Microsoft allowed independent creators to go wild with Kinect. Imagine if Microsoft had opened up Kinect to its independent scene on Xbox Live Indie Games. Imagine if Microsoft tried to do something like that with Kinect Fun Labs, a failed experiment almost nobody remembers because, once again, Kinect was exclusive to bigger developers. Imagine if Microsoft had created a publishing fund that encouraged creatives to try their hands at Kinect development. Instead, we were mostly left with what Microsoft backed and lots of fitness games.

Imagine.

We've seen people do amazing things with Kinect. How come none of this creativity translated to games?

Based on the conversations I've had with developers over the years, it's not for lack of trying. There were evangelists within the company who wanted to see Kinect achieve more, developers who wanted o try, but there was a very specific vision for what Kinect should be, and being more "open" wasn't part of that.

There's little surprise, then, that the most inventive use of Kinect in years, Fru, came from an independent developer. In Fru, players use their body to unmask hidden platforms, and use a controller to move a character around. Like Happy Action Theater, it's not concerned with absolute accuracy and embraces its imprecisions. The developers told me at GDC that they're building a new version for Kinect on Xbox One. Microsoft's takeaway should be to find ways to generate way awesome experiments like this. There's no sign of that.

It's sad. It's really sad. And that's without acknowledging how Microsoft has systematically dismantled almost every piece of its new hardware platform that was supposed to make it different. I don't know what the future holds for Kinect, but accessories in games don't have a particularly great track record, and it's not like Kinect on Xbox One has been at the center of the conversation around Microsoft's new machine. I suspect it will continue to be part of the interface, and we'll see some token games funded by Microsoft a few times per year.

It's probably too late now, but it's nice to dream. It could have been different.

Sorry, Kinect. It wasn't your fault.