By now, Nintendo has made exactly 87,493,029 versions of Mario Kart since the game was first introduced in 1992 for the Super Nintendo. (Okay, the company has really made 13—which is still a lot!) But a new sequel coming this fall to the Nintendo Switch changes the formula in an enticing way, thanks to super experimental UX.

Mario Kart Live: Home Circuit transforms the Nintendo Switch into a controller for an actual toy race kart. The kart is fitted with a camera, giving the player a first-person view of its perspective as it whizzes around your living room, bedroom, or wherever you have some open floor space to play.

How does the game build your course? You place a few gates that are bundled with the game on the floor. From there, how the exact setup and customization works is unclear (perhaps vision AI is involved?), but Nintendo—alongside its partner developer Velan Studios—demonstrates that one of several tracks, from a simple oval to complicated curves, can be set up to avoid existing couches, coffee tables, and perhaps even sleeping cats.

As you race your kart around the course, all sorts of augmented reality (AR) effects, ranging from glowing boundaries, to power ups, to your racing competitors, will appear on the screen, as if they exist in your actual home. If you run over a virtual item, like a nitro-boosting mushroom, the kart will actually accelerate. If you hit a troublesome banana peel, the car will actually lose some control. Oh, and assuming you have friends with their own games, up to four players can race their karts together in the same space.

With few exceptions, augmented reality has been little more than a gimmick. Snapchat’s zany face filters are still the most successful commercialization of this technology that, not so long ago, the tech world heralded as the next big thing.

Microsoft’s Hololens AR headset is technically impressive, but it’s being marketed as an enterprise tool to businesses (which demonstrates pretty clearly that it’s not ready for the mainstream just yet). The hyped company Magic Leap, with billions in venture capital from investors like Google, has done little more than release a developer version of its headset to mediocre reviews while it hangs on for life. The hardware is simply too expensive, too bulky, but, most of all, too useless to really be worth buying for a vast majority of people. Plus, it’s antisocial by nature to be experiencing a different version of reality than the people around you.

But Nintendo is doing what it does best. It’s figuring out how to transform a gimmick into shared fun—and make it halfway affordable, too. A lot of that comes down to Nintendo just understanding the ergonomics around technology and play. For years, AR demos tasked you to hold up your phone like a little window to peek through, to do something like transform a magazine cover into an animation. These novelties wore thin quickly because they’re more physically awkward than visually amazing.

Nintendo is taking a similar approach here to its predecessors. But instead of utilizing the camera in your phone, it’s built it into the kart. That allows you to play a game like you always do (sitting on your couch), but experience all of these enticing and additive effects of AR. No, Nintendo isn’t aiming as high as Magic Leap, teasing an entire world of digital objects that you can reach out and touch. But Nintendo is competent enough at game design that it’s figured out how to work with what it has to create an AR experience that’s both new and destined to be massively successful.

Mario Kart Live: Home Circuit will be out for $100 on October 16. The last version of Mario Kart sold more than 25 million copies to date. And if Home Circuit is only a fraction as successful, it will still be one of the most profitable demonstrations of AR ever built.

Written by: Mark Wilson, Fast Company
Posted by: Situated Research

The post Nintendo’s newest Mario Kart is the best video game you never knew you wanted to play appeared first on Situated Research.

A number of startups are working on this tech, with an aim to convince VR gear manufacturers such as Oculus Rift and HTC Vive to incorporate the feature in a next generation device. They include SMI, Percept, Eyematic, Fove and Eyefluence, which recently allowed USA Today to demo its eye-tracking tech.

“Eye-tracking is almost guaranteed to be in second-generation VR headsets,” says Will Mason, cofounder of virtual reality media company UploadVR. “It’s an incredibly important piece of the VR puzzle.”

At present, making selections in VR or AR environments typically involve moving the head so that your gaze lands on a clickable icon, and then either pressing a handheld remote or, in the case of Microsoft’s HoloLens or Meta 2, reaching out with your hand to make a selection by interacting with a hologram.

As shown in Eyefluence’s demonstration, all of that is accomplished by simply casting your eyes on a given icon and then activating it with another glance.

“The idea here is that anything you do with your finger on a smartphone you can do with your eyes in VR or AR,” says Eyefluence CEO Jim Marggraff, who cofounded the Milpitas, Calif-based company in 2013 with another entrepreneur, David Stiehr.

“Computers made a big leap when they went from punchcards to a keyboard, and then another from a keyboard to a mouse,” says Marggraff, who invented the kid-focused LeapFrog LeapPad device. “We want to again change the way we interface with data.”

Eye Tech Not Due for Years

As exciting as this may sound, the mainstreaming of eye-tracking technology is still a ways off. Eyefluence execs say that although they are in discussions with a variety of headset makers, their tech isn’t likely to debut until 2017. Other companies remain largely in R&D mode, and Fove has a waitlist for its headset’s Kickstarter campaign.

The challenges for eye-tracking are both technological and financial. Creating hardware that consistently locks onto an infinite variety of eyeballs presents one hurdle, while doing so with gear that is light and consumes little power is another.

And while a number of companies in the space have managed to land funding – Eyefluence has raised $21.6 million in two rounds led by Intel Capital and Motorola Solutions – some tech-centric VCs are sitting on the sidelines while they wait for the technology to mature and for headset makers to make their moves.

“What eye-tracking will do will be powerful, but I’m not sure how valuable it will be from an investment standpoint,” says Kobie Fuller of Accel Partners. “Is there a multi-billion-dollar eye-tracking company out there? I don’t know.”

Among the unknowns: whether the tech will be disseminated through a licensed model or if existing headset companies will develop it on their own.

Still, once deployed eye-tracking has the potential to revolutionize the VR and AR experience, Fuller expects.

Specifically, eye-tracking will “greatly enhance interpersonal connections” in VR, he says, by applying realistic eye movements to avatars.

Facebook founder Mark Zuckerberg, who presciently bought Oculus for $2 billion, is banking on VR taking social interactions to a new level.

“The most exciting thing about eye-tracking is getting rid of that ‘uncanny valley’ (where disbelief sets in) when it comes to interacting through avatars,” says Fuller.

Less Computing Power

There are a few other ways in which successful eye-tracking tech could revolutionize AR and VR beyond just making such worlds easy to navigate without joysticks, remotes or hand gestures.

First, by tracking the eyes, such tech can telegraph to the VR device’s graphics processing unit, or GPU, that it needs to render only the images where the eyes are looking at that moment.

That means less computing power would be needed. Currently, a $700 Oculus headset requires a powerful computer to render its images. Oculus’s developer kit with a suitable computer costs $2,000. “If you can save on rendering power, that could significantly lower the barrier to entry into this market for consumers,” says UploadVR’s Mason.

And second, by not just tracking the eyeball but also potentially analyzing a person’s mood and logging in details about their gaze, AR/VR headsets are in a position to deliver targeted content as well as give third-party observers insights into the wearer’s state of mind and situational awareness.

Police Use

The former use case would appeal to in-VR advertisers, while the latter would come in handy for first responders.

“Police and paramedics are looking for an eyes-up, hands-free paradigm, and eye-tracking can bring that,” says Paul Steinberg, chief technology officer at Motorola Solutions, an investor in Eyefluence.

Steinberg sketches out a scene from what could be the near future.

A police officer on patrol has suddenly unholstered his gun. Via his augmented reality glasses with eye-tracking, colleagues at headquarters are instantly fed information about his stress level through pupil dilation information.

They can then both advise the officer through a radio as well as activate body cameras and other tech that he might have neglected to turn on in his stressed state. What’s more, another officer on the scene can instantly scan through a variety of command center video and data feeds through an AR headset, flipping through the options by simply looking at each one.

“We would have to work with our (first responder) customers to train them how to use this sort of tech of course, but the potential is there,” says Steinberg. “But we’re not months away, we’re more than that.”

Demo Shows Off Ease of Use

An Eyefluence indicates that eye-tracking technology isn’t a half-baked dream.

Navigating between a dozen tiles inside a first-generation Oculus headset proves as easy as shifting your gaze between them. Making selections – the equivalent of clicking on a mouse – is also equally intuitive. At no time does the head need to move, and hands remain at your side.

After about 10 minutes in the demo, it feels antiquated to pop on a VR headset and grab a remote to click through choices selected with head movements.

Marggraff says Eyefluence’s technical challenges included making technology that could respond in low and bright light, accounting for different size pupils and ensuring that power consumption is minimal.

But, he adds, his team remains convinced of the inevitability of its product: “Just like when we started tapping and swiping on our phones, we’re going to eventually need a better interface for AR and VR.”

Written by: Marco della Cava, USA Today (via Presence)
Posted by: Situated Research

The post Next Big Thing for Virtual Reality: Lasers in Your Eyes appeared first on Situated Research.

When Apple introduced the iPad 2 in 2011, it laid out a noble goal for the future of technology.

“Technology alone is not enough,” an Apple ad proclaimed. “Faster, thinner, lighter, those are all good things, but when technology gets out of the way, everything becomes more delightful, even magical. That’s when you leap forward.” 

With the iPad, the notion of technology getting out of the way meant designing a computer so easy to use that the apps took center stage. But the result was in some sense counterproductive; we’ve become so sucked into our phones and tablets that technology is actually getting in the way of the real world.

It’s not going to be like that forever. In talking to leaders from some of the most innovative companies in consumer electronics, it’s clear that the next five years will represent an attempt to bring us back to reality. This may seem paradoxical, but a proliferation of wearable devices, smart-home gizmos, smart cameras, and augmented-reality systems will exist largely to save us from our screens.

Wearables return to the real world

The cynical way to view wearable technology is as yet another intrusion—another set of screens to keep us separated from the physical world. But Yusuf Mehdi, Microsoft’s corporate vice president of devices and studios, doesn’t see it that way. He believes these devices, more than ever, will help technology fade into the background.

Mehdi gives a basic example that applies today: Instead of sounding an alarm, many fitness trackers can wake you with a gentle vibration to avoid disturbing your spouse. It’s a seemingly minor feature, but one that takes the focus off the device itself and onto the people around you. “That’s an interesting thing where people are taking a personal device and saying, ‘Well, the win is for my spouse, not for me,’” Mehdi says.

Moving forward, Mehdi sees devices like Microsoft’s recently announced HoloLens as a way to stay present in the physical world without completely shutting out technology. The still-experimental headset works by projecting 3-D images into a head-mounted visor so they appear to be part of your natural surroundings.

Imagine a scenario in which two coworkers collaborate on a 3-D model projected into their headsets, or someone walking down the street who can see information about surrounding shops and restaurants. Mehdi points out that the original codename for HoloLens was “analog,” for the way it blends with the physical world.

“If you have a device that you’re wearing, and information is being overlaid on top of that, now you’re back in the real world, and you’re interacting, and you’re not missing what’s going on around you, because your head’s there,” Mehdi says.

On a more practical level, these “mixed reality” devices—as Mehdi calls them—will pave the way for more natural input methods like gesture control and eye tracking, which never quite made sense on tablets and laptops. “A lot of things become more human, and the technology kind of goes back out of the way, and we think that’s a big opportunity,” Mehdi says.

The Disappearing Smart Home

Wearable tech will also play a starring role in smart homes—at least if we expect them to offer the kind of breezy convenience that tech companies have been promising.

Dennis Miloseski, Samsung’s U.S. head of design, describes the dream scenario: You pull into your garage and your wearable connects to your Wi-Fi network, which in turn triggers your hallway lights and queues up some music on the living room stereo. “I like to call it the automatic future,” he says.

But he also notes how easily things can go wrong. Maybe your spouse is sleeping on the couch and doesn’t want the lights to come on. That’s why it’ll be so important to have intelligence that figures out what you want, along with some sort of way to confirm your intentions on a wearable device.

“We’re sort of in this archaic age right now, where we’re in this raw form of data readout, meaning, ‘this is how many steps you’ve taken,’ or ‘this is your heart rate,’ or the light is on or off,’” Miloseski says. “I think the next magical innovation is how do we take that data and actually create a form of valuable experience of that data.”

Again, it all comes back to getting the technology out of the way so the user doesn’t have to think about logistics or shuffle through a bunch of apps just to have a fully functional smart home. Miloseski likens it to starting a car or turning on a light switch, in that the complexity is completely hidden from the user.

“I think that we will hit a point in time where, when we think of technology and devices and gadgets and all these things, when they actually impact the social fabric and they become an essential part of how we live our lives, they will become invisible,” Miloseski says.

New Smarts for Dumb Cameras

Photography might be the one area where Apple’s vision of getting technology out of the way seems fully realized. Smartphone cameras are no longer just a quick and dirty image capture tool; they’re the best way to take photos that you can immediately touch up and share with the world.

But as wearable cameras like the GoPro and drone-based ones like DJI’s Phantom enable new kinds of photography, they’ve yet to receive phone-like smarts. Expect that to change in the coming years as capturing and sharing footage from these devices starts to feel as effortless as using the camera in your pocket.

“If I think specifically about us, and the things that we get super-jazzed about, that’s a big piece of it, it’s the whole solving of pain points from when you first capture content to seamlessly sharing it,” says CJ Prober, GoPro’s senior vice president of software and services.

Today, when you capture footage on a GoPro, you’ve got to load it into your computer—itself a time-consuming process—and spend hours looking for highlights and turning them into a YouTube-worthy video. But in the future, a wearable camera might tap into gyroscopes and accelerometers to flag exciting moments, or use machine learning algorithms to sniff out quality footage. It could even tie into other wearable sensors to measure things like jump height or speed, and bring those details straight into the video.

“It’s really important to not think of video and photo capture as an independent thing to do on the device,” Prober says. “It’s really, ‘What do you do with the content when it’s captured?’”

That question will become even more important as new tools like 360-degree cameras become available. Suddenly, you have a lot more footage to work with, which means cameras will need to get smarter at helping you tell the best story.

Drone camera makers like DJI face a slightly different challenge, but with similar overall goals. In the near term, it’ll need to make the actual flight mechanisms smarter so that drones can safely navigate on their own. But once that happens, and the drones themselves become cheaper and more commoditized, it’ll open up all kinds of new smart applications.

“There could be a really rich app economy that’s task-driven instead of product-driven,” says Eric Cheng, DJI’s ‎general manager in San Francisco.

A basic example, he said, would be some kind of live-blogging application that steers a drone as it follows you down the street. Or maybe you’d have an application that can automatically capture and reconstruct a scene in 3-D using cameras. “You can imagine a whole lot of functionality moving into the domain-specific and being a lot smarter,” Cheng says.

Room For The Familiar

None of this is to suggest that the tools we use today are going to vanish, or that you’ll never have occasion to get sucked into your phone, tablet, or computer for a while.

Rick Osterloh, president of Motorola Mobility, now part of China’s Lenovo, says that if anything, the smartphone will remain at the center of all these new smart devices. “It’s resonated so well because it’s actually well-designed, for both utility and a critical feature, which is carryability and pocketability,” he says.

While Osterloh imagines we will see some new technological twists for the smartphone in the form of folding screens and superfast charging, the biggest advances will come from all the different types of data a phone can gather and interpret. Think of it kind of like the Assist feature in Motorola’s current phones, but with more automation and intelligence.

“That is a pretty interesting area writ large, we believe, for the future, where the combination of context and probably sensors will give you a user experience that just helps your phone adapt to what you want,” Osterloh says, “like the magical ‘do what I want’ machine that people in computer science have been trying to develop for decades.”

Likewise, Microsoft’s Mehdi doesn’t see mouse-and-keyboard devices going away, since there’s nothing better for tasks like writing or data entry. “I don’t think this is like tapes and CDs that go away,” he says. “I think it’s more like TV and radio, that don’t actually go away. They just become another part of the media that you consume, and over time they kind of get tuned for the use case.”

The question, then, is how we’re actually going to make room for this expanding roster of wearables, drones, headsets, and smart-home devices. At some point, it might be too much to wrangle, but as Medhi points out, it wasn’t long ago that owning just a cell phone and a computer was hard to fathom. People make room for more devices when there’s sufficient value.

In other words, making all that technology disappear may only work if we own a whole lot more of it.

Written by: Jared Newman, Fast Company (via Presence)
Posted by: Situated Research

The post The Future of Consumer Tech Is About Making You Forget It’s There appeared first on Situated Research.

Mattel is relaunching View-Master, but as a virtual reality and augmented-reality phone toy. And I got to play around with it for a bit…or at least, some of the tech behind it. 

Announced at an event in New York City, the new View-Master is a collaboration between Mattel and Google, whose virtual reality Cardboard app has enabled cheap do-it-yourself accessories to turn any Android phone into a mini-VR viewer. Mattel’s plastic toy, which will debut in October, is like a more durable, plastic version of Google Cardboard, designed entirely for kids…or, maybe, also for grown-up kids like me. And the most brilliant part is it’ll only cost $30.

I used View-Master back when I was a little — who didn’t? It’s a classic 3D stereoscopic picture viewer. Many people had even said Google Cardboard looked a bit like a View-Master. So is isn’t a huge surprise that Mattel has suddenly announced a new View-Master with Google Cardboard VR capabilities added. I’ve always felt that virtual reality reminded me of early stereoscopic toys. And Mattel has keyed onto the same idea.

The toy was only viewable in a mock-up prototype form at Mattel’s event, but the design’s pretty cool: it looks half old-school View-Master, half Oculus Rift. The inner plastic housing extends to hold many types of phones: Mattel says it’s designed to fit the largest existing phones, and will even work with the iPhone 6 Plus and Nexus 6. A capacitive-touch side lever is used to “click” through scenes or into virtual environments, like the magnetized side switch on Google’s Cardboard viewers.

Mattel’s headset is designed with Google and Android in mind, but at launch is intended to work on “nearly all platforms,” which includes iOS. That would mean a dedicated Mattel app which interfaces with the View-Master, but Google’s Cardboard and Cardboard-ready apps — many of which already exist on iOS, like VRSE — will work too.

Mattel is planning to use View-Master not just for VR, but also for AR; little plastic reels that look like the old cardboard ones are really just flat coasters this time around, now with images on top which the View-Master reads and turns into pop-up augmented-reality models on your table, desktop or wherever else you place it. Multiple View-Masters could use one reel to access content if put down on a table, unlike the old pop-in reels. This type of augmented-reality tech has already existed for years in many apps and on some children’s toys like the Nintendo 3DS (with its AR cards) and PlayStation Vita, but mixing it into a VR headset is a novel idea.

I didn’t get to use the actual Mattel prototype, but we tried View-Master’s augmented-reality tech on phones and Google Cardboard viewers. There were three reels to try: a dinosaur one made a little dinosaur pop up on the disc on the table in front of me. When I aimed a dot and clicked on it, I was suddenly surrounded by a prehistoric 360-degree panorama with 3D dinosaurs. Clicking on them brought up facts, too.

Looking at the space disc with Cardboard on brought up a pop-up moon and Earth; clicking on it took me to a panorama of the moon, with pop-up clickable photos of NASA missions. A third, San Francisco-themed, had little mini-models of Alcatraz and the Golden Gate Bridge that turned into VR photo panoramas. To exit any of the virtual panoramas, you look down and click on the side…or, remove the View-Master from your face. The View-Master comes with one reel in its $30 package, and extra reels will cost around $15 each. No, older View-Master reels don’t work in here, but it sounds like Mattel is exploring re-releasing content from some of the back catalog 10,000 older ViewMaster reels.

There’s no strap to keep the View-Master on: this is a hold-to-your-face toy, much like older View-Masters and Google Cardboard. Mattel has promised that the tech has already been vetted by pediatric ophthalmologists, and is meant for children ages 7 and up — in short, bite-sized sessions.

The View-Master may work with other toys, too, like other app-ified toys in the past, but for now it’s really a fancier plastic Google Cardboard viewer, with additional Mattel support. That’s not a bad thing at all: at $30, this is a pretty awesome little stocking-stuffer idea, and a fun phone toy. Just keep in mind that if you give this to your kid, it won’t work without a phone popped into it.

By the time fall rolls around, Mattel may have other toys ready to work with it. Or, there might be many other companies ready to make cheap phone-enabled VR headsets, too.

Written by: Scott Stein, CNET (via Presence)
Posted by: Situated Research

The post Hands-on with Mattel’s new AR, VR View-Master appeared first on Situated Research.

We just finished a heavily scripted, carefully managed, and completely amazing demonstration of Microsoft’s HoloLens technology. Four demos, actually, each designed to show off a different use case for a headset that projects holograms into real space. We played Minecraft on a coffee table. We had somebody chart out how to fix a light switch right on top of the very thing we were fixing.

We walked on Mars.

You’ll notice there aren’t photos here, and that’s because before we were even allowed into the labs where the HoloLens team tests out its user experiences, we had to deposit our cameras and phones into a locker. No recording equipment of any kind was allowed, not even audio. We entered the basement below Microsoft’s visitor center laughing at the absurdity of it all — many reporters needed to get notepads from the company and weren’t carrying pens, either.

But it was all worth it, because HoloLens is probably the most intriguing (and, in many ways, most infuriating) technology we’ve experienced since the Oculus Rift. And there are many parallels with the Rift to be had: both are immersive, but in different ways; both require you to strap a weird thing on your head; both leave you grinning like at absolute idiot at a scene only you can see. And, crucially, both need more work when it comes to thinking through exactly how to control and interact with virtual things.

Minecraft IRL
by Dieter Bohn

By far, the most impressive demo for my money was the Minecraft demo — though Microsoft called it something like “Building Blocks” or some such, presumably so as not to fully commit to releasing a full holograph version ofMinecraft. But before we could enter this virtual world — actually, the virtual entered our world — we had to strap on the development unit for the HoloLens.

It’s a contraption, to be sure. There’s a small, heavy block you hang around your neck which contains all the computing power. It’s comprised of lenses and tiny projectors and motion sensors and speakers (or something that makes sound, anyway), and god knows what else. And then there’s a screen right there in your field of view.

A “screen in your field of view” is the right way to think about HoloLens, too. It’s immersive, but not nearly as immersive as proper virtual reality is. You still see the real world in between the virtual objects; you can see where the magic holograph world ends and your peripheral vision begins.

But before you can apply your jaded “I’ve done VR before” attitude to this situation, you look down at the coffee table and there’s a castle sitting right on the damn thing. It’s not shimmery, but it’s not quite real, either. It’s just sitting there, perfectly flat on the table, reacting in space to your head movements. It’s nearly as lifelike as the actual table, and there’s no lag at all. The castle is there. It’s simply magic.

You definitely have a big stupid grin on your face even though the contraption that’s strapped to it is pressing your eyeglasses into the bridge of your nose in a painful way.

Then it’s demo time. You can’t touch anything, but you can look and point a little circle at objects on it by moving your head around. You learn how a “glance” is just you looking at things and pointing your reticle at them, and an “AirTap” is the equivalent of clicking your mouse. The demo involves digging Minecraft holes and blowing up Minecraft zombies with Minecraft TNT. It’s basically incredible to see these digital things in real space.

You blow up a hole in the table and then you look through it to more digital objects on the floor. You blow up a hole in the wall and tiny bats fly out and you see that behind your very normal wall is a virtual hellscape of lava and rock. You peer into the hole, around the corner, and see that dark realm extend far into space.

And then the demo’s over.

Skype
by Tom Warren

Microsoft’s Skype demo was as equally impressive to me as playing around withMinecraft blocks in a living room. After a two-hour keynote, Microsoft wanted me to fix a light switch. It all started by sitting down and facing some tools and a socket with exposed wiring. A little dazed and confused, I looked up and scanned across the Skype interface which was suddenly appearing in front of me, and picked a face to call. The video call popped into a little window, and my journey to fix a light switch began.

On the other end of the call was a Microsoft engineer. I could see and hear her, but she could only hear me and see exactly what I was seeing in front of me. My eyes, or the headset on my head, was relaying everything over Skype. It was a support call of sorts — here she was to help me fix a light switch. We started by pinning her little window on top of a lamp. I could then look around the room and return to the lamp to see her face. She guided me where to go. It felt strangely natural, and I didn’t need to configure anything or learn gestures other than the same “Air Tap” you use to simulate a mouse click.

While I was being talked through which real world tools we needed for the job, the Microsoft engineer called my attention to the wall with wiring and then started drawing where to position the light switch right on the wall. Thinking about it now it sounds totally surreal, but during the demo I didn’t even think about it — it just felt like I was being guided around with annotations and a helpful friend. We connected the wiring, tested it for an electrical current, and then turned the power back on and switched the light on. It was all fixed, and all by using a crazy combination of a headset, augmented reality, and Skype. It might sound gimmicky, but the applications here are truly impressive. I use YouTube guides to figure out home improvements or to service my car, but this is on another level. Imagine a surgeon performing complex surgery and writing notes in real time and guiding a colleague through it all. Imagine support calls to resolve a problem with your PC. If this works as well as Microsoft’s controlled demo, then this really has the ability to change how we communicate and learn.

Microsoft’s next demo didn’t have us using the HoloLens prototypes directly. Instead, we watched as “Nick” (nobody in Microsoft’s blue-tinted demonstration basement has last names. I asked.) manipulate objects in digital space so he could build a Koala bear or a pickup truck. It was actually quite impressive, as cameras filmed him and screens showed both Alex and the virtual objects he was manipulating in the same space in real time.

The idea was to convince us that HoloLens would unleash a wave of creators who would be able to dream up 3D objects with little to no training. It’s much easier to understand what a thing is in your living room than it is in AutoCad.

But sitting there after our whirlwind of actually experiencing HoloLens, my mind was elsewhere. For example, there are only a few ways to interact with this system so far:

• Glance: you point your head at something.
• AirTap: you make a “Number 1″ sign with your hand, then move your finger down like you’re depressing a lever.
• Voice: you can issue commands, usually to switch what “tool” you’re using.
• Mouse: So actually the neatest thing is that objects you use to interact with computers can be used to interact with holograms.

That seems like enough, but it’s not nearly enough. It’s wildly impressive that these objects really do feel like they’re out there in your living room, but it’s equally depressing to know that you can’t treat them like real objects.

At one point in the demo, Alex needed to put a tire on his pickup. He had to twist his body and head around to get his pointer in just the right spot and get the tire arranged just right to fix on the axle. Then, AirTap! the tire is connected. But how much easier would it be if you could grab the tire in your actual hands?

Our hands are simply more dextrous than our necks. You have finer control over small motions, you can move your hands in so many different ways and vectors, with pressure and nuance and delicacy. Your neck and head, well, not so much.

But then Microsoft gave us 3D printed Koalas with a USB drive inside them, which was nice. And if this HoloLens thing takes off, you will be able to design your own and it will be way easier than learning current 3D design software. But not as easy as it would be if you just imagined building with holograms.

Walking on Mars
By Tom Warren

Microsoft has teamed up with NASA to let scientists explore what Curiosity sees on Mars. Instead of panoramic imagery on a computer screen, Microsoft’s demo lit up a room and turned it into Mars. I walked around the rocky terrain, bumped into the Curiosity rover, and generally just checked out a planet I will never visit in my lifetime. It’s a totally new perspective that felt like I was immersed in touring Mars, but not necessarily there. The field of view felt a little too limited to truly immerse myself and trick my brain into thinking I was really on another planet, but what impressed me most is what Microsoft has built into this experience.

I held a call with a NASA engineer and he talked me through the terrain. I squatted to look more closely at rocks, took snapshots of various rock formations, and even planted flags for points of interest. My jaw dropped when I ventured over to a PC in the room and started to experiment with the mouse. I pulled the mouse pointer off the screen and suddenly it was on the floor next to me, allowing me to set markers in the virtual environment. It’s everything I’ve seen in demonstrations from Microsoft Research before, but here it was on my head and working.

The collaboration part was the key here, allowing me to interact with this data in a unique way, but also alongside the NASA engineer who could drop flags on the Mars terrain and guide me to look at certain sections. While this isn’t traditional productivity with a mouse and keyboard, it’s certainly something new and intriguing. I could see this type of scenario working for big teams that need to communicate across time zones and on big sets of complex data.

Overall, HoloLens is Microsoft at its most ambitious. It’s a big bet on the future of computing, the future of Windows, and ultimately the future of Microsoft itself. While the company is struggling at mobile, it wants to catch the next wave of computing and lead. Is HoloLens the next wave? Developers and consumers will be the ultimate test of that, but if anything HoloLens is an incredibly brave and impressive project from Microsoft. It’s true innovation, which is something Microsoft has lacked during its obsession with protecting Windows. It’s also another example of an experience that takes the complex technology out of the way, leaving you to experience what really matters.

Written by: Dieter Bohn and Tom Warren, The Verge (via Presence)
Posted by: Situated Research

The post Welcome to the Age of Holographs appeared first on Situated Research.

This Place, an agency that specializes in creating user interfaces and experiences for programs used in the medical industry, developed a software called MindRDR that allows Google Glass to connect with the Neurosky MindWave Mobile EEG biosensor, a head-mounted device that can detect a person’s brain waves. 

EEG stands for electroencephalography, which is the measurement and recording of electrical activity in the brain. EEG biosensors have been around for decades, but until recently they were very expensive. Neurosky is a Silicon Valley company that sells EEG biosensors, some for as little as $79.99 from Amazon.com.

The system works by pairing the EEG biosensor with Google’s $1,500 Glass device using Bluetooth. Once the connection has been made, the user fires up MindRDR, which takes what the EEG biosensor detects and converts it into commands that Glass can process.

After turning on the app, users will see a camera interface on the screen of their Google Glass. They can then pick a subject, aim their head in its direction, and concentrate on it while Glass displays a meter showing the level of their brain waves. The more intently a user focuses, the higher the meter climbs until it reaches the top, triggering Glass’ camera. By repeating the process, users can direct MindRDR to upload the photo to one of their social networks.

For now MindRDR can only be used to snap pictures, but This Place Chief Executive Dusan Hamlin said he hoped the agency would continue developing the software so that it could eventually help users overcome mobility limitations. Specifically, Hamlin said he would like MindRDR to help people who suffer from locked-in syndrome, in which a patient has lost motor control but remains aware and alert, as well as quadriplegia.

“The ability to be able to use their mind to make outputs to a device could be a huge thing for them,” Hamlin told the Los Angeles Times in a Skype interview.

But the possibilities for MindRDR extend beyond the medical field. Hamlin said he sees MindRDR as the launching point for a world where people can interact with their digital devices by simply thinking about what they want. To that end, This Place has uploaded the code for its software onto GitHub, a popular website used by developers to share code they create with others for free.

“What we’ve done is just scratch the surface, and we hope that we’ve inspired people to build on what we’ve started,” Hamlin said.

Written by: Salvador Rodriguez, the Los Angeles Times (via Presence); more information is available from This Place and an article in BBC News
Posted by: Situated Research

The post London Firm Creates Mind-Controlled Commands for Google Glass appeared first on Situated Research.

Last year’s 5 in 5 list also focused on the rise of cognition in computing and how the five senses humans use to gain information about and manipulate the physical world are being emulated by computing systems like IBM’s own Watson artificial intelligence framework.

For this year’s edition, IBM got a little more specific about the ways that such advances in machine learning will affect us, touching more on data analytics and offering up the following predictions:

The classroom will learn you: Kerrie Holley of IBM described this as a concept “built on a lot of the technologies you see with how the Khan Academy works, cloud-based computing, and the like.” In the years to come, new learning technologies will use advanced analytics of “longitudinal student records” to help teachers better assess what individual students need, which ones are at risk, and how to help them in their education, he said.

Buying local will beat online. Less about a specific tech advance, this prediction is based on the idea that the “tables will turn” in terms of access to the kind of technology, cloud services, and analytics that can help “mom and pop” businesses compete more readily with big national and global retailers, Holley said. “Technology costs are dropping and as they do, proximity will allow local retailers to create experiences the big retailers are not able to do online.”

Doctors will use your DNA to keep you well. IBM presented this prediction as one involving more advanced computational work than some of the others in its 5-in-5 list. “Cognitive-based systems like Watson, along with breakthroughs in genomic research, will enable doctors to be better able to diagnose cancer and offer better treatments,” Holley said.

The city will help you live in it. In just a few decades, as many as seven out of 10 people around the world will live in cities, according to some projections. We’re already seeing more computational resources being dedicated to helping those city dwellers manage their urban lives and that will only accelerate, according to IBM.

A digital guardian will protect you online. Holley explained this prediction as an expansion on financial fraud protection services offered by banks and credit card companies, only much more personally tailored to individuals to safeguard their entire digital lives.

“This year’s IBM 5 in 5 explores the idea that everything will learn—driven by a new era of cognitive systems where machines will learn, reason and engage with us in a more natural and personalized way. These innovations are beginning to emerge enabled by cloud computing, big data analytics, and learning technologies all coming together,” the research team behind the company’s annual list of predictions said in a statement.

“Over time these computers will get smarter and more customized through interactions with data, devices, and people, helping us take on what may have been seen as unsolvable problems by using all the information that surrounds us and bringing the right insight or suggestion to our fingertips right when it’s most needed. A new era in computing will lead to breakthroughs that will amplify human abilities, assist us in making good choices, look out for us, and help us navigate our world in powerful new ways.”

Written by: Damon Poeter, PC Mag
Posted by: Situated Research

The post IBM Forecasts Major Advances in Cognitive Computing appeared first on Situated Research.

“It’s a privilege to be a part of this project as we explore how this exciting new technology might be incorporated into the everyday care of our patients,” said Dr.Christopher Kaeding, the physician who performed the surgery and director of sports medicine at Ohio State.  “To be honest, once we got into the surgery, I often forgot the device was there. It just seemed very intuitive and fit seamlessly.”

Google Glass has a frame similar to traditional glasses, but instead of lenses, there is a small glass block that sits above the right eye.  On that glass is a computer screen that, with a simple voice command, allows users to pull up information as they would on any other computer.  Attached to the front of the device is a camera that offers a point-of-view image and the ability to take both photos and videos while the device is worn.

During this procedure at the medical center’s University East facility, Kaeding wore the device as he performed ACL surgery on Paula Kobalka, 47, from Westerville, Ohio, who hurt her knee playing softball.  As he performed her operation at a facility on the east side of Columbus, Google Glass showed his vantage point via the internet to audiences miles away.

Across town, one of Kaeding’s Ohio State colleagues, Dr. Robert Magnussen, watched the surgery his office, while on the main campus, several students at The Ohio State University College of Medicine watched on their laptops.

“To have the opportunity to be a medical student and share in this technology is really exciting,” said Ryan Blackwell, a second-year medical student who watched the surgery remotely.   “This could have huge implications, not only from the medical education perspective, but because a doctor can use this technology remotely, it could spread patient care all over the world in places that we don’t have it already.”

“As an academic medical center, we’re very excited about the opportunities this device could provide for education,” said Dr. Clay Marsh, chief innovation officer at The Ohio State University Wexner Medical Center. “But beyond, that, it could be a game-changer for the doctor during the surgery itself.”

Experts have theorized that during surgery doctors could use voice commands to instantly call up x-ray or MRI images of their patient, pathology reports or reference materials.  They could collaborate live and face-to-face with colleagues via the internet, anywhere in the world.

“It puts you right there, real time,” said Marsh, who is also the executive director of the Center for Personalized Health Care at Ohio State. “Not only might you be able to call up any kind of information you need or to get the help you need, but it’s the ability to do it immediately that’s so exciting,” he said.  “Now, we just have to start using it. Like many technologies, it needs to be evaluated in different situations to find out where the greatest value is and how it can impact the lives of our patients in a positive way.”

Only 1,000 people in the United States have been chosen to test Google Glass as part of Google’s Explorer Program. Dr. Ismail Nabeel, an assistant professor of general internal medicine at Ohio State applied and was chosen. He then partnered with Kaeding to perform this groundbreaking surgery and to help test technology that could change the way we see medicine in the future.

Broadcast quality video and high-definition photos available for download: http://bit.ly/16jXc6c

Written by: The Ohio State University (via Presence); for details about the first international Google Glass surgery, in June 2013, see Clinica Cemtro; for a report about early reactions from those testing Glass see NPR

Image: Dr. Christopher Kaeding, an orthopedic surgeon at The Ohio State University Wexner Medical Center is shown wearing Google Glass

Posted by: Situated Research

The post Ohio State Doctor Shows Promise of Google Glass in Live Surgery appeared first on Situated Research.

SANTA CLARA, Calif. — Reality isn’t what is used to be. With increasingly powerful technologies, the human universe is being reimagined way beyond Google Glass’ photo-tapping and info cards floating in space above your eye. The future is fashionable eyewear, contact lenses or even bionic eyes with immersive 3D displays, conjuring up a digital layer to “augment” reality, enabling entire new classes of applications and user experiences. 

Like most technologies that eventually reach a mass market, augmented reality, or AR, has been gestating in university labs, as well as small companies focused on gaming and vertical applications, for nearly half a century. Emerging products likeGoogle Glass and Oculus Rift’s 3D virtual reality headset for immersive gaming are drawing attention to what could now be termed the “wearable revolution,” but they barely scratch the surface of what’s to come.

The wearable revolution can be traced back to Ivan Sutherland, a ground-breaking computer scientist at the University of Utah who in 1965 first described a head-mounted display with half-silvered mirrors that let the wearer see a virtual world superimposed on the real world. In 1968 he was able to demonstrate the concept, which was dubbed “The Sword of Damocles.”

His work was followed up and advanced decades later by researchers including the University of Toronto’s Steve Mann and Columbia University’s Steven Feiner. In the second decade of the 21st century, the technology is finally catching up with their concepts.

The necessary apparatus of cameras, computers, sensors and connectivity is coming down in cost and size and increasing in speed, accuracy and resolution to point that wearable computers will be viewed as a cool accessory, mediating our interactions with the analog and digital worlds.

Augmented Reality past and future

“You need to have technology that is sufficiently comfortable and usable, and a set of potential adopters who would be comfortable wearing the technology,” said Feiner at the gathering of the fledgling AR industry at the Augmented Reality Expo here Wednesday. “It would be like moving from big headphones to earbuds. When they are very small and comfortable, you don’t feel weird, but cool.” He added that glasses with a “sexy lump of bump” with electronics and display could also be cool to the early adopters, especially the younger generation that has grown up digital. However, he didn’t have any prediction for when wearable computer would reach a mass market.

In the last decade, AR has been primarily focused on immersive gaming that teleports users to another world and on vertical applications, such as tethered, interactive 3D training simulations.

But now augmented reality is about to break out into free space. “AR will be the interface for the Internet of things,” said Greg Kipper, author of “Augmented Reality: An Emerging Technologies Guide to AR.”

“It is a transition time, like from the command line to graphical user interface,” he said. “Imagine trying to do PhotoShop in a command-line interface. Augmented reality will bring to the world things beyond the graphical user interface. With sensors, computational power, storage and bandwidth, we’ll see the world in a new way and make it very personal.”

Will Wright, creator of the popular The Sims family games, likened AR to having super-sensory abilities, like flipping a switch to see what is underground, beneath your feet. “It’s not about bookmarks or restaurant reviews…it’s something that maps to my intuition.” He hoped that instead of augmenting reality, the technology could “decimate” reality, filtering out even more information than the brain already does to engage reality with less cacophony.

Steve Mann, who is rarely seen without one of his wearable computing rigs and is considered the father of AR, views the wearable revolution as a benefit to society. Quality of life can be improved with overlays of information, adding and subtracting it to facilitate improved “eyesight,” he said. “The first purpose is to help people see better,” he said during his keynote at the expo.

Just as the smartphone is compressing a lot of the function from antecedent computing devices into a single product, wearable computing will eventually make the handheld smartphone irrelevant.

“The value proposition of digital eyewear is having all devices in one, with a camera for each eye representing full body 3D, and the ability to interact with an infinite screen. We are architecting the future of interaction,” said Meron Gribetz of Meta, a Ycombinator startup working on a new operating system and hardware interface for augmented reality computing.

“There is no other future of computing other than this technology, which can display information from the real world and control objects with your fingers at low latency and high dexterity. It’s the keyboard and mouse of the future,” he claimed.

Atheer, a Mountain View, Calif.-based AR startup, is developing a platform that will work with existing mobile operating systems, such as Google’s Android. “We are the first mobile 3D platform delivering the human interface. We are taking the touch experience on smart devices, getting the Internet out of these monitors and putting it everywhere in physical world around you,” said CEO Sulieman Itani. “In 3D, you can paint in the physical world. For example, you could leave a note to a friend in the air at restaurant, and when the friend walks into the restaurant, only they can see it.”

The company plans to seed its technology to developers this year and have its technology embedded in stylish, lightweight glasses with cameras next year.

The transition to touch and gesture interfaces doesn’t mean that the old modes of human-computer interaction go away. Just as TV didn’t replace radio, augmented reality won’t obliterate previous interfaces. The keyboard might still be the best interface for writing a book. Nor is waving your hands in front of your face all day a good interface.

“Holding hands out in front of self as primary interface is the ‘gorilla arm’ effect,” said Noah Zerkin, who is developing a full-body inertial motion-capture system for head-mounted displays. “You get tired. We need to have alternative interfaces. If not thought-based, it needs to be subtle gestures that don’t require that you to wave hands around in front of your face.”

3Gear Systems is working on technology that allows 3D cameras mounted above a keyboard, like a lamp, to detect smaller gestures just above the keyboard, such as pinching to rotate an object on a screen, and can use input from all 10 fingers with millimeter-level accuracy.

Some companies are taking less radical approaches, focusing on inserting a layer of digital information into scenes via smartphones. Par Works, for example, image recognition technology makes it possible overlays digital imagery on real world data, such as photos and videos, with precision. A person looking for an apartment takes a picture of a building with a smartphone and the app overlays information on the image, or a shopper will see coupons or other information for various products on a shelf in a drug store.

Brands are adopting AR technology to increase performance of ads and sales. Several companies provide ways to turn a print ad into an interactive experience just by pointing the camera at the paper or an object with a marker. Blippar, for instance, recognizes images by pointing a phone camera at ads or object with its mark and inserts virtual layers of content.

The future of augmented reality

And where is all this heading over the next few years? It’s beginning to look like a real business, just as mobile did nearly a decade ago. Mobile analyst Tomi Ahonen expects AR to be adopted by a billion users by 2020. Intel is betting that AR will be big. The chip maker is investing $100 million over the next 2 to 3 years to fund companies developing “perceptual computing” software and apps, focusing on next-generation, natural user interfaces such as touch, gesture, voice, emotion sensing, biometrics, and image recognition.

Apple isn’t in the AR game yet, but the company has been awarded a U.S. patent, “Synchronized, interactive augmented reality displays for multifunction devices,” for overlaying video on live video feeds.

Eyewear will evolve over the next year with comfortable stylish glasses with powerful embedded technology. They will range from Google Glass-style glance-at displays that also replace the phone to stereoscopic 3D-viewing wearables for everyday use.

“You’ll get 20/20, perfectly augmented vision by 2020, with movie-quality special effects blended seamlessly into the world around you,” said Dave Lorenzini, founder of AugmentedRealityCompany.com and former director at Keyhole.com, now known as Google Earth. “The effects will look so real, you’ll have to lift your display to see what’s really there. There’s more of the world than meets the eye, and that’s what’s coming.”

He cautioned that the growth of the AR industry could be slowed by a lack of standards to connect disparate players and their formats for bringing a 3D digital layer to life. “The AR industry has to get together to power the hallucination of what’s do come,” Lorenzini said. He added that a key turning point will be the availability of the WYSIWYG (What You See Is What You Get) real-world markup tools needed to bring this digital layer to life.

When the AR industry does take off, Lorenzini envisions a trillion dollar market for animated content, services and special effects layered into the real world. “Imagine people tagging friends with visual effects like a 3D halo and wings, or paying for a face recognition service to scan and add a floating name tag over the head of everyone in a room,” he said. “AR will grow from specific vertical uses to mass market appeal, driven by young, early adopters.

“Anyone reviewing devices like Google Glass needs to take it to their kids’ school before they pass judgement,” Lorenzini added. “This is not a device from our time, it’s from theirs. They love it, use it effortlessly, and are totally unfazed by ad targeting or privacy concerns. It will be be a natural part of who they are, how they learn, connect and play.”

Eventually, wearable technology will become more integrated with the human body. With advances in miniaturization and nanotechnology glasswear will be replaced with contact lens or even bionic eyes that record everything, make phone calls and allow you to use parts of your body, or even your thoughts, to navigate the world.

“Contact lenses are difficult now but the bionic eye will become commonplace and AR will just be a feature,” Kipper said. “Some may choose to have eyes in back of their heads, and some won’t. Some will want to be cyborgs. We will always use tools as advanced as they can be to help ourselves.”

Brian Mullins, CEO of Daqri, an augmented reality developer of custom solutions, went even further in melding humans and technology. “Thinking is the future of AR,” he said. Mullins talked about measuring “thought intensity” with EEG machines and focusing the mind to manipulate objects during a panel discussion at the Augmented Reality Expo.

Of course, the technical challenges are accompanied by issues of social etiquette and privacy. Smartphones are now a well-accepted part of daily life in most countries, but issues around data ownership and access to the data abound. The subtlety and potentially always-on capacity of wearable technologies will create more privacy concerns and challenges to acceptance.

Feiner acknowledged that it’s “scary” in terms of the information available, especially when billions of people with cameras and microphones can capture anything in public. “There are no laws against it,” he noted.

He gave Google some compliments for not overloading Glass with features. “It not suffering from doing too much too soon,” he said. Whether Google Glass is the tip of the spear for the mass adoption of far more powerful AR is uncertain, but it is doing a good job of surfacing the issues around the introduction of a disruptive, new way of computing.

Nicola Liberati, a Ph.D. student in philosophy at the University of Pisa studying the intersection of humans and technology, suggested another line of thinking about AR in his presentation at the expo. “We should not focus our attention only on what we can do with the such technology, but even on what we become by using it.”

So, when you are strolling down the street wearing the latest digital eyewear from Google, Apple or some as yet unformed or now early-stage company, with your continuous partial attention on the 3D holographic screen feeding you all kinds of personalized information about the environment around you, zeroing in on the people and places in your field of view or piped in remotely from around the real and virtual worlds, and spaces in between, think about what we have become.

It all depends on your perspective.

Written by: by Dan Farber, CNET (via Presence)
Posted by: Situated Research

The post Beyond Google Glass: The Evolution of Augmented Reality appeared first on Situated Research.

Google’s mission is to organize the world’s information and make it universally accessible and usable. Note carefully that Google says nothing about the Internet in that statement. 

In the last few eye-blinks of human history, we’ve created virtual worlds: cyberspace, virtual reality, the World Wide Web … places that exist in our devices, on our computers, in our servers, on the internet, and in our heads. But there’s also a space in which we live and walk and eat and breathe. Realspace. Meatspace. IRL. The real world, so we say, that we can touch and taste and smell.

Google’s trying to bring those worlds together, partly through the work of Niantic Labs.

Augmented reality is nothing new, of course, with marketing-focused companies like Layar building connections between physical and virtual reality and Ikea’s most-downloaded branded app of 2012 doing similar things. Other startups have explored AR capabilities as well, such as Caterina Fake’s Findery, which invites people to leave geo-tied notes that others can discover and read.

But when a company with the resources of a Google tackles the problem, and has a tool in Google Glass that seems destined for significant developer (and probably user) penetration that can actually create interconnections between the real and the virtual perhaps more efficiently than any other previous product, you’ve got something interesting. And potentially huge.

So a couple of weeks ago, I chatted with the man who’s leading that effort.

John Hanke: the missionary of mapping

John Hanke is vice president of product for Niantic Labs, the year-old Google-but-not-Google division of just a few dozen engineers that brought us Field Trip, the app to explore the world around us with a virtual docent. And, of course, the virtual/real game Ingress.

Before Niantic, Hanke ran Google Maps, Google Earth, and other geo areas, and before Google, he was the cofounder and CEO of Keyhole, the innovative geo-mapping and visualization company. Google bought Keyhole in 2004, which brought Hanke in the search engine’s fold to lead the its maps, earth, street view, and local divisions.

Now, he told me, rather than let him leave to scratch his entrepreneurial itch yet again and do another startup, Google gave him a semi-autonomous group to, as his LinkedIn profile suggests, experiment at the “intersection of mobility, real world, and the Internet.”

“We set up Niantic as a group that could explore new types of mobile apps with ubiquitous always-on features,” Hanke said. “And we’re set up to act like a start-up.”

Virtual + physical = field trip

Field Trip was one of Niantic’s first creations, and while on the surface it’s an app that helps you find cool stuff, ultimately it’s a tool to merge metadata and data and then present them together. While you’re in the physical world, Field Trip pulls data about that experience from digital sources, feeding you that information, and changing — deepening, enriching — your experience of place. Layering with history, perhaps, or science, or culture.

Because, after all, one rock is very much like another rock, but if this is the precise rock where Geronimo attacked Mexican soldiers armed with only a knife and his courage, that changes our experience of this particular place. And the merging/melding/layering of virtual and physical makes it more real, in a sense — hyperreal.

Enabling that, of course, requires extensive virtual enhancement of the what-you-see-is-what-you-get world.

“One of the things that we’re trying to evangelize is the concept of geo-tagging everything,” Hanke told me. “I would have expected eight years ago that it would be ubiquitous now, but it’s still not. But I think we’ll get there.”

Geotagging everything digital is a key intersection point between virtual and real. If this blog post is written here, and not there, that adds flavor and nuance to the information. And if a particular historical fact is geotagged to a specific mapped location, that adds depth and dimension to our experience of that place.

“We’re applying some of the same techniques we currently use in standard web search, and the same kind of discipline, to pull really interesting, really good places up from everything else,” Hanke says. “The model is that you’re walking through an unfamiliar neighborhood, but with a friend who is telling you the best things around you. You enjoy it just like before, but you’re a little more informed.”

AR + MMO + IRL

Depth and dimension are definitely core components of Ingress, another Niantic Labs app/experiment/game. Ingress is a — take a deep breath — augmented reality massively multiplayer online video game.

The real world is real, but it’s fought over virtually by two shadowy groups: the Enlightened and the Resistance. Niantic has filled the Earth with virtual portals, usually coincident with actual physical landmarks or monuments, that players need to capture in order to gain territory. Capture territory with large numbers of people (aka “mind units”) and your faction gets more powerful.

Clearly, the massive integration of Google mapping technology with a sophisticated gaming engine is required. And the result is another intersection between the real and the virtual.

“Ingress is a massively multiplayer online game designed for mobile, with real location-based connections,” Hanke told me.

You play with everyone in your faction, and you might meet up with other players in real life, or you may just know them virtually as team members in another area. Along the way, Google learns an awful lot about how you use your mobile devices, about mapping physical locations, and about overlaying cyberspace on meatspace.

All of that knowledge is going to come in very handy with Google Glass.

Endgame: Google Glass?

Hanke is cautious when speaking about Google Glass, as is the PR handler who is copiloting our conversation. Even already public information is a question mark as we chat: Google is definitely being Apple-like in the control and distribution of Glass and its future.

But something tantalizing tidbits do come out.

“We definitely kinda had Google Glass in mind when we started work on apps at Niantic,” Hanke says. “We need mobile devices that are less intrusive than the phone is.”

And we need devices with different input/output modalities, he says. After all, it’s not easy to play Ingress running around holding an expensive and fragile device in front of you like a window ripped from its frame. And yet you need that portal from the physical to the virtual. For instance, while Field Trip is great to open the doors on human context for the world around us, it threatens to detract from our experience of the world by redirecting our eyes from the ultimate big screen of reality to the small screen of our mobile device.

Google Glass, on the other hand, sits unobtrusively on our foreheads, leaving our hands free and providing data as an overlay on top of the physical world rather than an alternative to the physical world. That model of layering, mixing, and intersecting is top-of-mind for Hanke.

“It just can’t be the case that people are walking around heads down tapping on a screen,” he says. “That just can’t be the future of the human race.”

Cyborg me now

Which, of course, is exactly what’s at issue: the future of the human race. Or, at least how we ingest, consume, and reconstitute digital data. And analog data. And meld the two into one harmonious whole of knowing.

That’s perhaps a little metaphysical for a small division of Google that focuses on maps and games and apps.

But the web has rewired our brains in a decade or so of virtually ubiquitous Internet access, and the smartphone has rewired our behavior in five years, taking us from creatures who look up to to see others to beings that look down at any opportunity to see small bits of plastic and glass and metal in our hands.

So is it really too much to expect from a transformation that brings us from clear divisions between what is real and what is virtual to an elegant blend of the two?

“This is not psychosis or some cognitive break, but an actual takeover of the mind,” Google’s introductory video for the Ingress game says ominously.

Art imitates life, I suppose, and life, in turn, imitates art.

Written by: John Koetsier, VentureBeat (via Presence)
Posted by: Situated Research

The post How Google is Melding Our Real and Virtual Worlds with Games, Apps … and Glass appeared first on Situated Research.