'Making invisible things visible': With iPhone 8 and iPhone X, Apple kicks off the augmented reality wars

​If you want to know where technology is heading, imagine a tape measure. You can drag it across the floor, or run it along the frame of a doorway. Except this tape measure is virtual — it exists solely on the screen of an iPhone — and it's about as accurate as the real thing.

A handful of developers are working on apps just like this one, and you'll be able to try them in the coming days. It may sound mundane, but that's precisely what makes it great. It's a sign that augmented reality — the layering of digital information onto a person's view of the physical world — is getting both good enough and accessible enough to be useful day-to-day.

It's also why, over the next few months you'll see two tech giants — Apple and Google — fighting to finally propel AR into the mainstream.

• Apple unveils iPhone 8 — and more — today
• Google wants to bring augmented reality to '100 million' devices this winter
• Behind Apple's magical future, a vision built on borrowed ideas

On Tuesday, at an event in its new Cupertino, Calif., office, Apple explained how it plans to come out on top.

The company's new iPhone 8 and iPhone 8 Plus phones include a new camera and image processor that it says has been specifically calibrated for augmented reality experiences. It's paired with new motion sensors — a gyroscope and an accelerometer — that the company says will make tracking the phone's position and movement space even more accurate.

On the company's new high-end iPhone X, Apple says a front-facing depth sensor designed to let users unlock their phones with a glance will also improve the accuracy of augmented reality face-tracking — like the kind that Snapchat uses for its filters.

And its new mobile operating system, iOS 11, will be released to the public on September 19 — and expose a massive mainstream audience of new and recent iPhone owners, hundreds of millions strong, to a varied new collection of augmented reality apps.

"Apple has always believed that technology infused wth humanity could improve people's lives and change the world," said CEO Tim Cook on stage inside Apple Park's Steve Jobs Theatre.

Apple is trying to show that augmented reality on a smartphone can do a lot more than what we've seen so far from Snapchat or Pokemon Go. But it's too early to say whether these experiences will actually lead to a whole new way of interacting with the world and with our devices, as the tech industry hopes.

"It depends on one thing and — to be honest — one thing only: it depends on the quality of the user experiences," says Brian Blau, an analyst with tech consulting firm Gartner who researches augmented and virtual reality technology. "There's a lot higher bar this time around."

Sensor fusion for the masses

During the keynote, Apple offered a glimpse at what it believes those experiences will look like. One was called The Machines, a multiplayer real-time strategy game where the play area is projected onto a floor or table, and players have to move to different vantage points around the board in order to play.

But more impressive was a brief demo from Major League Baseball's Advance Media Team. "You can hold up your iPhone and see real-time player information and stats on top of the game you're watching," said Phil Schiller, Apple's senior vice-president of worldwide marketing.

Over the past year, Apple CEO Tim Cook has spoken at length about how he believes AR experiences like these will be "big and profound" and will make the iPhone "even more essential than it currently is." In one interview, he even compared AR's potential reach to that of the smartphone itself.

"We don't have to think the iPhone is about a certain demographic or country or vertical market — it's for everyone," he told the U.K. Independent. "I think AR is that big, it's huge."

One way to experience augmented reality is with glasses or helmets that put digital information right in front of the wearer's eyes. Microsoft has been working on a head-mounted display called Hololens, and Google has Glass. But that technology is still in its infancy — and until it gets better, cheaper and more accessible, Apple and Google have decided to focus on bringing a slightly less futuristic experience to the devices that hundreds of millions of people already have.

The approach that Apple and Google are using — something called sensor fusing, where data from the phone's accelerometer and compass is combined with imagery from the camera — has been around for a while. But here's what's new: First, both companies released toolkits, or APIs, this past summer to make it easier for developers to build AR apps. Apple's is called ARKit, and Google's is AR Core. You no longer have to be a computer vision expert to make something cutting-edge and cool.

Second, they're rolling those experiences out to recent iPhone and Android phones, no special hardware required. The processors in phones have gotten powerful enough, the sensors accurate enough and the mapping algorithms smart enough to give smartphones a remarkably good understanding of a user's relative position and movement through a physical space.

"It's one thing to have [augmented reality] in academic circles, in papers and demo videos," Zach Lieberman says. "But to then have an API behind it, and to have it in your hands in a device that doesn't cost that much and is quite fast and easy to use is quite remarkable."

And Apple is in the unique position of being able to control the whole package, both hardware and software, which some think will give it an edge.

'The potential to delight'

Lieberman, a developer, researcher and artist, has worked on a handful of augmented reality projects in recent years. He likes the medium for its ability to "make invisible things visible." On Twitter, Lieberman has been posting short videos of some of the ARKit experiments he's made

In one, he suspends an audio waveform of his voice in the air. To play the recording back, he has to physically retrace the waveform's path with his phone. He says it only took him a few hours to build. In another, he demos a camera app where photos stay fixed in place to the location from which they were taken, like a photo album suspended mid-air.

His eight-year-old daughter "gasped" the first time she saw that, Lieberman says. "I feel like this technology has the potential to bring a lot of delight."

Many of the experiences that have been demoed over the past few months have been designed to do exactly that. One developer is working on a menu app that lets you preview startlingly realistic-looking food on a plate. There are drawing apps galore, and interactive stories. Many are still exploring the possibilities of projecting 3D objects into physical spaces — from Tesla vehicles to basketball players and the Very Hungry Caterpillar. Another person recreated A-Ha's classic 1984 music video for Take on Me.

"People don't know what AR is yet, and they have ideas for it that it definitely can't do yet," Lieberman said. "But I have no doubt that people will figure this out."

More functional than a dancing hotdog

And of course, there are the measuring apps — so far, the best demonstration of what this new technology can do.

Rinat Khanov, one of the developers behind the Safari ad blocker 1blocker, is working on one such measuring app. He calls it MeasureKit — and for a utility, it's a delight to use. You can point your phone at the edge of a table, drag a thin blue line to the other end, and watch the inches tick upwards along the way. It's functional in a way a dancing hotdog isn't.

For jobs that don't require millimetre precision, the result is pretty close to what a real measuring tape shows. But it's not without its quirks. For one, the line tends to drift from its starting point the more you move around.

Laan Labs is another development shop fighting for the crown of most accurate measurement app (theirs is called AR Measure). The company is three people — Jason Laan, his brother and a childhood friend — who have been working on AR projects for seven years. Face Swap Live is perhaps their best known app, before Snapchat copied the feature. But Laan points to an earlier app called AR Soccer, developed way back in 2010, to demonstrate how far smartphone AR has come.

"It didn't really have a good understanding of the 3D world," Laan said. "Now with ARKit, you're able to get a lot of information about real-world tracking just right out of the box. You don't have to write all that stuff yourself."

That used to be the hard part. Now, in a world that's about to become awash in AR apps, having the tech alone won't be enough. Developers will have to figure out what it is that people will actually want to use.

There will always be games and entertainment — like the interactive narrative from Peter Jackon's studio that Apple demoed back in June — but Laan thinks the arrival of more practical apps is where things get interesting. Measuring apps are merely the start.

Tom Mainelli, an analyst at IDC who researches both AR and VR, believes the nascent market for AR apps will be a lot like the early days of the App Store — lots of gimmicks for the first few months, until developers figure out what works.

"I think it absolutely will cause people to want to upgrade their phones," Mainelli says. "But I think there will be a fair amount of skepticism until people experience it."