What Augmented Reality Can Bring to the Industry and Why it Will Take Time

Blog: Apriso Blog

With Pokémon Go, augmented reality for gaming may have found its killer app. Gamers must locate, capture and battle Pokémon creatures that appear in their real-life environment.

But how long will it take for augmented reality applications to become standard in manufacturing? And, is Pokémon Go really an augmented reality app?

Virtual Reality vs Augmented Reality

You have probably heard of virtual reality (VR) and augmented reality (AR). But what is the difference?

Virtual reality creates a virtual world that users can interact with. The experience is immersive, thanks to the realistic simulation of vision and hearing. The applications of virtual reality are mainly found in gaming, entertainment and simulation. Virtual reality devices do not allow you to see the external world.

Augmented reality expands the real world with virtual artifacts, showing how you look with these beautiful earrings or highlighting the oil drain plug in your car engine. Augmented reality devices are see-through devices, with the potential to add virtual objects.

VR devices are immersive

AR devices are see-through displays

Key capabilities of AR devices are related to optics (resolution, field of view), environmental awareness (tracking of objects, spatial mapping) and user interactions (voice recognition, gesture input).

Ideally, AR applications layer information on top of real objects in a spatially intelligent way, which requires advanced technologies such as computer vision, depth sensing and object recognition.

For its part, the Pokémon Go app only relies on smart phone geo-location capabilities and camera to display flat Pokémons on top of real-life surroundings, sometimes with unrealistic results. This is why some consider Pokémon Go as a location-based game, and not as a true augmented reality app.

Augmented Reality Applications in Manufacturing

While VR is primarily for the entertainment market, business applications will drive a large part of the AR market, including industrial applications. The following scenarios within manufacturing are ideal and it will take time before they become common practice in the industry.

Assembly. The AR application assists the worker during the assembly process. For example, it can highlight existing parts involved, display virtual parts to assemble, show an animation illustrating the operation to perform, display an arrow pointing to the direction of an industrial closet, and highlight the bin containing the required fixtures when it becomes visible.
Maintenance and repair. In a similar way, the AR application helps users perform maintenance and repair operations on the shop floor. In the case of field service in remote areas, a remote expert can efficiently support the technician by actually seeing what the operator sees on-site and collaborating with him. In a consumer scenario, customers that need to perform a repair task at home, such as replacing the cooling fan for their laptop, are able to download instructions to their AR device. Looking at the laptop, they are then guided, step by step, on exactly how to disassemble the laptop then put it back together again with the new fan.
Training. The AR applications used for assembly or maintenance and repair can be adapted for hands-on training.
Quality control. The AR application assists the Quality expert during quality inspection procedures. For example, it allows the user to compare the actual geometry with the as-designed geometry or to check key characteristics at highlighted locations of complex products (e.g. to count the number of screws securing a lid). The user can also designate defective spots found directly on a part, then feed the location back to the Quality Management system or MES.
Warehouse management. The AR application assists warehouse pickers by showing arrows directing them to the items they are looking for and highlighting them once they are visible. Relevant inventory levels and KPIs are displayed on the screen.
Factory layout. The AR application helps manufacturing space designers assess the feasibility of changes in the factory layout, thanks to a realistic visualization of the current facilities mixed with additional virtual resources, such as new robots.

The devices supporting these applications can be handheld devices, such as tablets or smart phones, head-mounted devices, such as Microsoft’s HoloLens™, DAQRI Smart Helmet™, or projectors[1].

In most of these applications, integration with a PLM parts database as well as with various manufacturing processes (work instructions, quality, warehouse management, people skills management), is highly desirable. Coherence between 3D models (products, assets and factories) and manufacturing operations is also required.

Some important excepted benefits…

The benefits of AR for work instructions (our first scenario) were estimated by a Boeing study in 2014-2015. About 50 participants had to accomplish a series of tasks that were representative of a typical assembly process at Boeing. They were divided in 3 groups:

The first group used a touch monitor to read work instructions from a PDF document, as done routinely at Boeing. The monitor was at a fixed location and was not visible from the assembly workstation.
The second group read the same PDF document but from a tablet that was readily accessible and visible from the assembly workstation.
The third group also used a tablet, but with an experimental AR system showing immersive 3D work instructions in context.

Boeing measured the median number of assembly errors for first-time assembly (training) and for final assembly:

The first group made 8 errors at first, then 4 errors.
The second group made 1 error in both cases.
The third group (with the AR system) made 0.5 error at first and no error afterwards.

These results represent potential huge benefits for Boeing[2]!

Other qualitative benefits can be deduced from the scenarios above: better and quicker training thanks to hands-on realistic experience, best practices captured from highly skilled users, reduced need for travel to the job site thanks to remote assistance, quicker warehouse picking with less errors, and a faster, more confident path to an optimal factory layout.

…But adoption rate is still slow

Given all the benefits above, one could expect to find a lot of testimonies of AR usage in the industry. But in fact, while many companies experiment with AR, it is difficult to find articles describing advanced AR implementations beyond pilot projects.

Lockheed-Martin has communicated about the use of AR for assembly and repair of the F-35, allowing engineers to work 30% faster, according to the company that built the software. But this still appears to be a trial.

At Volkswagen’s Wolfsburg plant, the deployment of smart glasses for order picking has started with 30 employees. However, the AR usage appears pretty simple, with glasses used as barcode readers and to display storage locations and part numbers directly in the field of vision, allowing workers to work hands-free.

BMW has shared a great video of a mechanic repairing a car engine with an overlay of virtual parts upon real ones, and in-context animations. But the video was released in 2007, and augmented reality is nowhere to be found on the BMW corporate Web site at the time this article was written.

It is worth noting that AR first appeared in Gartner’s technology hype cycle in 2004. More than 10 years later, AR is still in the “trough of disillusionment” section of the 2015 curve and mainstream adoption will not occur for 5 to 10 years, according to Gartner.

This is because AR requires a lot of advanced technologies and some of them are still maturing. Areas of much needed improvement include:

Hands-free light-weight devices with higher resolution, larger field-of-view and elegant and accurate optics
Lower latency— AR needs to respond quickly (in less than 15ms) as the user moves his eyes, head or body[3].
Smarter object recognition and depth-sensing technologies
Eye movement detection and other tracking of human input (gesture, voice-based interaction)
Emergence of standards for the development of AR applications across devices
High-speed connections allowing AR apps to integrate with the manufacturer’s back-end systems (Manufacturing Operations Management, PLM and ERP systems), and other devices.

How elaborate will your AR application be?

Does this mean that industries have to wait the 5 to 10 years for the technology to be mature, as estimated by Gartner?

Well, it depends on the kind of AR app we are talking about. We can identify 4 levels of sophistication for AR apps, as shown in the figure below:

AR maturity levels

What is the situation today?

Level 4 is still a dream.
Level 3 experimentations are becoming possible with devices such as Microsoft HoloLens™.
Level 2 pilot projects are still being conducted
Level 1 applications already exist in the industry and bring tangible benefits. These examples include facilitating field service in the oil and gas industry or, as we saw with the Volkswagen use case above, optimized order picking

According to Patrick Ryan from Index AR Solutions, the quality control, assembly and training applications are the most mature applications of AR. Assuming industrials target the low-hanging fruit, we can expect in the short term to hear more stories of location-based AR applications in these domains – even though some say this is not AR!

But, for Pokémon Go, who cares?

If you liked this article, here are other materials you might also find interesting:

[1] For example, to project a digital jig to stud welding.

[2] Assembly time measurements also showed a reduction of the build time, though not in the same proportion.

[3] Otherwise, the experience will not feel authentic and simulator sickness will likely occur