Industrial augmented reality

Augmented reality device
Pick-by-Vision system developed by the Lehrstuhl fml and its partners (CIM GmbH and Fachgebiet für Augmented Reality)

Industrial augmented reality (IAR) is related to the application of augmented reality (AR) to support an industrial process.[1] The use of IAR dates back to the 1990s with the work of Thomas Caudell and David Mizell about the application of AR at Boeing.[2] Since then several applications of this technique over the years have been proposed showing its potential in supporting some industrial processes. Although there have been several advances in technology, IAR is still considered to be at an infant developmental stage.[3][4][5]

Some challenging factors of IAR development are related to the necessary interdisciplinarity knowledge in areas such as object recognition, computer graphics, artificial intelligence and human-computer-interaction. Where a partial context understanding is required for the adaptation to unexpected conditions and understand the user's actions and intentions. Additionally user intuitive interfaces still remain a challenge likewise hardware improvements such as sensors and displays.[4][6][7]

Further, some controversy prevails about the boundaries that define IAR and its potential benefits for some activities with the currently available technology.[8]

Technology

Origins and history

Although the origins of Augmented Reality dates from the 1960s, when Ivan Sutherland created the first head-mounted display [9] it did not gain strength until the early 1990s, when the David Mizell and Thomas Caudell developed the first industrial AR at Boeing. They used a head-mounted display (HDM) to superimpose a computer-generated diagram of the manufacturing process with a real-time world registration and the user's head position calculation. They coined the name Augmented Reality to this technology.[2][10]

Contemporary several prototypes were proposed to demonstrate AR's application to manufacturing:[11] a laser-printer maintenance application was proposed in 1993 by Steven K. Feiner and coauthors by introducing the concept of knowledge-based AR for maintenance assistance (KARMA).[12][13] Whitaker Ross et al. proposed a system to display the name of the part pointed by the user in an engine.[14]

By the 2000s, the interest in AR had grown considerably. Some important groups were funded:[10] the largest consortium for IAR backed by the German's Federal Ministry of Education and Research (ARVIKA), with the aim of researching and implementing AR in relevant German industries,;[15] the European Community founded several projects including Service and Training through (STAR), which is a collaboration between institutes and companies from Europe and the US,[16] and Advanced Augmented Reality Technologies for Industrial Service Applications (ARTESAS) derived from ARVIKA, focused on the development of AR for automotive and aerospace maintenance.[17] Likewise, from other countries such as Sweden, Australia and Japan with the aim of encourage the IAR development.

From the beginning of 2010 until today, advances in hardware devices such as the wearable Google Glass, the reduced cost of mobile devices, and the increasing user familiarity with this technology.[7] Besides the increasing product-development complexity where products are becoming more versatile and intricate, with multiple variations and mass customization. Opened new scenarios for this technology.[18]

Overview

Industrial part visual characteristics duplex ball valves

One of the most promising fields of AR application is industrial manufacturing, where it can be used to support some activities in product development and manufacturing[19] through providing information available to reduce and simplify the user's decisions.[20] The general issues of the development of an AR system can still be classified into:[21]

Enabling technology

MannGlas and GoogleGlass1

There are technologies needed to build AR systems. Some of them are directly related with the performance of the software and hardware that enable the deployment of AR, such as displays, sensors, processors, recognition, tracking, registration among others.[21] Thus AR uses different approaches to integrate the virtual and real worlds, where several technologies influence the usability and applicability.[22]

Some common unsolved issues concern tracking systems suited for industrial scenarios which mean: poorly textured objects with smooth surfaces and strong light variation; object recognition using natural features when it is not possible to use markers;[7] the improvement of accuracy and latency of registration,[4] and 3D context scene capture to allow context awareness.[6]

User interaction

The limited understanding of human factors is likely to be obstructing the spreading of IAR beyond laboratories prototypes.[4] Their study is challenged to overcome technological issues (deficiencies in resolution, field of view, brightness, contrast, tracking systems, among others) in order to separate the AR performance from interface factors and technological issues.[23]

It was suggested by[10] that for an IAR application to be successful in a commercial environment, it has to be "user friendly", meaning that it needs to be easy and safe to setup, learn, use and customize and the user should feel free to move with an AR system.[8] As well as the use of natural interfaces in order to control AR by using natural movements of the body have also motivated a good deal of research. The reason for this is that the usability not only depends on the system's stability but also on the control interface's quality.[22]

Further, the user interface should avoid overloading the user with information and prevent over relying on it in order to avoid losing important cues from the environment.[24] Other issues are also related with improving multiple user collaboration[6]

Social

It is the final challenge, given an ideal AR system (hardware, software and an intuitive interface) to be accepted and become a part of a user's daily life.[21]

Consequently, one of the most important factors related with the adoption of any new technology is the perception of usefulness, and AR needs to show a clear cost-benefit relation.[25] Some studies suggest that, in order for AR to be perceived as useful the task should be high enough to require its use.[26]

Other non-addressed but important issues for technology acceptance are related with fashion, ethics, and privacy.[21]

Applications

Assembly

Assembling is the process of putting together several separate components in order to create a functional one. It can be performed in different stages of the product's life.[27] Even though nowadays many assembly operations are automated, some of them still require human assistance as, in many cases, their bits of information are detached from the equipment. Thus it is necessary to alternate their attention which leads to decreasing productivity and increasing of errors and injuries[18]

The use of AR is encouraged by the premise that instructions might be easier to understand if instead of being available as manuals they are super imposed upon the actual equipment.[11] Some of the uses of AR in the support of the assembly can be categorized into:[6]

Similarly, by using AR it is possible to simulate the user's motion during assembly to acquire an accurate and realistic movement of virtual parts.[28]

On the other hand, some of the critical issues of the support assembly task are related with the dynamic reconfiguration of the state diagram, which allows to automatically identify the step of assembly and also adapt to unexpected actions or errors of the user.[6][7] Thus, defining 'what', 'where', and 'when' to display information becomes a challenge since it requires a minimum understanding of the surrounding scene.[29]

Maintenance and repair

Like the assembly task, maintenance serves as a natural application for AR because it is a tasks that requires keeping the user's attention on a specific area, and also synthesizing additional information such as complex sequences, component identification, and textual data.[30]

The maintenance activity can be supported by displaying related information about an unfamiliar piece of equipment to a technician instead of searching it in a repair manual.[31] Similarly, AR can support maintenance tasks by acting as an "x-ray" like vision, or providing information from sensors directly to the user.[24]

It can also be employed in repairing tasks. For instance, in the diagnosis of modern cars whose status information can be loaded via a plug-like connector. AR can be used to immediately display the diagnosis on the engine[3]

Training

Many industries are required to perform complex activities that needs previous training. Therefore, for learning a new skill, technicians need to be trained in sensorimotive and cognitive of sub-skills that can be challenging. This kind of training can be supported by AR.[32]

Additionally, the possibility to use AR to motivate both trainees and students through enhancing the realism of practices has been suggested.[33] By providing instructions with an AR, the immediate capability to perform the task can be also accomplished.[34]

Other advantages of using AR for training are that students can interact with real objects and, at the same time, have access to guidance information,and the existence of tactile feedback provided by the interaction with real objects.[32]

Quality control and commissioning

By displaying information about the components of manufacturing in real time. For instance, Volkswagen has used it to verify parts by analyzing their interfering edges and variance. AR has also been used in automobile development to display and verify car components and ergonomic test in reality. By superimposing the original 3D model over the real surface, deviations can easily be identified, and therefore sources of error can be corrected.[35][36]

Monitoring and visualization

The use of AR have been proposed to interact with scientific data in shared environments, where it allows a 3D interaction with real objects compared to virtual reality, while allowing the user to move freely in the real world.[37] Similar systems for allowing multiple user's with HMD can interact with dynamic visual simulations of engineering processes[38]

In the same way, AR simulation of working machinery can be checked from mobile devices as well as other information such as temperature and, time of use, which can reduce worker's movements and stress.[19]

Controversy

The benefits of implementing AR into some industrial activities have been of a high interest. However, there is still a debate as the current level of the technology hides all of its potential.[4][8]

It has been reported that, in maintenance and repair, the use of AR can reduce time to locate a task, head and neck movements, and also other disadvantages related to bulky, low resolution display hardware[39] In addition, in training it aims to improve the performance up to 30% and, at the same time reduce costs by 25%.[40]

Similar benefits have been reported by Juha Sääski et al. in the comparative use of AR versus paper instructions to support the assembly operation of the parts of a tractor's accessory power unit which showed a time and error reduction (six time less).[41]

But, on the other hand, the long-term use has been reported to cause stress and strain to the user. Johannes Tumler et al. compared the stress and strain produced by picking parts from a rack using AR versus using paper as a reference, which showed as result a change of the strain that could be assumed by the non-optimal system.[42]

Additionally, it has been suggested [32] that one potential danger in use AR for training is the user dependability of this technology. As a consequence the user will not be able to perform the task without it. In order to overcome this situation, the information available during training needs to be controlled.

The boundaries that define IAR are still not clear. For instance, one of the most accepted definitions[11] of AR implies that the virtual elements need to be registered. But in industrial field, performance, is a main goal, and therefore has been an extensive research about the presentation of virtual components in AR regarding the type of the task. this research has shown that the optimal visual aid type may variate depending on the difficulty of the task.[43]

Finally, it has been suggested that in order to have commercial IAR solutions, they must be:

References

  1. ↑ Fite-Georgel, Pierre (2011). "Is there a reality in Industrial Augmented Reality?". Mixed and Augmented Reality (ISMAR), 2011 10th IEEE International Symposium on. IEEE. pp. 201–210.
  2. 1 2 Caudell, Thomas P; Mizell, David W (1992). "Augmented reality: An application of heads-up display technology to manual manufacturing processes". System Sciences, 1992. Proceedings of the Twenty-Fifth Hawaii International Conference on. Presence: Teleoperators and virtual environments. 2. IEEE. pp. 659–669.
  3. 1 2 Regenbrecht, Holger and Baratoff, Gregory and Wilke, Wilhelm (2005). "Augmented reality projects in the automotive and aerospace industries". IEEE Computer Graphics and Applications. IEEE. 25 (6): 48–56.
  4. 1 2 3 4 5 Nee, AYC and Ong, SK and Chryssolouris, G and Mourtzis, D (2012). "Augmented reality applications in design and manufacturing". CIRP Annals-Manufacturing Technology. Elsevier. 61 (2): 657–679.
  5. ↑ Pentenrieder, Katharina and Bade, Christian and Doil, Fabian and Meier, Peter (2007). "Augmented Reality-based factory planning-an application tailored to industrial needs". IEEE: 31–42.
  6. 1 2 3 4 5 Wang, X and Ong, SK and Nee, AYC (2016). "A comprehensive survey of augmented reality assembly research". Advances in Manufacturing. Springer. 4 (1): 1–22.
  7. 1 2 3 4 Lamberti, Fabrizio and Manuri, Federico and Sanna, Andrea and Paravati, Gianluca and Pezzolla, Pietro and Montuschi, Paolo (2014). "Challenges, opportunities, and future trends of emerging techniques for augmented reality-based maintenance". IEEE Transactions on Emerging Topics in Computing. IEEE. 2 (4): 411–421.
  8. 1 2 3 Hincapié, Mauricio and Caponio, Andrea and Rios, Horacio and Mendívil, Eduardo González (2011). "An introduction to Augmented Reality with applications in aeronautical maintenance". 2011 13th International Conference on Transparent Optical Networks. IEEE. pp. 1–4.
  9. ↑ Sutherland, Ivan E (1968). "A head-mounted three dimensional display". Proceedings of the December 9–11, 1968, fall joint computer conference, part I. ACM. pp. 757–764.
  10. 1 2 3 Navab, Nassir (2004). "Developing killer apps for industrial augmented reality". Computer Graphics and Applications, IEEE. IEEE. 24 (3): 16–20.
  11. 1 2 3 Ronald T, Azuma (1997). "A survey of augmented reality". Presence: Teleoperators and virtual environments. MIT Press. 6 (4): 355–385.
  12. ↑ Feiner, Steven and MacIntyre, Blair and Seligmann, Dorée (1992). "Annotating the real world with knowledge-based graphics on a see-through head-mounted display". Proceedings of the conference on Graphics interface'92. Morgan Kaufmann Publishers Inc. pp. 78–85.
  13. ↑ Feiner, Steven and Macintyre, Blair and Seligmann, Dor{\'e}e (1993). "Knowledge-based augmented reality". Communications of the ACM. 36 (7). ACM. pp. 53–62.
  14. ↑ Whitaker, Ross T and Crampton, Chris and Breen, David E and Tuceryan, Mihran and Rose, Eric (1995). "Object calibration for augmented reality". Computer Graphics Forum. 14. Wiley Online Library. pp. 15–27.
  15. ↑ Wohlgemuth, Wolfgang and Triebfurst, Gunthard (2000). "ARVIKA: augmented reality for development, production and service". Proceedings of DARE 2000 on Designing augmented reality environments. ACM. pp. 151–152.
  16. ↑ Raczynski, A and Gussmann, P (2004). "Services and training through augmented reality". Visual Media Production, 2004.(CVMP). 1st European Conference on. IET. pp. 263–271.
  17. ↑ "Project ARTESAS - Advanced Augmented Reality Technologies for Industrial Service Applications". WZL. Retrieved 2016. Check date values in: |access-date= (help)
  18. 1 2 Ong, SK and Yuan, ML and Nee, AYC (2008). "Augmented reality applications in manufacturing: a survey". International journal of production research. Taylor & Francis. 46 (10): 2707–2742.
  19. 1 2 Shoaib, Huma and Jaffry, S Waqar. "A Survey of Augmented Reality". interactions. 24: 34.
  20. ↑ Gaukrodger, Stephen J and Lintott, Andrew (2007). "Augmented reality and applications for assistive technology". Proceedings of the 1st international convention on Rehabilitation engineering &assistive technology: in conjunction with 1st Tan Tock Seng Hospital Neurorehabilitation Meeting. ACM. pp. 47–51.
  21. 1 2 3 4 Azuma, Ronald and Baillot, Yohan and Behringer, Reinhold and Feiner, Steven and Julier, Simon and MacIntyre, Blair (2001). "Recent advances in augmented reality". IEEE computer graphics and applications. IEEE. 21 (6): 34–47.
  22. 1 2 Chi, Hung-Lin and Kang, Shih-Chung and Wang, Xiangyu (2013). "Research trends and opportunities of augmented reality applications in architecture, engineering, and construction". Automation in construction. Elsevier. 33: 116–122.
  23. ↑ Livingston, Mark A (2005). "Evaluating human factors in augmented reality systems". IEEE Computer Graphics and Applications. IEEE. 25 (6): 6–9.
  24. 1 2 Van Krevelen, DWF and Poelman, R (2010). "A survey of augmented reality technologies, applications and limitations". International Journal of Virtual Reality. 9 (2): 1.
  25. ↑ Davis, Fred D (1989). "Perceived usefulness, perceived ease of use, and user acceptance of information technology". MIS quarterly. JSTOR: 319–340.
  26. ↑ Syberfeldt, Anna and Danielsson, Oscar and Holm, Magnus and Wang, Lihui (2015). "Visual assembling guidance using augmented reality". Procedia Manufacturing. Elsevier. 1: 98–109.
  27. ↑ Whitney, D.E (2004). Mechanical Assemblies: Their Design, Manufacture, and Role in Product Development. Oxford University Press. ISBN 9780195157826.
  28. ↑ Wang, X and Ong, SK and Nee, AYC (2016). "Real-virtual components interaction for assembly simulation and planning". Robotics and Computer-Integrated Manufacturing. Elsevier. 41: 102–114.
  29. ↑ Yuan, ML and Ong, SK and Nee, AY (2008). "Augmented reality for assembly guidance using a virtual interactive tool". International Journal of Production Research. Taylor &Francis. 46 (7): 1745–1767.
  30. ↑ Henderson, Steven J and Feiner, Steven K (2007). Augmented reality for maintenance and repair (armar) (Technical report). DTIC Document.
  31. ↑ Schwald, Bernd and De Laval, Blandine (2003). "An augmented reality system for training and assistance to maintenance in the industrial context". UNION Agency--Science Press.
  32. 1 2 3 Webel, Sabine and Bockholt, Uli and Engelke, Timo and Gavish, Nirit and Olbrich, Manuel and Preusche, Carsten (2013). "An augmented reality training platform for assembly and maintenance skills". Robotics and Autonomous Systems. Elsevier. 61 (4): 398–403.
  33. ↑ Chang, George and Morreale, Patricia and Medicherla, Padmavathi (2010). "Applications of augmented reality systems in education". Proceedings of Society for Information Technology & Teacher Education International Conference. 1. pp. 1380–1385.
  34. ↑ Chimienti, Viviana and Iliano, Salvatore and Dassisti, Michele and Dini, Gino and Failli, Franco (2010). "Guidelines for implementing augmented reality procedures in assisting assembly operations". International Precision Assembly Seminar. Springer. pp. 174–179.
  35. ↑ Frund, Jurgen and Gausemeier, Jurgen and Matysczok, Carsten and Radkowski, Rafael (2004). "Using augmented reality technology to support the automobile development". International Conference on Computer Supported Cooperative Work in Design. Springer. pp. 289–298.
  36. ↑ Nolle, Stefan and Klinker, Gudrun (2006). "Augmented reality as a comparison tool in automotive industry". Proceedings of the 5th IEEE and ACM International Symposium on Mixed and Augmented Reality. IEEE Computer Society. pp. 249–250.
  37. ↑ Szalavári, Zsolt and Schmalstieg, Dieter and Fuhrmann, Anton and Gervautz, Michael (1998). ""Studierstube": An environment for collaboration in augmented reality". Virtual Reality. Springer. 3 (1): 37–48.
  38. ↑ Dong, Suyang and Behzadan, Amir H and Chen, Feng and Kamat, Vineet R (2013). "Collaborative visualization of engineering processes using tabletop augmented reality". Advances in Engineering Software. Elsevier. 55: 45–55.
  39. ↑ Henderson, Steven and Feiner, Steven (2011). "Exploring the benefits of augmented reality documentation for maintenance and repair". IEEE transactions on visualization and computer graphics. IEEE. 17 (10): 1355–1368.
  40. ↑ "Smart Maintenance An Augmented Reality Platform for Training and Field Operations in the Manufacturing Industry". AR BLOG. 2014. Retrieved 2016. Check date values in: |access-date= (help)
  41. ↑ Saaski, Juha and Salonen, Tapio and Liinasuo, Marja and Pakkanen, Jarkko and Vanhatalo, Mikko and Riitahuhta, Asko; et al. (2008). "Augmented reality efficiency in manufacturing industry: a case study". DS 50: Proceedings of NordDesign 2008 Conference, Tallinn, Estonia, 21.-23.08. 2008.
  42. ↑ Tumler, Johannes and Doil, Fabian and Mecke, Rudiger and Paul, Georg and Schenk, Michael and Pfister, Eberhard A and Huckauf, Anke and Bockelmann, Irina and Roggentin, Anja (2008). "Mobile Augmented Reality in industrial applications: Approaches for solution of user-related issues". Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality. IEEE Computer Society. pp. 87–90.
  43. ↑ Radkowski, Rafael (2015). "Investigation of Visual Features for Augmented Reality Assembly Assistance". International Conference on Virtual, Augmented and Mixed Reality. Springer. pp. 488–498.
This article is issued from Wikipedia - version of the 10/30/2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.