Human Error Prediction Using Eye Tracking to Improvise Team Cohesion in Human-Machine Teams

Human Error Prediction Using Eye Tracking to Improvise Team Cohesion in Human-Machine Teams
Name
Course
Date
Human Error Prediction Using Eye Tracking to Improvise Team Cohesion in Human-Machine Teams
Introduction
The upsurge in the incorporation of intelligent frameworks in most of the human-life aspects has formed the foundation for better possibilities regarding increased interactions with machines and devices. The complexity of the collaboration has facilitated the discovery of human-machine teaming (HMT) as a newly emerging field. The evolving relationship between human and machines has become a primary theme with most developments aiming at addressing the complexity factors. Today, the intelligence emanating from machines has been instrumental in the incorporation into different elements of the human life as indicated by IBM survey results. The international organization shows that 75% of respondents in their survey show that intelligent machines ought to have a meaningful impact in business in the next three years. A variety of approaches have been used to identify the reliability of human predictions. Several industries admit that depending on the intuition of human beings have often led to catastrophic events, some of which could be prevented efficiently if machines were used. The use of intelligent devices to measure errors is presumed to offer an enhanced method to alleviate the occurrences of catastrophic events. Still, 70% of organizations involved in the survey believed that intelligent machines could result in a higher-value work for employees if adjustments were made to HMTs.

Essentially, the machines have made it possible to increase the performance of humans in physical, emotional, and cognitive domains in most industries.

Figure SEQ Figure * ARABIC 1: Intelligent Automation Adoption across Industries (https://public.dhe.ibm.com/common/ssi/ecm/gb/en/gbe03879usen/human-machine-interchange.pdf)
Moreover, this is apparent from the results of technologies such as bioacoustics sensing, quantified self, and human augmentation among other innovations focusing on augmented reality and gesture control. Even so, there lacks comprehensive understanding of the dynamics of trust and cohesion between people and machines.
Nature of Human-Machine Framework
The customary human-machine framework is characterized by associations and processes that mostly take place in realistic contexts. Additionally, depending on the application of the system, the machines interact with the humans through interfaces and controls (See Figure 2). The interfaces mostly entail panels and indicators for displays in addition to decision support tools. Conversely, the controls helps the humans in automation and operating on the system as the standard ways for implementing their intentions and strategies.

Figure SEQ Figure * ARABIC 2: Typical Structure of a Human-Machine Interaction System (http://www.springer.com/cda/content/document/cda_downloaddocument/9781852337056-c1.pdf?SGWID=0-0-45-130559-p26633351)
Despite the practicality of this explanation, it only refers to the most typical HMTs such as those used in factories. Additionally, it does not allude to the contemporary human-machine interaction that is a multi-disciplinary field focusing on facets such as artificial intelligence, robotics, voice control interfaces, and social sciences, among others. The system has been subject to augmented evolution and alterations. Today, the human-machine teams involve collaborative efforts from both entities with a mutual goal of accomplishing a specific goal and task. Nonetheless, despite having an intelligent framework, the machines do not have comprehensive control over a task. Therefore, the completion of the task involves the leveraging of best skills of human and machine. For instance, humans have optimal skills such as Correlation identification, Perspective & Emotional intelligence while the machines are more dominant in Scouring of Data, All-pairs testing and Classification. Some of the common machines used in HMT include hardware robots with Artificial Intelligence-based operating systems or complete software agents. Moreover, the completion of the multi-agent setup includes linked data resources that positively contribute to the interactions (See Figure 3).

Figure SEQ Figure * ARABIC 3: Multi-Agent Environment [Example of Modern HTM] (https://www.semanticscholar.org/paper/Distributed-Linked-Data-as-a-Framework-for-Human-M-Pareti/f03704a9d6dde129203b8dca382b2b00448bfb26)
Importance of Human-Machine Teams
Machines are simulated agents with capabilities of perception and action in the natural world. In the modern setting, their utilization has become a paramount factor in technologically advanced societies that play equally significant roles in crucial domains. In the context of human-machine teams, a range of elements come into play in determining the degree at which human trusts the machine, and their level of productivity in dealing with specific tasks. Some of these elements include the reliability of the machine, the self-confidence of the humans, and the synchronized workload.
Certain tasks require the machines to classify objects, identify, and detect humans and their emotions. Typically, the urge for energetic dimensions prompts the expansion of all the sub-fields of human-machine operations. Moreover, there are increased possibilities and potential for HMT. Consequently, with the expansion of human-machine cohesions, the current and advanced technologies could facilitate scenarios where the team members collaboratively assist each other in control, co-ordination, and achievement of tasks without any errors.
Both humans and machines can be erroneous, particularly in circumstances with high levels of indecision and indistinctness. Therefore, a better calibration of cohesion between machines and human is a significant necessity of good team performance. Commendably, industrial machines have been a major integration into industrial assembly lines and have indicated collaborative effectiveness with humans. However, as part of the team’s functionality, it is significant that team members have that forecast the courses of action of fellow members for improved accuracy through minimal errors.
Focus of the Paper
Despite the prevalence of multiple protuberant approaches that utilize facial lexes to identify prompt errors or task-related performance errors, HMTs lack the capabilities of forecasting errors during task execution. Resolutely, the paper entails the proposition of an approach to exploit eye tracking with eye point data as a means for measuring the concentrated area followed by the utilization of data to forecast both the type and percentage of errors. Notably, the proposed plan utilizes chronological and spatial procedures to recognize eye fixations and saccades.

Related Work
Source Details Research Premise/Background Area of Focus for the Study Results Conclusions and Gaps
Mu et al (2015) An increasing dependency on machines marked with augmented accidents from human error
Need for a human reliability analysis (HRA) Construction of a prediction method of human error probability (HEP) through the integration of performance shaping factors and Bayesian networks theory The value of human error probability = probability interval predicted with the CREAM prediction method
The method could effectively address challenges on correlation and the dissimilar weight of PSFs by casual model of Bayesian networks and CPT of node
Increased gaps such as the difficult acquisition of CPT in Bayesian networks
Yang and Shah (2017) The probable advantages of predictive alarm systems that could identify human errors before they occurred Discovering how interface design could influence the calibration of trust and reliability between humans and machines Human-subject experiment (N=91) indicated trust of entirety evolves over time
In high-risk industries, the inception of identifying an error are mostly set very low in an attempt to solve every crucial error
The limited threshold has multiple deficiencies such as false alarms resulting in loss of trust among the human users, which eventually results to additional errors. The predictive alarm would be influential in helping alleviate the ‘cry wolf’ effect as a sensation popular in high-risk industries operating with both humans and machines
Does not focus on how HMTs can be improved through prediction of human errors by machines
Yang and Hölttä-Otto (2016) [SUTD] Humans calibrate their trust in machines based on past experiences, which is a key determinant of human error Memory and recognition task with the help of an automated decision aid Respondents under-trusted highly reliable aids and over-trusted the unreliable ones
2.1% increment of trust upon receiving a valid recommendation
4.8% decrement due to invalid recommendation The cohesion between humans and machines evolves and stabilizes over time, which positively influences the prevalence of errors
Fails to indicate how HMTs can be improved through prediction of human errors by machines

Method
Eye-tracking mechanisms that can be used for error predictions are believed to provide superior outcomes that can eliminate human errors. The methodology section looks at gazing eye mechanisms and the measurement of cognitive eye movement to estimate error-making possibilities. The method section will base the findings of the error prediction mechanism on the outcomes of the experimental processes, derived by using the gazing mechanisms and the web cam applications to identify incidences of errors
Use of Video-Based Gaze Tracking
Video-based methods of eye tracking provide a suitable method of gaze tracking. The experiment relied on the non-invasive video-based tracking method to attain accurate results of when errors are made. The non-invasive method relied on a multi-camera eye tracker. A major benefit of the methodology is that it entails minimal mathematical details, which may make the calculation processes and methods of generating results complex.
The first step was to illuminate the eye with an infrared light source. The light was essential in the production of glints on the cornea of the eye. The reflection generated (corneal reflection), was a useful output source to help identify the reference point for the estimation of the gaze. The glint vector produced by the pupil would remain constant irrespective of the eye and head movement. Employing a single camera fails to produce the desired illumination effect. The same outcomes were also evident in the research by Chennamma and Yuan (2013), who emphasized that a collection of mirrors would yield accurate results than when using only one camera.
Another lighting source was used to help in the estimation of the gazes. The results would be tabulated on a mounted camera using an incremental approach. The system also captures 3-D positions of the eyeball and depending on the head pose of the party involved in the experiment. The gaze tracing mechanism using a web cam alternative would also broaden the scope and application of the eye tracking solutions in use. Using several systems and light centers also enhances the possibility of obtaining clear corneal glints and pupil centers. The methodology offers a higher degree of accuracy and provides significant tolerance to head movements.
The image below is an example of a video-based gaze tracking system, with two light sources

The above gazing method bases its working mechanism on the ability to trace the directions of eye movements. The prediction mechanism is based on the eye position and the tracker for the calibration balls. Based on the light source, the calibration camera source ensures that the eye targets a given calibration ball at a go. Errors are tracked whenever the light hits the eye and the gaze is based on a different calibration ball. Accordingly, the process provides evidence on the ability to generate different sets of images for different eye movements. When the eye is captured on various actions, it is simpler to develop feasible mechanisms upon which the simulation of errors can also be identified . The gaze tracker experiment indicates that the eyes move in varying directions depending on the nature of its reactions. The proposed methods of eye tracking provide feasible mechanisms upon which data garnered from the eye tracking method can be used to develop a workable prediction model and a corresponding mechanism to predict accuracy and hence minimize incidences of errors.
Suitable outcomes on the assessments were dependent on the results garnered in the process. For instance, irrespective of whether a post-completion error occurred or not, the data collected in the process would be useful in generating a fixation measure to conduct better prediction capacities of alerting human beings on the nature of errors that they commit.
Measurement of Cognitive State Impact on Eye Movement in Real Time
Eye movement measures are dependent on the cognitive state of the individual. The fact that the eye movements are an essential aspect in evaluating human abilities to avoid making errors implies that a better understanding of the cognitive state of a person would yield superior outcomes. Ratwani and Trafton cite Bryne and Bovair in which they developed eye movement methods collected in real time. Resolutely, the eye movements were presumed to provide relevant predictors of the likelihood of errors. In this experiment, the use of webcams to record eye movements was found useful. The predicted position plotted on the axis, yielded a straight-line graph that was to be followed in the process. The results obtained out of the process were plotted to identify incidences of errors and accuracies in the process, to help identify any shifts in the eye movements. The prediction model was able to offer real-time information on the possibility of the occurrence of an error. Unfortunately, there is no real time real-time system that can be used for the detection of individuals making mistakes. Nonetheless, applying the methodology of responding to cognitive tasks based on stimulus responses shows that accurate results may be obtained through data processing and brain activity in real time.
Eye movement measures provide feasible results that can be used in making conclusive results regarding the possibility of making human errors. The eye movement measures offer available results that can be processed immediately as users interact with their environments. The method is useful compared to others, which often record substantial time delays. Moreover, Shumeet and Pomerleau recommend that the process is nonintrusive and noninvasive. Using a reliable method of eye tracking technology offers unaltered mechanisms to collect data that is of free-range nature, in natural setups. The process can be achieved in seconds once the required calibration method is set. The eye-tracking options have proved to provide superior outcomes in a variety of field experiments. The methodology has been more effective in field experiments such as tracking the ability of truck and car driver’s fatigue. A similar method was employed in the research conducted by Maguire to help identify human errors amongst military aircrews when in a new working environment. The use of the predictive model was bound to give accurate results, which help in identifying the increases in the probability of making errors. In effect, a logistic regression model can be created to help build a graph that can show the predicted probabilities of making an error when engaged in a task.
Visual samples of the way the eye reacts to different noise types. Retrieved from http://www.pnas.org/content/early/2018/01/22/1717948115

A drawing of the x and y-axis to predict incidences of errors was found to offer highly efficient results for the experiment. The predicted probabilities for all the cases are plotted as shown above on the graph. The outcomes indicate that when a user looks the post-completion button, there is a lower probability that a post-completion error will occur. Unfortunately, when the experiment gave too much dependence on the post-completion button it was bound to give inconsistent results given the fact that errors would occur during the completion of a task. Nonetheless, the model proved efficient in evaluating the probability of making errors provides a viable mechanism to help undertake corrective actions to ensure that post-completion errors do not take place.

Results and Discussion
Predictive eye movement approaches have been effective in facilitating a better translation of incidences of human errors. The error prediction systems discussed all rely on human intuition and the capacities of individuals to determine the robustness of a given model to make accurate predictions. The use of the models was found to be applicable in various sectors of the economy including the airline industry, whereby pilots were found to benefit from the outcomes of the model. Besides, those in the transport industry can rely on the predictive model to make accurate predictions thus minimize incidences of accidents amongst other common errors encountered in the process.
The first meth
od of understanding the cognitive mechanisms of human intuition indicated that better results could be yielded when implemented to facilitate real-time tracking. Nonetheless, the results showed that there are incidences when post-completion errors diminish accuracies, thereby leading to inevitable events. The efficiency of the process is also a by-product of different eye measures, which are developed to feasibly provide relevant results of the movements parallel with the performance of a task. The predictors from the regression model may vary depending on the degree of the error change, and the actual probabilities notably predicted. The graph shown below as described in the research by Raj and Trafton shows continuous fixations on the predicted probabilities and the binary fixation of the same. The constant fixations are represented on the x-axis and the expected probability on the y-axis.

The above graph derived from the concept used by Raj and Trafton illuminates the tabling of the X and Y coordinates of the eye movements based on the number of fixations. It shows instances of errors and correctness during the practice
The above graph indicates that when a user focuses on the post-completion button, there is a low probability that post-completion error will occur. However, failure to look at the fixated points implies that the user is highly likely to make an initial error in the first instance. Poor targeting of the action buttons tends to increase the number of fixation and hence a higher likelihood of error. The outcomes of the results are indicative of the initiatives that users ought to follow when using computers. Accordingly, a person must first understand all the relevant components of a system’s interface before operating it. Thus, the measure of observing the corresponding points is a measure of the ability of a user to remain focused and thus avoid making of errors when operating machinery.
Resolutely, the use of gaze tracking methods ensures that users have a better mechanism to resolve errors. Gaze monitoring methods are dependent on superior artificial intelligence technologies, which help to predict eye movements based on different environment setups and conditions. The methodology also proved to be sufficient in the assessment of environmental factors that may trigger error-making capacities. A combination of the methods is touted to provide precise mechanisms to predict possibilities of making errors. System developers may rely on the results garnered to make informed systems that can alert the users to ensure that making of errors is eliminated during operations. It would also ensure that a mechanism that supports automatic error correction is recommended, rather than a post-completion method.
Conclusion
Real-time error prevention mechanisms can be attained after the effective learning of the way they occur. The human eye is often exposed to various intuitions all of which have a significant effect on the possibility of making an error. Real-time post tracking methods have been found to offer effective mechanisms to minimize errors, especially in high-risk environments. Real-time post-completion prediction systems are suitable for capturing a user eye movement. Alternative systems also have a reliable error-prevention mechanism once the eye movements do not correspond with the intended action. The selection of an ideal method is dependent on various factors. Firstly, the paper illustrated that the ability to predict real-time eye movements is a necessary ingredient for the reduction of human errors. Besides, the ability to gauge human intuitions provides a feasible to support user interaction in various ways. Prediction models can be used in multiple ways to measure cognitive functions and evaluate the gazing aspects of the users. The ability to comprehend information on the predictive model was also found to play an instrumental role in improving the predictability of eye movements. The efficiencies of the systems employ reliable techniques with cutting-edge technology that can be used to resolve the challenges underwent in respective areas whereby errors occur. The accuracy of the methods should be independent to ensure the abilities to gauge performances of relevant structures to realize the efficiencies of the model.
BIBLIOGRAPHY
Andrist, Sean, Dan Bohus, Bilge Mutlu, and David Schlangen. “Turn-taking and coordination in human-machine interaction.” AI Magazine 37, no. 4 (2016): 5-7.
Baluja, Shumeet, and Dean Pomerleau. “Non-intrusive gaze tracking using artificial neural networks.” In Advances in Neural Information Processing Systems, pp. 753-760. 2014.
Chennamma, H. R., and Xiaohui Yuan. “A survey on eye-gaze tracking techniques.” arXiv preprint arXiv:1312.6410 (2013).
Dzeng, Ren-Jye, Chin-Teng Lin, and Yi-Cho Fang. “Using eye-tracker to compare search patterns between experienced and novice workers for site hazard identification.” Safety science 82 (2016): 56-67.
Eddy, Nathan. “Relationship between Humans, Machines Must Evolve; Gatner.” E-Week. 2013. http://www.eweek.com/small-business/relationship-between-humans-machines-must-evolve-gartner
Goswami, Anurag, Gursimran Walia, Mark McCourt, and Ganesh Padmanabhan. “Using Eye Tracking to Investigate Reading Patterns and Learning Styles of Software Requirement Inspectors to Enhance Inspection Team Outcome.” In Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, p. 34. ACM, 2016.
Guszcza, Jim, Harvey Lewis, and Peter Evans-Greenwood. “Why Humans and Computers Think Better Together.” Deloitte Insights. 2017. https://www2.deloitte.com/insights/us/en/deloitte-review/issue-20/augmented-intelligence-human-computer-collaboration.htmlHan, Junwei, Dingwen Zhang, Shifeng Wen, Lei Guo, Tianming Liu, and Xuelong Li. “Two-stage learning to predict human eye fixations via SDAEs.” IEEE transactions on cybernetics 46, no. 2 (2016): 487-498.
Johannsen, Gunnar. “Human-machine interaction.” Control Systems, Robotics and Automation 21 (2009): 132-62.
Kootstra, Gert, and Lambert RB Schomaker. “Prediction of human eye fixations using symmetry.” In The 31st Annual Conference of the Cognitive Science Society (CogSci09), pp. 56-61. Cognitive Science Society, 2009.
IBM Corporation. “The human-machine intelligent: How intelligent automation is reconstructing business operations.” IBM Institute for Business Value. 2017. https://public.dhe.ibm.com/common/ssi/ecm/gb/en/gbe03879usen/human-machine-interchange.pdfMaguire, Richard. “Validating a process for understanding human error probabilities in complex human computer interfaces.” Complexity in Design and Engineering (2005): 313-326.
Mashat, Arwa Abdulwahab. “An Eye Tracking Study to Investigate the Influence of Language and Text Direction on Multimedia.” PhD diss., Old Dominion University, 2017.
Massachusetts Institute of Technology (MIT). “Building Better Trust between Humans and Machines.” Robotics at MIT. 2018. https://robotics.mit.edu/building-better-trust-between-humans-and-machines
Mu, L., B. P. Xiao, W. K. Xue, and Z. Yuan. “The prediction of human error probability based on Bayesian networks in the process of task.” In Industrial Engineering and Engineering Management (IEEM), 2015 IEEE International Conference on, pp. 145-149. IEEE, 2015.
Ratwani, Raj M., and J. Gregory Trafton. “A real-time eye tracking system for predicting and preventing postcompletion errors.” Human–Computer Interaction 26, no. 3 (2011): 205-245.
Stanton, Neville A., and Christopher Baber. “Validating task analysis for error identification: reliability and validity of a human error prediction technique.” Ergonomics 48, no. 9 (2005): 1097-1113.
Stanton, Neville, Paul M. Salmon, and Laura A. Rafferty. Human factors methods: a practical guide for engineering and design. Ashgate Publishing, Ltd., 2013.
Yang, X. Jessie, Christopher D. Wickens, and Katja Hölttä-Otto. “How users adjust trust in automation: Contrast effect and hindsight bias.” In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 60, no. 1, pp. 196-200. Sage CA: Los Angeles, CA: SAGE Publications, 2016.
Yang, X. Jessie, Vaibhav V. Unhelkar, Kevin Li, and Julie A. Shah. “Evaluating Effects of User Experience and System Transparency on Trust in Automation.” In HRI, pp. 408-416. 2017.
Ziemke, Tom, Kristin E. Schaefer, and Mica Endsley. “Situation awareness in human-machine interactive systems.” (2017): 1-2.