Technical flying skills can be measured relatively easily, but qualities such as situational awareness, mental capacity, decisiveness, communications and resource management are harder to assess objectively.
The consequences are material. UK military pilot training is delivered through the UK Military Flying Training System (UKMFTS), a collaborative partnership between the UK Armed Forces, the National Armaments Director Group and industry, led by Ascent Flight Training. Training takes several years, so avoiding attrition in the later stages is a priority, when the investment in a trainee is already significant. Without objective insight into human factors, it can be difficult to identify performance problems early enough for targeted intervention.
Visual attention is guided by both an individual's goals and motivations, known as top-down control, and by the visual characteristics of the environment, known as bottom-up attention. These systems normally work together, but fatigue and demanding or stressful situations can affect attentional control and increase reliance on bottom-up attention. This shift can be seen in eye behaviour. Studies involving inhibition or target-locking of gaze have provided evidence of reduced top-down control of eye movements under stress, workload and fatigue.
Eye tracking can also be used to examine the patterns and efficiency of eye movements associated with expert performance. These can be linked to core airmanship principles within a competency-based assessment.
Endsley's theory of situational awareness provides a direct link. Level 1 SA, the perception of cues, depends on effective visual scanning to identify relevant information. That information supports understanding and comprehension at Level 2, and contributes to Level 3, the ability to forecast future events.
These relationships have been discussed in the literature for years. Turning them into useful signals, however, requires complex pre-processing, where advances in machine learning have helped considerably. This makes it possible to develop reliable construct models for performance assessment and incorporate other signals, including pupil size and heart rate. These are frequently, and inaccurately, treated as direct proxies for stress and cognitive load.
TACET-M is the defence-focused extension of the Cineon TACET platform. It is designed to introduce objective, data-driven assessment of airmanship into existing training environments without disrupting established training protocols.
At its core is Cineon's Empathic Learning Engine (ELE), a proprietary AI architecture that uses eye-tracking data to assess pilot behaviour and cognitive performance, including visual scanning, prioritisation, attention management and responses to workload under pressure. ELE has been trained on hundreds of richly labelled datasets collected across task-relevant operational contexts. This is intended to keep its models accurate and task-specific, identifying behavioural signals rather than relying on indirect proxies.
Eye-tracking, behavioural and task-performance data are processed through performance frameworks tailored to the operational context, producing an outcome against the relevant airmanship qualities.
Situational awareness is measured through scan patterns, visual coverage and fixation behaviour between critical areas of interest. Mental capacity is inferred from the ability to redirect attention and reprioritise information as task demands change. Decisiveness is indicated by visual search and repeated checking behaviours, providing insight into the information-gathering that underpins decision making. Resource management is assessed through how effectively attention is balanced between internal cockpit tasks, external threats, mission objectives and supporting assets. Communications is assessed primarily through communication content, timing, accuracy and procedural compliance, rather than through eye tracking alone.
These assessments are not intended to replace the instructor. They reveal aspects of performance that can be difficult to observe directly, but their outputs are only useful when the context of the task is understood. The instructor's expertise remains an important part of the assessment alongside the simulator data.
Mixed reality headsets integrated with fixed-base simulators provide a platform for training and assessment, reducing reliance on aircraft availability while also providing additional data from biosensors built into the headsets.
The Varjo XR-4 Focal Edition was selected for its high-resolution mixed reality, gaze-driven autofocus passthrough and integrated eye tracking. The requirements included high-resolution, high-refresh-rate passthrough cameras, a suitable field of view, accurate eye and movement tracking, low latency and a future-proof system. Eye-tracking quality was particularly important, as the data enables ELE to detect subtle, real-time changes in eye behaviour and identify different attentional modes and shifts in cognitive state during task performance.
Working with Ascent Flight Training as delivery partner, TACET-M was tested with trainees at RAF Cranwell during UKMFTS Phase 1 training. A cohort of students at Number 3 Flying Training School completed a training exercise using the mixed reality headset within a Grob Prefect T1 flight training device.
The 15-minute simulation included a series of progressively challenging events designed to assess airmanship qualities. A Qualified Flying Instructor managed the fixed-base simulator, controlling the timing of fault initiations and providing an expert view of the students' actions.
The exercise had three objectives: to gather feedback on the usability of mixed reality for training and assessment within a fixed-base simulator; to analyse eye-tracking data for gaze patterns associated with competency and refine the outputs in Cineon's advanced playback analysis software; and to collect psychophysiological data to support machine learning models assessing cognitive load, stress and mental fatigue. Head and hand movement data were also captured through the headset, alongside cockpit interactions.
The project demonstrated the potential for benefits to defence training organisations, instructors and trainees.
Eye tracking and AI can provide evidence-based insight into situational awareness, attention management, workload and stress, linking data to airmanship qualities that have traditionally been difficult to quantify. Identifying performance and cognitive challenges earlier in the training pipeline could allow instructors to intervene sooner and reduce the likelihood of costly late-stage attrition.
Data-driven insight can also reduce subjectivity and support more consistent assessment decisions across instructors and cohorts. Greater use of fixed-base simulators and XR technology could ease pressure on full flight simulators while maintaining assessment quality.
The architecture is also scalable to additional aircraft and scenarios, as well as other safety-critical domains, including unmanned systems and land- or sea-based operations.
Ellie Willis is at Cineon Training, pioneers in developing immersive technology and eye-tracking solutions for a wide spectrum of training and therapeutic applications. TACET-M was developed with Ascent Flight Training as delivery partner.
READ THE FULL TACET-M CASE STUDY ON CINEON.AI