The evolution of incident reporting, from pure statistics to nuanced insight
This article focuses on air safety occurrence reporting, and the provision of an improved Air Safety Information Management System (ASIMS).
ASIMS logo
Rarely did anyone attempt to grasp the wider implications of a particular incident for the future, or spot trends or patterns or read across issues to other aircraft. There was a corresponding lack of corporate memory as to related incidents which had occurred in the past…
The Nimrod Review: An independent review into the broader issues surrounding the loss of the RAF Nimrod MR2 Aircraft XV230 in Afghanistan in 2006, Charles Haddon-Cave QC, October 2009, P149
I make recommendations for a new safety culture…
Ibid, P569
A reporting culture is one of the cornerstones of an engaged safety culture. As such, the Military Aviation Authority (MAA) requires that all air safety occurrences are reported and managed, with appropriate action taken, see RA 1410: Occurrence reporting. As such, the MAA provides the Air Safety Information Management System (ASIMS) as both a ‘bottom-up’ and ‘top-down’ system: ASIMS is designed to help the defence aviation community report air safety occurrences in order that Aviation Duty Holders (ADH), see RA 1020: roles and responsibilities: ADH and ADH facing organisations, may actively manage air safety, monitoring and mitigating associated Risks to Life (RtL) within their defined areas of responsibility (AOR) to As Low As Reasonably Practicable (ALARP) and Tolerable. In addition, ASIMS is used by the MAA in the monitoring and oversight of the Defence Aviation Environment (DAE) as a whole and is one of the main tools used to establish the ‘risk picture’ across the DAE, enabling the MAA to practice risk based assurance.
As with any system of this kind, there are 2 components that make it a success, or failure. The design of the system itself, and the willingness and ability of personnel to fully engage with it, with the former often influencing the latter. Following the analysis of ASIMS report data-quality in 2013, it was recognised that, whilst there was a healthy reporting culture (with reporting increasing year-on-year), codification of why incidents occurred was inconsistent, hindering the MAA’s ability to gain a true picture of aviation safety risk held by Defence.
Whilst personnel understood the need for and were willing to report incidents, they either could not, or would not, complete the process. One major issue was the outcome-focussed taxonomy, which led reports to focus in on the outcome of an occurrence, not what led to that outcome. Another issue was allowing reports to be closed-down without mandating that investigation findings were declared.
Consequently, in April 2016 the MAA introduced a major ASIMS upgrade. From improved codification and taxonomy, to the introduction of hashtags, the upgrade has provided a step change in the ability to positively influence aviation safety through improved recording, trending and analysis of safety occurrences. The most significant changes are discussed below. However, the system remains reliant on the willingness of personnel to fully engage.
The stats
Currently, every location in which military aviation is conducted, and every unit which conducts military aviation, has access to ASIMS through DII. There are over 19,400 user accounts* belonging to 1,063 units registered at 212 stations** (including every aviation-capable ship). An average of 13,000 individual reports are raised each year, and there are over 225,000 unique reports stored on ASIMS (the vast majority of which pre-date the system).
*It should be noted that a user account is not required to raise an occurrence report within ASIMS.
**ASIMS statistics page dated 15 January 2018.
Investigations
The ASIMS v3 upgrade added the requirement to record the level of investigation being undertaken: Local Investigation (LI), Occurrence Safety Investigation (OSI) or Service Inquiry (SI). Investigators could devote as much or as little time to the investigation as required to satisfy the Aviation Duty Holder/Accountable Manager (ADH/AM) Review Group that the details of the occurrence have been accurately identified and recorded. If appropriate for the occurrence, an LI could be as simple as performing a desk-level investigation.
Findings
The introduction of a mandatory requirement to complete the Findings section and the associated recommendation details for the Cause and each Causal Factor was the single biggest element of the upgrade. The understanding of why an incident occurred was further enhanced by allowing multiple findings, which improved our overall analysis of occurrences.
This mandatory requirement to report findings, and the ability to report multiple findings has caused some additional work for the report investigators, but has added far greater value to each report submitted and thus the insight possible from analysis of ASIMS.
From 1 April 2015 to 31 March 2016, only 27% of reports recorded why an occurrence had happened. Following the upgrade, from 1 April 2016 to 31 March 2017 that figure had risen to 93%, however, there is still some way to go. Recent analysis (December 2017) has shown that 25% of Causal Factors are recorded as ‘Cause Undetermined’.
Further understanding of why this is the case is ongoing: defence aviation cannot claim to have good corporate memory, or an effective learning culture, if a quarter of occurrences are not codified in such a way as can be readily exploited.
Why record multiple findings?
A technical fault in flight may have resulted in a report-worthy safety incident. However, the subsequent investigation may establish that an associated maintenance activity had been both incorrectly completed and not properly supervised. The conclusion is that 3 elements led to the incident occurrence and each element requires investigation:
- why was the maintenance activity being carried out incorrectly?
- why was supervision inadequate?
- why did the technical fault in flight lead to an air safety incident?
Previously, only the top level would be recorded, i.e. the technical fault. However, by having the ability to add a separate finding for each phase of the occurrence, allowing each element to have their own outcome, cause and causal factors, it is possible to easily see which causal factor initiated which cause and subsequently led to the outcome.
In addition, this function enables improved analysis of the data; allowing deeper interrogation to establish not only multiple findings for a main outcome, but also multiple outcomes from a single finding.
Recommendations
Recommendations are formal and require ADH/AM approval. The upgrade introduced the requirement for the Investigator to make a recommendation against the cause and each causal factor for every finding. There are occasions when it is appropriate that no recommendations are made. These occasions must be positively acknowledged and recorded and as such, an associated free text field has been made available for this purpose.
Acceptable reasons for recording ‘No Recommendations’ are:
- the risk remains ALARP and tolerable, therefore no mitigation or action is required
- one recommendation mitigates multiple entries. Thus the mitigation is managed on one entry only and ‘Mitigated by Recommendation xxxx/xxx/Rx’ is entered in the ‘No recommendation’ field for the others
- the intention is to implement local mitigation/prevention action however, it does not change the current level of associated risk and so does not required ADH/AM approval. These actions can be implemented and manged outside of ASIMS
Hashtags
The hashtag functionality enables the tracking and management of local issues as well as analysis of global trends. Hashtags are created via the MAA ASIMS helpdesk. However, Occurrence Managers (OMs) are responsible for reviewing every report within their AOR for completeness and accuracy and it is their responsibility to add appropriate hashtags to a report once created. This action is vital for ensuring the reliability of hashtags when interrogating ASIMS at either a local, or a global level. An example of adding hashtags can be seen at Figure 1.
Figure 1: Adding Hashtags
ASIMS has proven that full, open, timely and accurate reporting of aviation occurrences is essential to the maintenance of aviation safety. The changes discussed here: the clarification of investigative levels, the mandatory reporting of findings and recommendations and the addition of hashtags to enable easier monitoring and analysis, all contribute to vastly improved data fidelity within the system.
Accurate reports, which include details of resulting investigations, allow important lessons to be identified and so play a major part in future accident prevention. ASIMS provides the mechanism for all reports to be passed to higher and other associated formations to form part of the wider situational awareness of the aviation safety risk picture, and for details of remedial or preventative actions to be recorded and promulgated through the appropriate specialist channels.
The success of all ASIMS activity is dependent upon User input and engagement, in a positive reporting culture. Continual upgrading of ASIMS to make it easier to use therefore directly influences the generation, maintenance and promotion of a positive reporting culture, which in turn influences aviation safety. For us to successfully interrogate ASIMS regarding both trends and culture, it is essential that the key fields are completed accurately. Fostering this behaviour is a key step on the path to a fully engaged Aviation safety culture, and we all need to play our part.
Further help and feedback
ASIMS continues to be developed and enhanced through the ASIMS Working Group and by suggestions from users. The Working Group is open to station flight safety teams and local occurrence managers and meets every 6 months, if you would like to get involved please contact the ASIMS helpdesk. If you have any suggestions or feedback on ASIMS please complete the online form, accessible from the ASIMS logon page, or contact the helpdesk via email [email protected] or by phone 030 679 84188.