Risk Assessments Blunders

Risk Assessments Blunders

By Catalina9

A safety management system in aviation is a reactive system to data collected, but a proactive to the degree of comprehension of data received. The aviation industry, being airports and airlines regularly collects data for analysis. Data collection does not in itself prevent incidents from occurring unless data is comprehended with initiatives or corrective action plans. If data is not comprehended operations may continue without corrective action plans. Often, data is ignored if the type of incident which this data would generate is without prior exposure. After high publicity and fatal incidents comments for explanation is often this was the first time this type of incident happened, which in itself identifies a lower data comprehension level. 

Anything is possible with systems comprehension

When an enterprise has decided that it is impossible to predict 100% of special cause variations 100% of the times, that is when SMS becomes the most valuable tools for proactive CAPs. This is done by risk assessments. Risk assessments could take a few minutes to several days, weeks or months to complete. One thing all risk assessments have in common is to establish a risk level or risk rating.

Risk analysis is in the aviation industry are done in several different ways. A bush-pilot does ongoing risk analysis by the minute while flying. In larger organizations, someone other than the flight crew are conducting risk analysis and communicating their decisions to the flight crew. After a risk assessment is completed it is filed for never to be seen or reviewed again. In a new SMS world, an associated risk assessment needs to be reviewed after an incident. An enterprise needs to pick up their old and closed risk assessment to find out how their data caused an incident to happen. Instead, it might be tempting to find fault with a person who failed to recall or complete certain tasks.

Risk assessments analysis is to stay balanced and on track.

The first step after any incidents is to pick up an old risk assessment for an analysis. When reviewing a closed risk assessment, it must be reviewed backwards. The review begins at the end or after the incident and tracks backwards until reaching the-fork-in-the-road where a different decision would have made a change in outcome.

Risk assessments are not just to establish a risk level, but also to apply as a reactive action to the risk assessment itself after an incident.

A standard risk analysis is based on a 5x5 risk level matrix. A risk matrix takes into account likelihood and severity. However, the descriptor of the likelihood is based on opinion and not fact or data. With only one opinion-based criteria, the whole risk assessment falls apart. When SMS in its infant years got a bad reputation was because risk analyses were based on someone’s opinion and not data.

The five descriptions of likelihood are: Improbable-Rare, Remote-Unlikely, Occasional, Probable-Likely and Frequent-Almost Certain. None of these definitions are making factual statements, but rather opinions of likelihood. The risk matrix definition also includes statistical probabilities, which also is a probability of the opinion itself and not of the hazard. When applying the statistical probability to a hazard several years ago, the answer was that an airport of a certain size and complexity would have to operate for 100 years or more before the hazard would affect its operations. What is not possible to predict is if a hazard will affect operations the first year or in the last of the one hundred years.

The time of a risk assessment runs out sooner or later.

After an initial risk rating is established, subsequent risk assessments may be broker down to year, months, days, time of day or hours. E.g. a risk rating for inflight icing has a lower rating (less chance that icing will occur) during the summer months and can therefore not be analyzed as equal during a 12-month period. This same concept applies to any risk assessment and an enterprise must define safety critical areas and common cause variations of hazards. Often these variables are not identified since thy are not as obvious as winter and summer seasons.

In a risk analysis matrix likelihood must be analyzed in intervals of times since hazard affected operations. If an event occurs for the first time, then the time interval equals zero second between intervals. Without data available to analyze intervals a fist time accident must be analyzed as zero seconds. The incident would also be defined as a special cause variation since it is not a part of a regular process. As with any special cause variations a root cause analysis is required.

Likelihood needs to be defined as “interval between”. At one end of the scale a likelihood that is inconceivable is when times between intervals are imaginary, theoretical, virtual, or fictional. At the other end of the scale is a time between intervals that are reliable and dependable. Without comprehending data collected anyone could fall into the trap of applying a possibility scenario to the lowest level of likelihood and apply it to a reliable timeframe.

Risk assessment blunders are associated with data comprehension, applying possibilities scenarios as opposed to probability and accepting the associated risk assessment as true after an incident.

Catalina9