Is Aviation Risk Assessment really “VooDoo?”

Is Aviation Risk Assessment really “VooDoo?”

By Dennis Taboada, M.eng.,CQE,CQM

Coming from a strictly Quality Assurance, QA, background, both education and vocation, I have been engrained with the concept of “OBJECTIVITY!” All analysis and action must be based on DATA and ACCEPT/REJECT criteria. When I was first introduced to the process of Risk Assessment, RA, using a risk matrix, my reaction was, “Are you kidding me?”  The Risk Assessment was mostly based on OPINION and SUBJECTIVITY of the team conducting the Risk Assessment session.  The result of the Risk Assessment was a numerical RISK RATING based on the OPINION of a safety committee. Depending on the composition of the Risk TEAM, the Risk Rating can change to meet the objective of the company. I say that this process itself is a “RISK” to SAFETY!  Hey, at least the process is using the QA concept of  TEAMING!  

Risk Assessments are Subjective

In 2003, Sol and I were contracted by NASA at Kennedy Space Center to Design, Develop and Deploy Quality Assurance Training for all NASA Safety and Mission Assurance personnel. This was the first time I was introduced to the NASA Goddard Risk Program. 

Dennis and Sol Taboada Contracted by NASA at Kennedy Space Center

The NASA Goddard Space Flight Center (GSFC) Risk Assessment Tool is based on the Quality Assurance concept known a Failure Mode, Effects, and Critical Analysis, (FMECA).  FMECA was originally developed in the 1940s by the U.S military, which published MIL–P–1629 in 1949. By the early 1960s, contractors for the U.S. National Aeronautics and Space Administration (NASA) were using variations of FMECA under its Risk Assessment Program.

When Sol and I were first contracted by Transport Canada to help design and develop QA training for the CAR 107 Safety Management System, SMS, deployment to 705 carriers, I mentioned FMECA as a more “objective” means of risk analysis.  The idea was promptly rejected, why?  I touched upon the “3rd Rail” of risk in Aviation:  MONEY!  The Goddard Risk Matrix includes “cost” and “scheduling” as factors!  No way could cost be considered as part of “SAFETY” in aviation! 

Another division of the Quality Assurance science is something called, “Cost of Quality.”  That was a misnomer because the actual process determines the cost of “NON-QUALITY.”  Why can’t we have the “COST OF NON-SAFETY” in SMS?  This is a discussion for future articles. 

Is it possible to use the FMECA concepts to help “Objectivize” Risk Assessment even without cost factors? The answer is YES! We can use the actuarial Failure rate formula:

lambda = ln( 1 – pf )/- time

Where lambda represents the density of occurrences within a time interval, as modelled by the Poisson distribution. We can us lambda as “Failure Rate.” 

Where is our DATA for calculation coming from?

In Aviation Risk Management, we are required to have a “HAZARD REGISTER” in which we categorize the Hazards and Incidents. By simple modification of the Hazard Register categorization cells to include a probability calculation, we can now obtain objective probability numbers that can be introduced into our every day Risk Assessments through a modified Risk Matrix. Let’s face it, We use the Risk Matrix to create our Safety Risk Profile that drive our Safety Goals and Objectives.  Then why can’t we use the Hazard Registry to provide Quantifiable information for our Risk Matrix? At least this would make the PROBABILTY side of the RISK MATRIX more OBJECTIVE. The SEVERITY side can also use the “MODE” and “EFFECT” components of the FMECA to quantify the “Effects” of an incident or hazard based on history. 

Yes I know this is going to raise questions in the AVIATION Safety world! Why can’t we bring together the science of Quality Assurance into the world of Safety Management Systems to provide better “Processes” that can actually make Aviation Safer and more efficient? Without the Voodoo!

Dennis Taboada, M.eng.,CQE,CQM

For more information:  Request Training from  dtiquality.com.