Root Cause Analysis

Root Cause Analysis

Post by CatalinaNJB

When the root cause of an accident is determined to be that a pilot failed to act on something, whatever this failure might be, the root cause analysis becomes a simple task of making that statement. The root cause could be assigned by management or by the pilot and accepted by management in their root cause analysis. A pilot may state that: “I inadvertently failed to meet the pilot-in-command requirements for a tailwheel airplane…” or a statement of: “The failure of the pilot to maintain adequate clearance with the terrain during aerobatic maneuvering.” Other root cause human error failure could be that: “The pilot failed to maintain clearance with a fuel truck and failed to request a marshaller and wingwalkers”.  With any of these root cause statements accepted by management, the root cause analysis is completed, the file is closed and put on the shelf with comments of “no further action required. “Assigning root cause of failure not to perform a specific task, or two, is acceptable practice in the aviation industry from an airport, to the smallest air operator, a major carrier or a freight operator. 

The root cause of a nonconformance, a logistics interruption, an unplanned event or an undesirable event is the course of action selected at a decision point, or when arriving at the fork in the road within a system.

There is a lot of talk about an SMS system, but nobody explains what it is. The regulator gives system findings but does not identify the system that failed. Well, they might say that the SMS system failed to conform to regulatory requirements, but that doesn’t explain what system failed. Operators make corrective actions to fix the system that the regulatory had identified as failed, but do not identify what system they are correcting. SMS is at the verge of falling apart, since there were no tangible goals to aim for after the SMS itself had been implemented. The only goal was to implement SMS and no there are no goals anymore. SMS has become a circle of opportunities. As we all know, travelling the circle of opportunities only lead to the same place as we were before. The SMS is lacking visions, goals and root cause guidance that makes a difference.   

SMS fell out of the sky since there was no visions after implementation.

A root cause cannot be assigned to a task that was not performed. E.g. the pilot failed to follow procedures. If this is the true root cause, the conversation ends here. It is not possible to make improvements to the task itself when it wasn’t performed. A better way to state the root cause would be for an operator to state that the pilot acted willfully when omitting procedures during aircraft operations. 

Now we are cooking… at least there are events to work with for the root cause. This root cause must include that it was done willfully, or we are back to the first scenario that the pilot failed to follow procedures. Now that it has been established that the pilot acted willfully, there is an opportunity to change this willful behavior. Often, this is managed by firing of the pilot, or placing the person on unpaid leave for a day, week, or two. At least the operator feels that they are doing the right thing when punishing someone who willfully caused an incident. The only thing for the operator has left to do is to write a letter to the pilot’s file, or cause of firing, that the pilot acted willfully when causing the incident. The only barrier to this letter is that most operators reject the option to make it official. 

So, if an operator rejects to publicly and officially state that a pilot acted willfully, there is no possible way to address this root cause by blaming the pilot. In addition, the operator must then also assess other areas of operations for willful behavior that could have caused this system failure.  There are no reasons to include any of the standard phrases of willful misconduct, reckless behavior, criminal intent or illegal substance use, since none of these are job performance criteria and are not performed as a part of a Safety Management System.  Job performance criteria within an SMS system are safety critical areas and safety critical functions. 

Assigning wrong root causes over time chips away a perfect Safety Management System

The only time when a true root cause analysis can be conducted, and a real corrective action produced, is when an operator accepts accountability and the fact that an omission of something is not a root cause. With acceptance of accountability, an operator needs to exit their endless travel in the circle of opportunities and make real system changes. A root cause must be assigned to the system and not to a process, procedure, practice or expectation. The simple reason for this is that systems that are built with inherent flaws do not improve by making changes on any level below the system level itself. E.g. If the system doesn’t require a risk management analysis, there is no tools available for the operator to comprehend a latent hazard in the process. 

Let’s for a make up a virtual scenario where the finding was that a pilot failed to follow procedure and hit a fuel truck when taxiing. This is a small operator, operating out of a small airport and with a designated parking sport for loading and unloading freight. One day there is an event with 400 arrivals at this airport that normally has 50 or fewer movements per day. This event would last for one week.  For this to work out for these new airport guests, the airplane is moved to the back and behind the hangars for all business jets to be parked in a preferred location.  The pilot of the must taxi behind a hangar, make a 90 turn and then taxi between a hangar and the business jets. The pilot has 7 FT clearance to business jets. For several days the pilot taxied this route without any issues. Then, one day a fuel truck is parked next to the hangar and the clearance is reduced to 2 FT. The pilot observes the parked fuel truck coming around the corner, but the left wing strikes it. An immediate response from the pilot is that it was pilot’s fault, with the same response from the operator. Problem solved… or maybe not. 

A root cause analysis would show that there was a system failure of the logistics system, in that several parties missed their opportunity to exercise their accountability to safety and to conduct their own safety risk assessment of the changes. The airport, operator, freight customer and fuel operator all assumed this to be normal operations and that mitigation would not be required.  Highways do a safer job than aviation, in that they mitigate the changes with traffic cones, speed and other accountable actions. The aviation industry has missed their opportunities of being accountable to safety. These missed opportunities were contributing factors to the incident. 

This logistics system includes the air operator safety office, accepting the taxi route without mitigation, the airport authority which had an incomplete assessment of the taxi route form the new parking area to ensure that their customers had taxi clearance, the airport authority provided incomplete communication to flight crew of business jets that there is a frequent use taxi route in front of parked aircraft, the marshaller of business jets for parking had incomplete training in marshalling business jets where to park for taxi clearance, the fuel vendor provided incomplete training to fuel truck operator where to park for taxi clearance and the system of the freight operator missed an opportunity to mitigate the back-alley parking and standard departure times for non-standard operating conditions with a longer and complex taxi route.  

It was the system itself that failed by established expectations of normal operations without mitigation or elimination of hazards with an abnormal. A root cause cannot be that a pilot failed to perform. A root cause is tagged to the system level for the parties involved to discover the facts and effects of abnormal taxi operations and to mitigate or eliminate these facts. Somewhere there is someone who owns that root cause. When there is a comprehension of the true root cause an operator can exit their travel in the circle of opportunities and discover the benefits of SMS with continuous improvement of return on investment. In addition, a causal effect of temperature was overlooked with the temperature in the cockpit being 100°F, or 38°C. 

CatalinaNJB

DUCK BOAT TRAGEDY.....A CASE FOR SMS.

DUCK BOAT TRAGEDY.....A CASE FOR SMS.

On July 19, A duck boat sightseeing vessel capsized and sunk on Table Rock Lake in Branson Missouri after a thunderstorm created turbulent waters and high winds. 17 people died. The duck boat industry is one of the few public transportation services that regulators do not require Safety Management Systems, SMS. If Ripley Entertainment, the company that owns the duck boat service, had SMS, this tragedy could have been avoided. Here is the evidence why: 

Duck boats in Branson Missouri 

SMS requires hazard identification, incident analysis and risk assessment. In this case here are the facts. 

Ignoring Risk Factor Data

A witness’s video of the Branson duck boat just before it capsized suggests that its flexible plastic windows might have been closed and could have trapped passengers as the hybrid boat-truck went down. It does not show passengers jumping clear. In 1999 the NTSB recommended that all Duck boat "...canopies be removed and mandatory use life vests for each passenger." NTSB went on to suggest that the canopies could entrap passengers if the boat sank. This was the major cause of deaths in the Branson tragedy. Duck boat companies chose not to take this risk factor seriously.

"There is always a trail of things going out-of-control before every accident." 

The Coast Guard prohibited the vessel from operating from January 2015 to April 2015, but the report does not state a reason other than "hazardous/unsafe condition." Another report from February 2015 cited leakage in a wheel well caused by sealant failure.

The owner of an inspection service in the St. Louis area said he issued a written report in August 2017 to the Branson duck boat operator, Ripley Entertainment, after inspecting two dozen boats. In the report, Steve Paul of Test Drive Technologies explained that the vessels' engines — and pumps that remove water from their hulls — might fail in inclement weather

Duck boats have a history of fatal and less serious accidents. It's often led to criticism about their design and use as tourist vehicles. For example, 13 people died after a duck boat sank on Lake Hamilton near Hot Springs, Arkansas in 1999.

The owner of the Branson Duck boat operation admits that the boat should have NOT been in the water. Andrew Duffy is an attorney whose firm represented victims of a deadly 2010 duck boat crash in Philadelphia. He and other lawyers with his firm have called duck boats "death traps" and called for them to be banned. They have specifically pointed to the canopies as problematic.

In 2015, five college students were killed and 69 others were injured in Seattle after a duck boat collided with a bus. Ride the Ducks International of Branson, which operated the Seattle boats, was fined $500,000.

In 2010, a barge plowed into a duck boat that had stalled in the Delaware River in Philadelphia.Two of the 37 people on board drowned. They were 16- and 20-year-old Hungarians visiting the United States through a church exchange program.

All these facts lead to Root Cause Analysis and subsequent Corrective Actions. Unfortunately the Corrective Action suggested by the NTSB were not implemented. 

Using the Deming “Interaction of Forces,” lets examine the facts.

The Machine....additional risk factors.....ignored. 

Duck boat design is inherently unsafe

Duck boats were designed for military use during WWII and never considered for private or commercial use. Here are some of the problems with Duck boats:

1. Tend to be top heavy which leads to easy capsizing. 

2. They take on water easily and require a fairly powerful bilge system.

3. Not designed to have ridged canopies or windows. 

In addition. In August 2017, mechanical inspector Steven Paul saw a glaring problem when he examined the duck boat. “One of the most prominent things I found was the exhaust being in front of the vessel, which -- according to Department of Transportation standards -- would not pass regulation," he told CNN's "New Day" on Monday. "The exhaust has to come out past the passenger compartment.” When he saw footage of the boat sinking, Paul said "with the exhaust coming out the front and going down below the water line, the waves are obviously pushing water up in that exhaust." If water gets in the exhaust, he said, "the engine is eventually going to stop."

Methods...NOT following Procedures.

The amphibious vessel changed the route it took on Thursday, Missouri Attorney General Josh Hawley said Saturday. The boat capsized as a storm and high winds lashed the lake. Investigators want to know "when did the driver and (captain) of this vessel know about this storm forecast? When did they decide to alter the route of the boat?" he said. "Because they did alter the route of the boat. When did they decide that? Why?"

Environment ......Risk Factors ignored.

Severe storms were quickly approaching

The first storm warnings came the morning of the incident about 11:20 a.m., when the National Weather Service in Springfield issued a severe thunderstorm watch, which was scheduled to last until 9 p.m. A former president of the American Meteorological Society said the radar "clearly showed a very large complex of storms approaching the lake." Specifically, Branson straddles the line separating Stone and Taney counties.  A more serious alert, a severe thunderstorm warning, was issued for Stone County at 6:07 p.m., and was extended to including neighboring Taney County at 6:32 p.m., about the same time the duck boat is believed to have entered the water. That's also about the time the storm struck Table Rock Lake.

People and Methods…lack of Policy, Procedures and Training. 

Interviews with survivors indicate that there was mention of life jackets and where they were located. The decision to go ahead even though the wind had picked up to 45 miles an hour, indicated either a lack of policy or ignoring of policy. If the company had a proper safety risk profile, all the risk factors should have been mitigated. We will, of course, find out more as the investigation continues. 

Conclusion

Programs are created and put into place for a reason. SMS was implemented as a result of the Exxon Valdez accident. Unfortunately we are very good a reacting when a tragedy takes place. Hindsight is always 2020. In the world of Safety and Quality, we must study data. Not to merely record it but, to use it to develop and improve Safety and Quality. I am sure now, there will be changes in the regulation of this industry. We in the Aviation and other industries need to be reminded that what we do in SMS has life saving implications and we must never wane from our diligence in promoting and continuously improving it. 

Author: Mr. Dennis Taboada M.eng, CQE,CQM

dtiatlanta.com/workshops 

How To Build A Safety Policy

How To Build A Safety Policy

Post by CatalinaNJB

A Safety Policy in an effective Safety Management System is a forward-looking policy, the policy is organizational guidance maternal for operational policies and processes, a road map, a vision of an end-result with increased profit margin or higher return on investment and a plan to prepare each flight or operations at an airport to be completed without an incident. One of the questions a safety policy must answer is the reason for implementing a safety management system when an operator is operating safely without incidents, without a safety policy and without a Safety Management System. A safety policy is not about the operator’s commitment of safety, or what an organization is committed to in regards to the safety of operations, but rather how the organization is committed to make safety work. 

If the Safety Policy does not build the future, it doesn’t build anything

Building a safety policy needs a blueprint just like the construction of a building or building a road. Without a safety policy blueprint, or directions, the policy could become anything but a safety policy. A safety policy blueprint is to establish directions for the policy, directions for the accountable executive and all personnel. A blueprint is to avoid making the policy a safety-first priority policy, which could take the policy all over the map without directions as long as it was determined to be safe by someone’s opinion. If a safety policy states that safety must be first priority it must be accompanied by a safety risk level assessment of how safety always will be a priority. Safety cannot be a priority, since there will always be an inherited risk in aviation. When safety is an organizational priority operations becomes incidental to safety, or takes second place. In an organization where safety is priority there is no activity, since there are risks involved at the moment an aircraft is in motion or at the moment when an airport operator is on the field. If there is a risk present, safety is no longer the priority. However, when there are acceptable safety risks levels defined, then safety is paramount and safety can be achieved in operations. 

The first building block in a safety policy is to decide and commit to what the purpose of the policy is. A safety policy that is paramount is a policy with a purpose, it is a practical policy that personnel comprehend, it is a vision of the future and a vision of where the organization is headed, and a commitment to safety with zero tolerance to compromise safety. The base and first building block in a safety policy is to make safety paramount where there zero tolerance to compromise safety, or acceptable short-cuts. 

The second building block of a safety policy is to establish safety critical areas and safety critical functions. Without safety critical areas in the policy there are no criteria established to develop safety goals and objectives. Safety critical areas are your tools as an operator to establish measurable goals, and to develop steps of objectives to reach these goals. With step one and two completed, we know what direction we are headed and where we are going. 

The third building block of a safety policy is to get everyone onboard, or on the same page. It is a comprehensive task to get everyone onboard and to agree. That someone disagree with the SMS process does not imply that they are against safety. As a matter of fact, a Safety Management System that does not allow for personnel to disagree or question the system itself, has become a failed safety management system. Any safety opinion given, or process decisions made, must be backed up with documented data and facts. That there are individuals who disagree with is safety decisions an the SMS, is because the Director of Safety or Accountable Executive not been able to deliver, backed up, or communicate their safety improvements with reasons, facts or data. There will also be times in an organization where management is incorrect and personnel in the field have the better and safer solutions. 

Leadership in safety management is to guide with clear directions

Safety Management System requires leadership. This leadership is not only for safety improvements, but leadership when implementing policies that will be acceptable to all personnel. SMS is not a paperwork program to document safety records, but a live program, and a program in motion. SMS is a program where hazard, incident and accident reports are submitted for assessment, investigation and risk analysis to improve safety in operations. 

One option to establish a reason for personnel to buy the SMS is to include a confidential reporting system for all personnel. A confidential reporting system is not an anonymous reporting system, but a reporting system with limited access at the receiving end. This system is not to mislead operational management, but a system made available to address the issue as a safety issue rather than as an operational issue. In addition, operational management has agreed on, by the implementation of SMS, that none of these reports are available to use for punitive actions against a person. The question many asks is if this is a “get out of jail free” card, or where is the line in the sand drawn. 

It is oversimplified to state that there is no disciplinary action unless the act was illegal activity, negligence, willful misconduct. When the bar for disciplinary action is set at illegal activity, negligence, willful misconduct, any report submitted will be below that bar, or will not cross the line drawn in the sand. By implementing these activities in the safety policy, the policy itself becomes an obstruction to safety. Disciplinary actions may be the only option, even if the action was not illegal activity, negligence, willful misconduct. For a moment, let’s assume that an aircraft taxiing at a higher than normal taxi speed, and the flight crew is pre-occupied with after landing checks. During the taxi the aircraft strikes a taxiway edge lights, misses the turn in the taxiway, then taxi across the island of grass and enter the taxiway on the other side. When the reports comes in to safety department, the report states that the taxi was slow, flight crew was blinded by another aircraft and by the time they realized they were off the taxiway the aircraft was entering back onto the taxi way on the side. When reading this report, the operator has no way of knowing if this was either an illegal activity, negligence or willful misconduct. An operator may assume that it was, but it is not an illegal activity, negligence or willful misconduct unless the notice of suspension or termination states this specifically. There are very few operators that would make such bold statements in a termination report.  By raising the bar to this level, there is no room for safety improvements. In the virtual example above there is lots of room for disciplinary actions when the operator, during the investigation, discovers that the flight crew did not make true statements. 

On the other side, when the bar for unacceptable activities is set at the level of Safety Critical Areas and Safety Critical Functions, there are opportunities for safety improvements. Pilots may be offered refresher training, stress management training, operational policies review training or other training to improve skills and accept safety as being paramount. By defining safety critical areas, flight crews, maintenance crews and others know what these areas are and they may establish their own goals in their tasks of duties to improve safety. 

These are the three first steps, or building blocks of an effective Safety Policy. Without safety being paramount, there is no strategy for safety in operations. Without safety critical areas there are no defined goals for safety in operations. Without personnel accepting and participating in an organizational safety management system, there are no human resources available for safety in operations.  

CatalinaNJB

So..Your SMS is not Continuously Improving.....Chill Out!

So..Your SMS is not Continuously Improving.....Chill Out!

The Greyhound is the fastest dog in the world, but they need to chill out too. 

After working with dozens of different enterprises, implementing Safety Management Systems, SMS, and Quality Assurance Programs,QAP, I have discovered several reasons why the SMS/QA stops working after a while. 

The following are some reasons you need to examine in your own company:

Loss of Interest in the Regularly Scheduled Safety Meetings:

When an SMS is first implemented, everyone is on board and enthusistic about the Process so the weekly safety meeting are well attended. Over time people begin to lose interest in the weekly safety meetings. Managers start falling back into their pre-sms modes. It is imperative that upper management stress the importance of the participation in the weekly meetings. These meetings are where we roll up our sleeves and really look at the reports and analyze root cause. The result of the weekly meetings should be corrective action plans that actually provide the action to continuous improvement.

The Deming Cycle for Continuous Improvement

Hazard/Incident reports as well as audits results drive the Risk assessment, Root Cause Analysis and Corrective Action Plans that are results of the weekly meeting. 

This is the engine of the SMS. See https://youtu.be/F3GuPJrB7_4.

Reduction of DATA Collection

There are some great SMS Software programs. I have experienced SMS Pro and Vortex and found these to be excellent programs. But, "Garbage in - Garbage out." In order to maintain "CONTROL" we need to consistently measure our processes to provide quality data. Yearly audits just don't cut it. 

A consistent audit verification program provides excellent data to enter into the SMS engine analysis system. The Hazard and Incident reporting system must be encouraged by middle and upper management. A good SMS Software program will keep track of the progress of each report, communicate to key people, and help the SMS manager to control the flow and completion process of each report. 

Poor Corrective Action Plan implementation

Once the Causal factors have been determined through a robust Root Cause Analysis, RCA, it is important that the corrective action be "realistic" and supported by upper management. Of all the impediments to a successful continuous improvement program, lack of follow-through on the corrective action plan is the highest.  All CAPs should be monitored and checked by the Quality Assurance department to make sure that what we said we would do is actually what we did.

Turn Over

According to the United States Small Business Administration,SBA, "..employee turn-over is the highest cost to a small business." Because of this reason, it is important that your enterprise have a "system" of control. All job functions throughout your company must be controlled through robust procedures. New employees need to be taught to depend on procedures to perform functions. Do things by the book. 

Dr. Joseph Juran, Founder of the famous Juran Institute 

Dr. Joseph Juran, " all processes should be so well documented that you could replace an entire department with new employees and you should be able to make the same product with the same quality..." The "system" approach assures that processes are not people dependent. 

In addition, new employees must hired and trained with the same or greater commitment to the SMS. It is the job of management to permeate the organization with a safety oriented culture.

Communication breaks down

Its so easy to fall back into old habits. Managers get absorbed into their day-to-day routines. Communication begins to slow down between the SMS teams and we begin to forget the importance of robust safety communications between departments. Management needs to continually emphasis to the workforce the importance of reporting incidents and hazards. 

Entropy

Entropy defined, "lack of order or predictability; gradual decline into disorder." How do we fight entropy in our system?  The secret is constant renewal of Goals and Objective. As we gain control of an area or process, identified as high risk areas, we need to replace the goals and objectives with new ones that come from our Safety Risk Profile and Hazard Registry, ( see https://youtu.be/WwQuMtDTmJ4 goals and objective from your Safety Risk Profile.) 

Another tool to fight entropy is training.  There should be constant and robust training offered throughout the organization. Training not only equips people to do a job, but it is also a very effective tool of communication. Many policies can be reinforced in the training courses. 

Entropy can be fought off through recognizing success stories within the organization. Remember we are collecting data through reports and audit results. Well, do we recognize areas that pass their audits. SMS and Quality Assurance is often looked at as a system that recognizes failure. We should applaud success as well. 

In my 30+ years implementing Quality and Safety Management Systems I can testify to the fact that if we remain diligent, we can achieve a true process of continous improvement. We have many success stories to back up this claim. It is up to management to recognize this and support it in order to make the organization successful. 

The SMS Memory Jogger II 

Dennis Taboada, M.eng.,CQE,CQM                                                                                                 CEO, President DTI Training Consortium Atlanta USA and DTI Training Canada Ltd. Richmond BC Canada

Suggested Reading: "The SMS Memory Jogger II." Available at Publisher GOAL/QPC , goalqpc.com search SMS. Also, Amazon.com 

dtiatlanta.com

dtitrainingcanada.ca












When SMS Stays On The Shelf

When SMS Stays On The Shelf

Post by CatalinaNJB

Products that do not generate revenue greater that the cost of the shelf in a supermarket are removed from the shelf. If a product is required to be carried to conform to regulatory requirements, the product may be sitting on the shelf untouched as a dust collector. SMS is in the same boat since it was introduced as a regulatory requirement. As a regulatory requirement the SMS might just sit on the shelf and collect dust since an enterprise keeps it there without a return on investment. That the SMS is put on the shelf does not imply that the SMS processes are ignored, that hazard and incident reports are not processed, that root cause analysis are not conducted or that personnel are not trained in SMS. All this is happening while the SMS is sitting on the shelf. An enterprise that has placed the SMS on the shelf is an organization that is not able to identify and describe what the SMS does for the operator. SMS is intended to be a program for improved safety in aviation. 

If the implementation of a safety program does not directly contribute to safety improvement, the program has become nothing else but a paper-shuffling task and labor-intensive ticking checkboxes. The one reason SMS is kept on the shelf is due to regulatory requirement by the Aviation Authority (Government). The SMS stays on the shelf even if it does not serve customer service or produce what the cost of the shelf is. This cos is not to produce a positive return of investment, or cash flow, but also with reference to return of safety-investment or continuous improvements. 

SMS is a system where discrepancies are clarified

The safety management system has been described as an umbrella of the operations. The purpose of an umbrella is to shield, protect, buffer, shelter or safeguard. If an SMS operating as an umbrella the enterprise has a system in place to ensure a safe flight for all passengers and freight. When the SMS is an umbrella, the system functions and protects without any further actions. It’s a wonderful system. Or, maybe not…a system that protects is not a fail-free system and may give a false impression that safety is paramount in existing operations, while the opposite could be the actual fact. 

If the SMS is controlled by the umbrella, an enterprise, operational systems must unconditionally follow the umbrella wherever it goes. This is the old way of looking at safety, where safety is the “big-bad-wolf” who directs operations in all different safety directions. Or, on the other hand, if operations under the umbrella are in control of where the umbrella travels, then safety decisions rests with the operations, which is the NextGen of aviation safety. The purpose of the umbrella is oversight of safety processes, and not the safety in operations itself. When SMS takes on this role, it becomes the Quality Assurance program. 

Generally speaking there are two ways to operate with a safety management system. One way is to apply SMS as a duplicate control system to discover if there are any errors in the first audit. This is a method when the SMS system conducts audit of operations and validates or invalidate the current result. It becomes a control system of the first or prior checks conducted. If both results come up with the same conclusion it is assumed that the first result also was correct. If the second result is different than the first result, it is assumed that the first result was incorrect and a corrective action plan (CAP) must be implemented. This CAP is not a process or system CAP, or a CAP of how the task is performed, but a CAP of the failure itself. If the issue was a light bulb, this CAP would replace the burnt out light bulb, but not address the issue of why a newly replaced light bulb burned out. This is when safety is covered under the umbrella without directions and difficult to define how SMS improves safety. 

1)     Time how long it takes to spot the 6 differences;

2)     When differences are spotted, time it again how long it takes to spot the 6 differences;

3)     This is SMS – to spot the differences and it only becomes more effective with training.

The other method of applying SMS within the organization is to apply SMS as the oversight body of safety processes. Operations establish processes for acceptable safety levels, and the SMS assess these processes in an oversight capacity. The SMS process is to audit all operational processes by random sampling of selected operational audits. If there are only a handful of audits conducted the process would audit all the samples. As a system audit of operational safety checks, this audit would establish if the organization is performing safety tasks to acceptable risk levels established by organizational policies. E.g. If there is an annual company flight-check scheduled, the audit would discover if these flight checks were conducted. The flight check result of the audit is irrelevant to a system audit, since this task, or CAP, was already performed by flight operations and based on the flight check result.  If the system audit found that the flight checks were conducted to a 95% confidence level of all operational checks, then the system in in statistical control. Should the process be out of statistical control, then a CAP must be initiated to move the line to an acceptable safety risk level. 

As a Quality Assurance system the SMS has an opportunity to change incompetent processes. However, if keeping SMS on the shelf for reasons of regulatory requirement only, the SMS in itself becomes a hazard to aviation by its travel in the wrong direction at the fork in the road.     

CatalinaNJB