SMS Assessments

SMS Assessments
By Catalina9

Over the years, the Regulator has conducted SMS Assessments, Program Validation Inspections (PVI) and Process Inspections. In addition, they have conducted several nonstandard Compliance Inspections, which is an old terminology for any type of inspections. The purpose of inspections is to establish an operator’s level of regulatory compliance. Aviation safety inspectors may apply subjectively portions of their Staff Instruction (SI). The SI is not available to the public by searching databases and it could be a conglomerate task for an enterprise to ensure compliance with expected processes. This does not imply that an enterprise does not conform to regulatory compliance, but that aviation inspectors, who don’t comprehend airline or airport operations, may have unreasonable expectations of what to expect of the processes.   

Nonstandard inspections or audits skews Quality Control.

When nonstandard inspections are conducted it becomes extremely difficult for an enterprise to comprehend what aviation safety inspectors actually are inspecting for. Other inspection methods are to go directly to audit findings and inspect if the finding was corrected. The Regulator’s mandate is to conduct surveillance of an enterprise, or close observations. An observation is always in plain view, but becomes an investigation when inspectors are following a lead from an observation.    

Surveillance activities fall into two broad levels: systems and process level. Systems level surveillance is conducted using assessments and/or Program Validation Inspections. During these activities, a surveillance team reviews an enterprise's systems to develop an understanding of how they comply with regulatory requirements. The point of entry becomes the critical factor of how an aviation safety inspector proceed with the inspection. If the inspection is an Assessment, the point of entry is the Safety Management System itself. For a PVI the point of entry is the Quality Assurance Program. Documentation reviewed by inspectors are publications that provide process, policy and procedures. Internally generated records are reviewed on-site as part of the sampling plan.

Process level surveillance is conducted using Process Inspections. During the PI, inspectors review an Enterprise’s processes and traces processes backwards. When inspectors are tracing the process backwards, gaps in the process are discovered and origination of the input defined. A PI also defines the fork in the road or the place where the process may lead back to the intended input or to a totally separate input. When inspectors are tracing the process forward, they must apply virtual scenarios to proceed when arriving at the fork in the road, which leads to incorrect findings. Simplified, when moving forward any turns are justified with an explanation, or emotions, while tracing backwards any turns are justified by facts.   

Issuing SMS system findings is wasting valuable time .when based on PVI

The only time a finding is an SMS system finding pursuant to Canadian Aviation Regulations 107.02, is when all systems pursuant to 107.03 are in noncompliance with the objectives. CAR 107.03 lists nine SMS subsystems that must comply with their objectives. These systems are the Safety Policy System, Goal Achievement System, Hazard System, Training System, Reporting System, Process System, Quality Assurance Program System, Audit System and Other Regulations System. The Regulator frequently issues SMS system findings, while the finding should be a process finding of one of the nine subsystems. This is spelled out in the Staff Instruction’s point of entry.  

Since SMS was implemented in Canada there has been much opposition to SMS from the public and inspectors. Multiple surveys were conducted, which most showed that SMS was more of a hazard to aviation than improvement of safety. Questions in some of these surveys were designed to skew the outcome to discredit SMS. It was easier for inspectors to reject than to put effort into learning about SMS. SMS is in concept very simple and practical. One simple step both the Regulator and Enterprises conducting internal audits must take when assessing their SMS, is to develop a random sampling plan of their operational regulatory requirements and operational standards. In addition, both the Regulator and Enterprises must inspect and not investigate to arrive at a correct result.

      

Catalina9