How to Audit SMS

How to Audit SMS

By Catalina9

Conventional wisdom of how to audit the Safety Management System (SMS) is to generate an audit checklist based on regulatory requirements for an SMS, and develop expectations, or processes, in a checklist form to determine level of regulatory compliance. There are several itemized expectations for an SMS enterprise to audit every single aspect of operations for compliance. Auditing by expectations does not paint a true picture of an SMS enterprise level of compliance since an expectation audit does not audit for reliability. 

Research and development is the responsibility of an AE.

An airline or airport may be required to comply with hundreds of regulations in addition to just as many operational standards. One regulation may be compliant by applying several different operational methods, or expectations, which may be interpreted differently by inspectors, auditors or organizational management. When expectations are applied to an SMS audit, all operators are grouped into one expectation and that one-fits-all. Auditing by expectations is a hazard in itself, since an SMS enterprise may change their operational behavior to please the inspector’s or auditor’s checkbox, rather than trusting their own operational judgement. Auditing by expectation is also an avenue to group safety with ratings. A high rating number becomes equal to a high, or superior, level of safety. As an operational oversight system SMS paints a picture of results, or process outputs, and not of a predetermined input. A shopping list contains expectations or inputs, and when used correctly each item is checked off, but the condition, output, or quality of each item is unknown until after the shopping is done.  

The first level of audit of an SMS enterprise is to audit for scalability, or size and complexity. There is a regulatory requirement that a safety management system shall be adapted to the size, nature and complexity of the operations, activities, hazards and risks associated with the operations. Humans are great at making simple tasks complex, or even unmanageable. An unmanageable SMS is a system where hazards to operations are unknown. In an unmanageable SMS, or where an SMS is scaled beyond their operational needs, operations tend to drift towards informal, and simplified processes. An SMS workload is not the SMS itself, but research and develop to scale down systems to size and complexity for regulatory compliance, for safety in operations compliance, for compliance with operational needs and compliance with the SMS policy. An SMS system should be scaled to a level where it can be explained in just a few words. If an SMS enterprise is unable to explain how to maintain regulatory compliance and safety in operations, don’t expect the regulator to explain it for you.

A public speaker is a highly regarded expert.

 A speaker at an aviation safety conference made a statement that the regulator has decided to only issue findings against regulatory non-compliances and no longer issue findings to an SMS enterprise for non-compliance with their own internal manuals. That the regulator no longer plans to issue findings to an internal manual is a step in the right direction. When a finding is made to non-conformance with an internal manual an operator has two corrective action plan (CAP) options. The first is to make a change to the manual, or the second option to make a change to the process, or how things are done. Either way, the regulator must approve the CAP by directing what the manual text must read or direct the operator to what specific process they must implement. When the regulator mandates, or locks-in text or processes, they are interfering with the operators sole responsibilities pursuant to the regulations to operate with an SMS that is scaled to size and complexity. An benefit to an SMS enterprise when the regulator only issue finding to regulatory non-compliance is that they now have an opportunity to self-correct their own manuals or ineffective processes.
 

An accountable executive (AE) is responsible for operations authorized under the certificate and accountable on behalf of the mayor, council, airport authority, CEO, corporation or business owner for meeting the requirements of the regulations. In the regulations it states that an appointed AE must have control over human and financial resources. In the past, this regulation was interpreted by regulatory oversight that their only responsibilities was to apply cash to safety and hire personnel to do the jobs. As long as an AE could answer yes to these two questions, they passed their part of the audit. What was overlooked by the inspectors or auditors, was that an AE was not only responsible for cash and personnel, but also responsible for meeting all the other requirements under the regulations. When the regulator no longer issues findings for compliance with the SMS manual itself the manual or processes becomes much more flexible to change, and both airlines and airport operators having an opportunity to perform a true audit of their SMS enterprise. 

The second part of an audit is to audit for outputs, or results. Inputs for these audits are the daily, hourly, or frequently assigned operational tasks. Inputs are how the job is done and what tools are used to support these operational tasks. As an oversight system an SMS enterprise documents the results as they are completed, or at the time of their transactions. Just as upon completion of the shopping list, an itemized receipts and cash exchanged is documented at the time of transaction. Counting the cash is the first step in a quality control system for an upcoming financial audit. Counting outputs and results of an assigned task within an SMS world is a control system for a safety audit. In a financial audit, the auditors do not audit against expectations, how an organization plans to do their inputs, or if they are compliant with their expected inputs. In a financial audit the end result is audited by confirming receipts and financial entries. In a safety audit, the same process is followed by auditing the end results, or outputs, and confirmed by receipts, or data, and entries into the SMS system. An airline or airport conducts daily quality control, regular surveillance of their systems by random sampling and classifies their data to a level of security to preserve its integrity. The audit of an SMS then based on results and not based on virtual, or opinion-based expectations. SMS is to build a portfolio of safety.   

 

 

Catalina9