Tuesday, November 16, 2021

How to Audit SMS

How to Audit SMS
By Catalina9

Conventional wisdom of how to audit the Safety Management System (SMS) is to generate an audit checklist based on regulatory requirements for an SMS, and develop expectations, or processes, in a checklist form to determine level of regulatory compliance. There are several itemized expectations for an SMS enterprise to audit every single aspect of operations for compliance. Auditing by expectations does not paint a true picture of an SMS enterprise level of compliance since an expectation audit does not audit for reliability. 

Research and development is the responsibility of an AE.
An airline or airport may be required to comply with hundreds of regulations in addition to just as many operational standards. One regulation may be compliant by applying several different operational methods, or expectations, which may be interpreted differently by inspectors, auditors or organizational management. When expectations are applied to an SMS audit, all operators are grouped into one expectation and that one-fits-all. Auditing by expectations is a hazard in itself, since an SMS enterprise may change their operational behavior to please the inspector’s or auditor’s checkbox, rather than trusting their own operational judgement. Auditing by expectation is also an avenue to group safety with ratings. A high rating number becomes equal to a high, or superior, level of safety. As an operational oversight system SMS paints a picture of results, or process outputs, and not of a predetermined input. A shopping list contains expectations or inputs, and when used correctly each item is checked off, but the condition, output, or quality of each item is unknown until after the shopping is done.  

The first level of audit of an SMS enterprise is to audit for scalability, or size and complexity. There is a regulatory requirement that a safety management system shall be adapted to the size, nature and complexity of the operations, activities, hazards and risks associated with the operations. Humans are great at making simple tasks complex, or even unmanageable. An unmanageable SMS is a system where hazards to operations are unknown. In an unmanageable SMS, or where an SMS is scaled beyond their operational needs, operations tend to drift towards informal, and simplified processes. An SMS workload is not the SMS itself, but research and develop to scale down systems to size and complexity for regulatory compliance, for safety in operations compliance, for compliance with operational needs and compliance with the SMS policy. An SMS system should be scaled to a level where it can be explained in just a few words. If an SMS enterprise is unable to explain how to maintain regulatory compliance and safety in operations, don’t expect the regulator to explain it for you.

A public speaker is a highly regarded expert.
 A speaker at an aviation safety conference made a statement that the regulator has decided to only issue findings against regulatory non-compliances and no longer issue findings to an SMS enterprise for non-compliance with their own internal manuals. That the regulator no longer plans to issue findings to an internal manual is a step in the right direction. When a finding is made to non-conformance with an internal manual an operator has two corrective action plan (CAP) options. The first is to make a change to the manual, or the second option to make a change to the process, or how things are done. Either way, the regulator must approve the CAP by directing what the manual text must read or direct the operator to what specific process they must implement. When the regulator mandates, or locks-in text or processes, they are interfering with the operators sole responsibilities pursuant to the regulations to operate with an SMS that is scaled to size and complexity. An benefit to an SMS enterprise when the regulator only issue finding to regulatory non-compliance is that they now have an opportunity to self-correct their own manuals or ineffective processes.

An accountable executive (AE) is responsible for operations authorized under the certificate and accountable on behalf of the mayor, council, airport authority, CEO, corporation or business owner for meeting the requirements of the regulations. In the regulations it states that an appointed AE must have control over human and financial resources. In the past, this regulation was interpreted by regulatory oversight that their only responsibilities was to apply cash to safety and hire personnel to do the jobs. As long as an AE could answer yes to these two questions, they passed their part of the audit. What was overlooked by the inspectors or auditors, was that an AE was not only responsible for cash and personnel, but also responsible for meeting all the other requirements under the regulations. When the regulator no longer issues findings for compliance with the SMS manual itself the manual or processes becomes much more flexible to change, and both airlines and airport operators having an opportunity to perform a true audit of their SMS enterprise. 

The second part of an audit is to audit for outputs, or results. Inputs for these audits are the daily, hourly, or frequently assigned operational tasks. Inputs are how the job is done and what tools are used to support these operational tasks. As an oversight system an SMS enterprise documents the results as they are completed, or at the time of their transactions. Just as upon completion of the shopping list, an itemized receipts and cash exchanged is documented at the time of transaction. Counting the cash is the first step in a quality control system for an upcoming financial audit. Counting outputs and results of an assigned task within an SMS world is a control system for a safety audit. In a financial audit, the auditors do not audit against expectations, how an organization plans to do their inputs, or if they are compliant with their expected inputs. In a financial audit the end result is audited by confirming receipts and financial entries. In a safety audit, the same process is followed by auditing the end results, or outputs, and confirmed by receipts, or data, and entries into the SMS system. An airline or airport conducts daily quality control, regular surveillance of their systems by random sampling and classifies their data to a level of security to preserve its integrity. The audit of an SMS then based on results and not based on virtual, or opinion-based expectations. SMS is to build a portfolio of safety.   




Monday, November 1, 2021

The Swiss Cheese

 The Swiss Cheese

By Catalina9

The other day when I was on my way to the store to buy swiss cheese and it was raining. I had opened my umbrella inside, walk under a ladder and a black cat crossed in front of me by the time I got to my car. My day was off to an unpredictable day. I purchased a block of swiss cheese and a package of sliced. I could not see the swiss cheese holes in the block, but I could see them in the sliced cheese, and all the holes were lined up. An unavoidable incident seems to be on the march in my direction today. I had opened an umbrella inside, walked under a ladder, black cat crossed in front of me and now all the holes in the swiss cheese lined up. And to make things worse, I embrace the principle that more is less and less is more. 

Umbrellas are attainable and measurable goals to be used for
a purpose.
An effective Safety Management System (SMS) is expected to run smoothly, and that safety will come by itself if we just do the right tings. The right thing is to find the holes in the swiss cheese and to stop the flow of accidents by plugging or diverting holes. If we make safety objectives and goals, we will be safe, or if we just remain vigilant, observant and follow the rules, we will also be safe. Accidents are built from a blueprint for a system to fulfil an undesired purpose, or aim, and the swiss cheese analogy is an imaginary description to simplify how integrated micro-systems builds accidents.

They key to a successful SMS is to accept that there are micro-systems within larger systems. These micro-systems are defined as at random, since there is no obvious logic to how they form or are placed within its own system. The definition of at random in the Marriam-Webster dictionary is without definite aim, direction, rule, or method, and lacking a definite plan, purpose, or pattern. Applying these micro-systems as random and unpredictable is how they must be applied within an SMS enterprise.  

A latter is a tool to reach new goals, so don’t walk
under it.
The swiss cheese principle is an exceptional good description of at random, or of how accidents are built and the many interactions of events that must take place to build up to the accident itself. However, this principle is only effective as a reactive tool for analysis after an accident, since when arriving at one hole, there is no road map or directions as what turn to take next to avoid lining up another swiss cheese hole. The swiss cheese analysis is non-directional, it is operating within a dark space and each hole in the cheese are individually and specifically placed within its own micro-system and without connections to current events. Holes in the swiss cheese may appear to be randomly placed, but they are systematically placed within its own micro-system produced by carbon dioxide. Each hole in the swiss cheese is a result of a cause which creates the effect. The cause is its own system within the swiss cheese creating these pockets of gas. From outside the swiss cheese, these holes may appear to pop-up randomly, while within the dark spaces of the micro-system itself their location placing becomes predictable

Conventional wisdom is that more is less, and less is more. Professional organizations are rigidly applying this principle in their decision-making process. When applied correctly, simplifying processes is a tool to achieve more. However, simplifying processes does not include a reduction in level of service, or removal of regulatory compliance processes. When more is less, and less is more, there is much more work, research, design, and project planning needed to produce a simplified system output on the front line.  

Looking for the black cat is active hazard identification
Safety in aviation is beyond being a miracle, a matter of luck or dreams come true. Everything happens for a reason, good or bad, positive, or negative. Accepting that at random are micro-systems affecting your goals, and that this system appears as at random is vital to the goal achievement process. If at random is without definite aim, direction, rule, or method, and lacking a definite plan, purpose, or pattern, then accidents are meaningless. A meaningless event has no purpose or reason. An accident is emotionally meaningless since there is no reason or purpose for people to be harmed or loss of life due to unexpected events. However, when applying meaningless, or without definite aim in an SMS organization it becomes impossible to establish cause and integrated factors of occurrences and unexpected events. That everything happens for a reason does not imply that events magically occur, is a statement that there are micro-systems affecting operations which cannot be determined, recorded, or predicted. In an SMS world, these systems are also defined as special cause variations.

A Regulator is responsible for the development and regulation of aeronautics and the supervision of all matters connected with aeronautics. When a new regulation comes into force, an airport or airline operator only have one choice to continue operations, which is to maintain compliance with the new regulation. Public opinions, political environment and aviation incidents are all triggers for new regulations. Over time these regulations add up to an overwhelming task for both airlines and airports. In addition, the regulations require an SMS enterprise to run a safety management system that is adapted to the size, nature and complexity of the operations, activities, hazards, and risks associated with the operations. More regulation and a scaled down SMS system are two opposing regulatory requirements.  

Scaling down an SMS enterprise is not to remove or decline specific regulations, but to combine operational tasks applicable to each regulatory requirement. Scaling down is to make your job as the Accountable Executive userfriendly and manageable for the SMS Manager. When regulations are performance based, an operator is obligated to demonstrate how their activities conform to the regulations. Demonstrating compliance is more than demonstrating compliance with one regulation, but to demonstrate how each task maintain compliance, and how any of these separate tasks does not interfere or causing non-compliance with other regulations.

The swiss cheese principle is a valuable analogy to describe actions, reactions and results of a micro-hazard travelling through the cheese and setting the main system up for failure by travelling through each hole in the swiss cheese. However, if an SMS enterprise establish a safety goal to avoid the swiss cheese holes and objectives are to navigate safely around the holes, they are doing the right thing, but operating with unmeasurable goals since the distance and directions to the holes are not measurable.  


Applying the principle that less is more and more is less gives the same output as the swiss cheese principle. Both principles come with a valuable application, but it is impossible to establish measurable goals from these principles. On the other hand, opening an umbrella inside, walking under a ladder or the black cat crossing, are all events that can be used to establish measurable goals. Find the umbrella, ladder, and black cat within your SMS enterprise micro-systems to build a portfolio of safety. 


How to Audit SMS

How to Audit SMS By Catalina9 C onventional wisdom of how to audit the Safety Management System (SMS) is to generate an audit checklist base...