More is Less and Less is More

 More is Less and Less is More

By OffRoadPilots

An accountable executive (AE) once said that they operate their safety management system (SMS) and airport operations by a principle that more is less and less is more. When operating by this principle, their regulatory compliance was in essence non-existing, and the regulator demanded the surrender of their airport certificate. The airport operator presented a corrective action plan to abolish the principle that less is more, and the regulator accepted their corrective action plan. Their airport certificate was secured, but an enormous task was ahead to establish regulatory compliance with all SMS and airport regulations. After the airport certificate was secured and compliance level established, the airport operator abandoned their quality control system and reverted back to their previously less is more principle.

A regulatory requirement is for the AE to be responsible for operations or activities authorized under the certificate and accountable on behalf of the certificate holder for meeting the requirements of the regulations. Traditionally, the airport manager (APM) was the certificate holder and would also remain the certificate holder after implementing the safety management system. As the certificate holder, the APM is the airport authority, the decision maker, and an AE is accountable to the APM to maintain compliance with the regulations. Compliance with all regulations and standards are comprehensive tasks, with compliance established with a line-item audit. When operating by the less is more principle, airport operators take it upon themselves to exclude regulations they have decided not to be applicable to their operations. Airport operators do not take into account that there is none, or minimal, scaling of the regulations to suite size and complexity of airport operations. The scaling is a regulatory requirement applicable to scale the processes as opposed to decline compliance with the regulatory part.

When applying the less is more principle, they are applied laterally to any systems without considerations to the issue at stake or expected outcome. Statements such as, “remember that less is more” are commonly used when undefined expectations are a part of the outcome, or minor tasks are removed from regulatory requirements, or lack of process comprehension, or when non- compliant tasks are excluded from the equation. There are times when the less is more principle is true, but the less-is-more system, is not a system to integrate into a safety management system.

The less-is-more system absolutely has its place within many systems, and advertising is one of them. Imagine for a minute that you are driving down the highway and you see a sign that says something like “Our breakfast menu has pancakes, toast, eggs, farmer’s gravy, bacon, sausage, eggs, and coset between $10-15 per adult person.” By the time you get to pancakes, you have passed the sign and wondering what it said. In a less-is-more system, the sign would say “Hungry? Next exit” With this sign the business gets more visitors who are hungry and generate more revenue. In addition, since there are fewer words, the sign cost less to make. The less-is-more system is a trigger to the imagination to fill in the blanks, and the blanks give positive, or happy feelings of what the imaginary outcome is. Online advertising has also changed to the less-is-more system by shortening their advertisements to five seconds to hit their target points and for the imagination to fill in the rest. Whenever there is a void, it will be filled with something. Other examples of less-is-more is it lower an item price below regular price to sell more units, it is to offer 25-cents video machines to attract more plays, it is to show less of the neighbourhood when advertising a home to attract more customers, it is to pay less for internet with slower connection and spend more time to upload and download, it is to spend less money on personal improvements to assert more internal control of personnel, it is to spend less money on training for a more uniform and conforming environment, or possible most important reason to operate with a less-is-more system is to play ignorance after occurrences. 

An AE once said that it is difficult to work with hazards that are unknown. The less-is-more system absolutely serves a purpose for an AE operate with a less productive safety management system and to some extent the regulator accepts the ignorance play. Just a few weeks ago, an airline gave this excuse for an aircraft that took off with contaminated surfaces saying that the pilots did not follow safety rules and the regulator accepted without further investigation. In 1956 two airlines were operating in a less-is-more environment causing a midair collision. Most people would not chose the less-is-more system when selecting medical treatment, but it is accepted in aviation safety. Ignorance is bliss, or if you do not know about something, you do not worry about it.

The less-is-more system is a destructive system for a safety management system for both airport and airline operations. The role of an accountable executive is to maintain compliance with records keeping. The regulatory requirements for records keeping are to maintain a record system, that do not comprise the integrity of the records system, measures are taken to ensure that the records contained in the recording systems are protected against inadvertent loss or destruction and against tampering, and a copy of the records contained in the recording systems can be printed on paper and provided to the regulatory on notice given. It would take some imagination to make less-is-more out of these requirements, but if works when processes are combined to cover multiple requirements. This is only possible with a daily quality control system, and a user friendly software that comply with all requirements. When a quality control process is established and determined to conform to regulatory requirements, there is minimal work needed in daily operations. Without a proven daily quality control system an airport or airline operator must complete the same tasks daily and start from the bottom every day to ensure compliance. E.g., using paper format records without continuance to the next day or the historical records.

The regulatory requirement foranAEistobe responsible for operations or activities authorized under the certificate and accountable on behalf of the certificate holder for meeting the requirements of the regulations. This is an enormous task and it make sense that an AE sets performance goals to minimalize these tasks as much as possible. Exempting operations from the regulations is not the way to go. A small to medium airport operator may only receive a turbojet aircraft a few times a month and decide on their own that compliance with obligations is not justified since this is how “we always did it.” This is the less-is-more system in that complying with fewer regulations provide more options, or opinions to how airport operations should run. With this approach the safety-card is played, and any tasks or actions are justified by the word “safety.” When the word safety is applied, there is very little opposition to the tasks, and especially if airside personnel remain untrained and without knowledge of oversite requirements. Keeping workers in the dark is a prerequisite when operating with the less-is-more system. Only after a complete line-item audit is completed of the operations, the daily quality control system is in place, and processes assigned to regulatory requirements, the less-is-more system could be applied by monitoring drift and operations daily, and make adjustments as required when personnel are drifting from design operations. However, the AE who decided to change over to the less-is-more system, also excluded the audit requirement from compliance system.

The more is less and less is more system is incompatible with operation of an airport or aircraft, and the safety management system. The litmus test of systems compatible with airport and airline operations is in their daily quality control system.

OffRoadPilots

How to Capture Unknown Hazards

How to Capture Unknown Hazards

By OffRoadPilots 

There is a difference between an unknown hazard and a hidden hazard. Unknown hazards are unknown, but they are not hidden. An unknown hazard is a hazard without a hazard classification, it is a hazard defined by likelihood where times between intervals are imaginary, theoretical, virtual, or fictional.

Unknown hazards are incomprehensible to common sense but are still real hazards. An unknown hazard is in the open and in plain view but is not recognized as a hazard for the purpose of an immediate task to be performed. Unknown hazards also need to be assigned a scope and sequence to learn their whereabouts. A person may be exposed to unknown hazards without knowing it. Exposure to an unknown hazard is a higher risk to aviation safety than exposure to known and hidden hazards since they are unknown and cannot be mitigated.

Hidden hazards are known, but they are hidden and may become visible, or active, if triggered by human factors, organizational factors, supervision factors, or environmental factors. A hidden hazard is removed away from operations in a 3D environment and measured in time (speed), space (location), and compass (direction). Hazards also becomes hidden by remote management environment since the immediate threat to aviation safety does not affect a remote location. A hidden hazard may be hidden for one person, but still be active to another.

A widely accepted method to learn about hidden hazards is to ask personnel to search and identify them in their workplace. One person may identify a condition as a hazard, while another person do not see the same condition as a hazard.

Hazards identified by personnel are often based on emotions, past experiences, based on public opinions, or based on expectations. There are as may reasons to identify a condition as a hazard as there are workers. Mandating a search of hidden hazards is in itself a hazard, since a worker’s attention will be moved from their job activity to searching for hazards. Requesting voluntary hazard reporting as any hazards affects job performance is different, since the workers at that time are focusing on their job tasks rather than identification of what is hidden. After hazards are identified, the role of an SMS Enterprise is to analyze each hazard received, assign a classification, and enter into a hazard register. Identifying a hidden hazard is not the same as identifying an unknown hazard, since hidden hazards are known, but the condition for those hazards do not exist at this time. A prime example of a hidden hazard is when weather conditions are conducive to ice or frost formation on aircraft surfaces, although there is no observable precipitation or fog while an aircraft is on the ground.

Unknown hazards go unattended until there is an incident, accident, or published by a social media post. An unknown hazard is also a special cause variation to aircraft operations since exposure and likelihood has not been accounted for. However, the hazard may be a common cause variation within the process itself. Ice and snow accumulation is known to be a hazard to aviation safety, but at the time of conducting task at hand the hazard is unknown to flight crew until exposed by an incident, unstable flight or published on social media. When this happens, airlines are quick to place blame on pilots, who were just doing their job as expected. A prime example is when an air operator suspends pilots pending investigation into a failure to follow de-icing procedures. In this particular true story, there were no de-icing policy or process established by the air operator to operate out of this airport during icing conditions. 

An aircraft does not carry its own ground deicing equipment and fluids and agreements with airports and contractors are required to deice prior to departure. Without a contract agreement between the airport and airline to deice prior to departures when temperatures are below freezing, pilots complied with management expectations to operate without deicing the aircraft. The aircraft departed without issues, but the hazard became known when posted on social media.

Capturing unknown hazards is an analysis task as opposed to an observation task. Asking workers to actively search for hazards is an observation task. Several hazards can be identified by this method, but the process in itself is reactive since a mitigation plan, or control action is pending on the hazard first being identified. A safety management system (SMS) is simple in concept which is to find the hazards and do something about it. Also, identifying unknown hazards is a regulatory requirement. An SMS enterprise is required to operate with a process for identifying hazards to aviation safety and for evaluating and managing the associated risks. A requirement to identify hazards is for an SMS enterprise to find a hazard, name a hazard, assign a classification to the hazard, and record the hazard in the hazard register. When all this is done, they need a process for setting goals for the improvement of aviation safety and for measuring the attainment of those goals. Capturing unknown hazards is an invaluable tool for goalsetting.

The four items to analyze and capture unknown hazards are within human factors, organizational factors, supervision factors and environmental factors. When searching for unknown hazards, these are the starting points and work backwards from there until hazards are identified. Applying the process inspection flowchart is the same system as the process to capture unknown hazards.

HUMAN FACTORS

An SMS enterprise has an obligation pursuant to the regulations to assign duties on the movement area and any other area set aside for the safe operation of aircraft, including obstacle limitation surfaces, at the airport, which are described in the airport operations manual, only to workers who have successfully completed a safety-related initial training course on human and organizational factors. A human factors training course includes identification of unknown hazards by recognizing that human factors is different than human error. Human factors are behaviors triggered by the five senses. Human error is to complete a task knowing that the task is completed by a non-standard process. This does not imply that that human error is a direct hazard to task at hand, but that unwritten processes are used to “get the job done”. When unwritten processes, or shortcuts are used, the foundation for operational safety analysis are based on unknown criteria, undocumented hazards, or unknown hazards. A shortcut to “get the job done” may actually be the preferred process, but it needs to be documented and unknown hazards identified within the process.

Human factors are vision, hearing, smell, taste, and touch. The SHELL model is the foundation of human factors interactions as the five senses observe and interprets the components in the SHELL model.

• S=Software includes regulations, standards, policies, job descriptions and expectations.

• H=Hardware includes electronic devices, documents, tools, and airfield.

• E=Environment includes designed environment, user friendly environment,

  design and layout, accessibility, and tasks-flow.

• Social Environment includes distancing, experiences, culture, language

• Climate Environment includes geo location, weather, and temperature.

• L=Liveware is yourself and Liveware is other workers within your environment

  Within these areas there are unknown hazards to search for and how they affect operations. An example could be that a regulatory requirement induces stress and shortcuts, or that regulatory compliance increases a level of risk. Tenerife airport disaster is a prime example of how requirements and compliance were contributing factors to the incident. In addition, there are several additional components that could be added to search for unknown hazards within the SHELL model.

ORGANIZATIONAL FACTORS

Organizational factors are factors are strategy solutions, acceptable cultures, technology, regulatory compliance factors and systems information flow within the organizational structure. When an organization, the CEO, President or Accountable Executive of an organization makes a statement to the fact that an incident was caused by non- compliance with a process, is an acknowledgement that their policies, processes and procedures within the organization is perfect and without flaws. There are multitude of organizations that are perfect and without flaws and they operate very successfully. Just recently a large global carrier experienced an unsuccessful event, which they did not have a policy, process or procedure in place to mitigate the hazard. The event was beyond what management expected and the hazard was unknown until it became one of the most disastrous events they experienced. Within an organizational structure data is collected, then turned into information, information is turned into knowledge and knowledge is turned into comprehension. The triennial line-item audit is a tool to identify unknown organizational hazards. An example is an audit line-item 34- 0403 and the debriefing after an emergency response tabletop or full-scale exercise. 

The organization placed the reason for the finding on the auditor who identify the non-compliance. When combining organizational observations, such as operating with icy runway, clearway across a highway, airport vehicle without radio communication on the runway, haying contractor with uncontrolled access to movement areas, construction operations with open trenches, and more, are examples of widespread unknown hazards within organizational factors. An accountable executive, or the regulator, would be unaware of this unless they monitor their daily quality control system. Without comprehension, and training to meet an acceptable comprehension level, unknown hazards will remain unknown.

SUPERVISION FACTORS

Generally speaking, there are four types of supervision. However, in aviation an additional supervision level is introduces. Some of these levels are air traffic services (ATS), air traffic controllers (ATC), flight planning, weather services, control towers, airport ground control, runway, taxiway and apron lights, runway status lights, approach lights, airside markings, markers, and signs. Any of these items are supervisory tasks communicated by other means than words.

Autocratic or Authoritarian supervision:

Under this type, the supervisor wields absolute power and wants complete obedience from subordinates. The supervisor wants everything to be done strictly according to his instructions and never likes any intervention from subordinates. This type of supervision is resorted to tackle indiscipline subordinates.

Laissez-faire or free-rein supervision:
• This is also known as independent supervision. Under this type of supervision, maximum freedom is allowed to the subordinates. The supervisor does not interfere in the work of the subordinates. In other words, full freedom is given to workers to do their jobs. Subordinates are encouraged to solve their problems themselves.

Democratic supervision:
• Under this type, supervisor acts according to the mutual consent and discussion or in other words he consults subordinates in the process of decision making. This is also known as participative or consultative supervision. Subordinates are encouraged to give suggestions, take initiative, and exercise free judgment. This results in job satisfaction and improved morale of employees.

Bureaucratic supervision:
• Under this type certain working rules and regulations are laid down by the supervisor and all the subordinates are required to follow these rules and regulations very strictly. A serious note of the violation of these rules and regulations is taken by the supervisor. This brings about stability and uniformity in the organisation. But in actual practice it has been observed that there are delays and inefficiency in work due to bureaucratic supervision.

The task for an SMS enterprise is to conduct system analyses to find unknown hazards as they apply to operations. An unknown hazard may remain unknown to a

ground crew or aircraft mechanics, but is crucial that the hazard has been found and identified to the flight crew. An example of an unknown hazard is the non- punitive SMS policy, which is only appliable within the jurisdiction where the certificate holder is.

ENVIRONMENTAL FACTORS

Environmental factors are about the surroundings and its affect on accountable executive, managers, workers, personnel, aircraft, cockpit, passenger cabin, aircraft operations, airport operations, or anything else that becomes a part of operations. Environmental factors are more than just the weather, or environment itself, it is also about how work tasks are laid out to function effectively. Environmental factors are about tool boxes and marked tools, it is about checklist and userfriendly flow, and it is about the organizational culture the everyone works within. Airport operators are changing slower to comply with the safety management system environmental factors than airlines are. Airport like to do what they “always” didand not make any changes. In the pre-SMS era, an airport operator could place all blame on the pilot after an accident, as long as the airport had NOTAM’d an issue. This changed with the new airport standards and the safety management system.

Today, the role of an airport operator is to assist flight crew to maintain compliance with their responsibility to ensure that the aerodrome is suitable for the intended operation. Airport operators are still NOTAM 100% ice on runways and expect the flight crew to decide course of action. What airport operators are doing, is preventing medevac or air ambulance from using the airport since most flight crew would not use an icy runway. Environmental factors are also factors withing the regulatory frameworks, which establishes the basis for an environment. Regulations are not minimum safety requirements, but compliance factors to remain as a certificate holder. An example of an unknown hazard within an environmental environment is fear of failure. 

It is critical for an SMS enterprise to accept that not all hazards can be mitigated to an acceptable risk level without cease operations. One such hazard is the probability that a flight crew could establishes an aircraft on an unplanned course any time during a flight but does not justify ceasing operations.

OffRoadPilots

    

The 95% Confidence Level

The 95% Confidence Level

By OffRoadPilots 

A confidence level is the percentage of times you expect to get close to the same estimate if you run your experiment again or resample the population in the same way. A confidence interval consists of the upper and lower bounds of the estimate you expect to find at a given level of confidence. If an airport is estimating a 95% confidence interval around the mean proportion of daily tasks, based on a random sampling of reports, you might find an upper bound of 0.56 and a lower bound of 0.48. These are the upper and lower bounds of the confidence interval. The confidence level is 95%.

With the introduction of a safety management system (SMS) to the aviation, two new terminologies were introduced to the global aviation industry, which were the confidence level, commonly known as a 95% confidence level, and confidence intervals. Prior to the introduction of a confidence level, trends were assessed by the criteria that fewer events are good, and more events are bad. In this system, a trend was established when there are two events, or datapoints, in a row moving in the same direction. When an airport or airline accept two datapoints as a trend they are building their own overcontrolling trap. Determining their level of safety and based on two data points, or events, is overcontrolling of processes. Overcontrolling happens when management do not comprehend information contained in variations with the result that overcontrolling will actually increase variations and causing more unexpected consequences. The old carpenter law when using one stick as a measuring tool for where to cut, is to use the same stick each time or the last stick will be at an incorrect length. Overcontrolling after two data points requires a newtarget to measure from and the last output will be incorrect. For airports and airlines to change over from a reactive SMS culture to a safety culture where they work within an SMS and its confidence level, their first task is to conduct a system analysis of the confidence level system.

A confidence level system is a forward-looking system for strategic planning and designing processes that conform to expected, or desired outcome. A confidence level system within an SMS provides for goal setting, planning, and measuring performance. It concerns itself with organizational safety rather than conventional health and safety at work concerns. An organization's SMS defines how it intends the management of airport and airline safety to be conducted as an integral part of their business management activities. An SMS is woven into the fabric of an organization and becomes part of the culture, or the way people do their jobs. Operating within a confidence level system is a comprehensive and systematic approach to the management of aviation safety, including the interfaces between the airports and airlines, and its suppliers, sub-contractors and business partners. A confidence level approach is also a regulatory requirement for airports to maintain procedures for the exchange of information in respect of hazards, incidents and accidents among all operators of aircraft and the airport operator.

When operating within a forward-looking system, or confidence level system, an SMS becomes a predictive SMS. A predictive SMS is when statistical analyses projects datapoints into the future, by applying process reliability. A process without special cause variations is an in-control process when outputs are as expected or are within the upper and lower confidence interval limits.

The confidence level is in the method, or process itself, and is not in a particular confidence interval. If the sampling method was repeated many times, 95% of the intervals constructed would capture the true population mean. As the sample size increases, the range of interval values will narrow, meaning that a larger sample size, or an increased number of data collected, the mean of the sample will generate a much more accurate result if compared with a smaller sample, or fewer tasks completed.

A confidence level interval are the upper control limits, and the lower control limits. These limits are based on statistical principles for assessing process performances. An SMS is not the old-fashion occupational health and safety reactive system, but is about the health of organizational safety performance, and measured by process reliability performance. A reliable, or stable process, and an in-control process may produce unacceptable results based on the average and calculated upper and lower control limits. An in-control process may be changed as desired to conform to expectations. As an example, if a call center has a policy to answer any calls before the fourth ring, but most calls are answered on the fifth or sixth ring, the process is stable, it falls within the upper and lower control limits, and in-control. However, it is outside of customer service expectation to answer by the third ring. A change in process is then required to meet that goal. When applying the analysis to a predictive SMS, the expectation is that the majority of calls during the next 12-month period will not meet the third-ring goal unless it is changed. When a process is changed, make one change at a time and monitor results.

It is a misconception that a 95% confidence level is unacceptable safety goals for airports and airlines. A wise person once said that you will capture a more correct number of hazards by applying a 95% confidence level to your operations than you will by capturing all hazards. An unknown hazard is also a hazard with an opportunity to affect an outcome in operations. Working with anything else but a confidence level is an unmanageable task.

An airport operating outside a predictive SMS system, assigns airport operations responsibilities to the captain of an intended arriving or departing flight. Conventional wisdom is that publishing NOTAMs releases an airport from all airport operations responsibilities and any incidents are causedby an aircraft captain’s own faulty judgement. A commonly applied airport operations manual (AOM) policy is that an airport is operational 24 hours per day, 7 days per week, supports both day and night VFR and IFR operations to non-precision approach limits, and departures visibility limits to 1⁄2 statute mile (SM) or greater. When a 24/7 policy is published in the aeronautical publications, the airport operator is required to have someone onsite 24/7, but they don’t staff at night, weekends or holidays. 

Airports may be operating with a paper documented SMS, but their operations are still proactive without acceptable processes. Operating without incidents or accidents at an airport, does not equal an operation with a healthy or successful SMS. A process may be in-control, but by the same token it is performing in non-compliance with regulatory requirements, standard requirements and an airport’s SMS safety policy.

When operating a safety management system and applying the confidence level system, both airports and airlines have a golden opportunity to go above and beyond regulatory requirements in both safety in operations and customer service. An airline providing scheduled air service is required by the regulations to operate out of certified airports, but they are not required to use certified airports as their alternate airports. A certified airport must comply with airport standards, while a non-certified airport, or registered aerodrome, is not required to comply with any airport standards at all. When an airline is using an aerodrome as their alternate destination, this alternate aerodrome may not be suitable for operations. All that the airline knows, is what was published in the aero publications and NOTAM, but there they are unable to verify current suitability upon arrival. A registered (non- certified) aerodrome operates in a reactive culture without responsibility to ensure compliance. The only requirements for an aerodrome to be registered and publish their airfield in the aeronautical publications is that warning notices are published for low-flying aircraft, that they have a wind direction indicator installed, if operating at night they need lights to be installed, they need no entry signs installed and they need no smoking or open flame signs installed. Everything else required for the safe operation of an airport or aircraft are voluntary tasks. This includes NOTAM, snow clearing, obstacle limitations on approach, runway, taxiway and apron aircraft size support, fuel availability and more. There are also certified airport operating under the same reactive principle and believe that by publishing NOTAMs they transfer all responsibility to an aircraft operator. When working within a confidence level system with confidence limits established, both airports and airlines have an opportunity to analyze data to conduct an accept or reject risk assessment.

An airport operator is also required to conduct an airport inspection daily, or more often, depending on type of operations and cause of runway contamination. An airport inspection include runways, taxiways, aprons, lights, signage, markings, markers, approaches and items such as new obstacles outside of airport property. Let’s assume that they are required to produce one report daily. Over a year 365 reports are generated, or 1,095 reports over a 36-month period. The first question to answer is if the tasks were completed daily, with a yes or no answer. There is an expectation that over 36 months, 1,095 reports were generated. Let’s assume that 1,095 reports were submitted for an inspection. In a predictive SMS culture, or when working within a confidence level culture, the next step is to learn if the process generated expected results, or output 1,095 times. What makes airports feel secure or safe, is not so much objective security or safety in operations, as a sense of confidence in their own ability to take care of themselves as they did in the past.

Airport and airline operators need to learn what to measure. SMS is to analyze processes and the health of organizational operations, which can only be discovered by applying a confidence level system with confidence limits.

This is the second reason why the global aviation industry, being airlines or airports, need a safety management system today, when they were safe yesterday without an SMS.

OffRoadPilots

How to Run SMS

 How to Run SMS

By OffRoadPilots

The biggest challenge to run a successful safety management system (SMS) is to operate with a system where regulations are performance based, as opposed to prescriptive. Over the years, since SMS was fist implemented, both airports and airlines had, and still have, difficulty to change over to a system where “the regulations does not say that”. Changing from prescriptive regulations to performance-based regulations did not make sense to airport and airline operators, or the regulatory oversight inspectors themselves. When SMS was implemented, it was assumed that a primary challenge would be to change to a just culture and a non-punitive reporting culture. This assumption was wrong, since the most difficult obstacle to overcome was the change from a prescriptive regulatory culture to a performance-based regulatory culture.

A performance-based culture assesses processes and acceptable work practices for compliance with the regulation, while a prescriptive culture assesses compliance with the text of the regulations. A prescriptive culture is a culture where the tasks are to comply with the letter of the text, while a performance-based culture is to apply processes that produces an outcome that conforms to regulatory requirements. Complying with the text does not necessarily prevent occurrences. A prime example is the Grand Canyon midair in 1956, when two airlines collided midair while complying with prescriptive regulations.

An airport or airline working within a prescriptive regulatory environment is required to maintain 100% compliance with 100% of the prescriptive regulations at 100% of the times. On the other hand, an organization working within a performance compliance environment works within a 95% confidence level. A confidence interval (CI) is a range of values that is likely to include a population value with a certain degree of confidence. It is often expressed as a % whereby a population mean lies between an upper and lower interval. A confidence level is the percentage of times you expect to get close to the same estimate if you run your experiment again or resample the population in the same way. The confidence interval consists of the upper and lower bounds of the estimate you expect to find at a given level of confidence.

For airport and airline operators it is a leap of faith into the unknown to jump from a prescriptive compliance culture over to a performance-based compliance culture. When emotions are involved, it is difficult to accept and change from a 100% prescriptive compliance level to a 95% performance confidence level. Emotions do not comprehend that it is impossible to comply and maintain requirements of a prescriptive compliance level. Conventional wisdom is when there are no occurrences or events reported, an operator maintains 100% compliance with the prescriptive regulations.

A prescriptive regulatory requirement sets out a specific standard, requirement or process to follow or actions that a regulated party must take in order to achieve compliance. There is no leeway for errors, or to start over again when tasks don’t go according to plan. The only way to maintain compliance is to avoid occurrences by not reporting any. Since a task must be completed when operating in a prescriptive environment, it becomes impossible to go back and correct a non- compliance since the non-compliance item already has happened. A prescriptive culture is a system where the last link in the chain of events becomes the focus item since all prior tasks were completed to 100% satisfaction. In a prescriptive culture an occurrence may be justified by management statements only. A general statement after an occurrence would be that “Our training and policies were not followed and failure to comply with our policies is not consistent with our culture of safety. We have been in contact with the regulator about this isolated event. No accident took place and there has been no impact on operations.” In a prescriptive operational culture, someone in the organization must be assigned a role and responsibility to wash their hands after occurrences for operations to continue. Within prescriptive compliance system occurrences are justified with little or no improvements made to safety.

For a prescriptive compliance culture to be effective, monitoring of operations must be ongoing, and from a regulatory oversight perspective view all prescriptive regulations or standards must be applied equally to all operators. Since it is impossible for the regulator to maintain 100% oversight of 100% of the certificate

holder at 100% of the time, a prescriptive system will always be reactive. In addition, before a violation can be assessed, it must have occurred, and a reactive approach required.

When analyzing two separate events within a prescriptive compliance system, no changes were made to operations and the violation repeated itself. It was reported that the aircraft landed on runway 24 when the runway was snow covered. During the landing roll the left main gear contacted deeper snow and the aircraft veered to the left. The left main gear caught a 14-inch windrow along the south edge of the runway and the pilot lost directional control. The aircraft departed the runway surface to the left and the nose gear collapsed. The aircraft sustained substantial damage to the nose and propellers. A prescriptive non-compliance was discovered after two events took place. The first event was a snow-covered runway, and the second event when an aircraft hit the windrow. Fast forward 11 years and the same non-compliance occurred, but this time without affecting aircraft operations. An airport operated with 1/8-inch snow on top of ice covering 100% of the runway. In addition, the airport operated with 4 feet tall windrows extending 10 feet onto the runway. Another example is when an aircraft departed with ice and crashed just after takeoff. Fast forward another 5 years and an aircraft was reported to take off with ice adhering to the critical surfaces, but this time also without affecting aircraft operations. Both were non-conforming in a prescriptive environment, but without any changes made to the processes. Runways remain snow covered, and aircraft keep on departing with ice. After severe accidents the prescriptive compliance environment approach comes in handy to pass judgement, point fingers and arresting flight crew after runway incursions. With this approach there is no need for an investigation since judgement has already been passed.

For an enterprise to run their SMS successfully, a daily quality control system must in place as a system where there is monitoring of drift and deviations. Drift and deviations are not necessarily hazardous to aviation safety, but they are undocumented and therefore the output of the process does not conform to an expected outcome. Drift and deviations often occur due to ineffective processes, processes that are cumbersome to work with, management may expect unnecessary tasks to be completed, or a process compliance factor within a performance-based compliance system is not comprehended.

When a successful SMS is run with a daily quality control system, each process is linked to a regulatory requirement. The outcome of the process must meet its expected outcome and conform to regulatory requirements, standard requirements, and the SMS policy. Every process, task and action must conform to the SMS policy for an enterprise to run a successful SMS.

The very first item of a daily quality control system is to establish the roles and responsibility of the accountable executive (AE). Their role, for both airports and airlines, is to be accountable on behalf of the certificate holder (CH) for meeting the requirements of the regulations. This can be achieved by combining multiple systems, or it can be achieved by applying the quality assurance system as the primary compliance requirement. A quality assurance system is the single most important system within the safety management system to maintain compliance in a performance based regulatory environment. A prerequisite for the quality assurance system is to operate with a daily quality control system. When operational management, or operational control, for each operational task is conducted at regular intervals, their data points become immediately available for analysis, monitoring and oversight management. Just as cash in the bank is counted daily (or more often), process conformance must be counted daily. Running an SMS is hard work. There are no shortcuts, or simple way to run an SMS since individual SMS enterprises operates with their own currency, or conforming processes tailored to their operations.

Depending on size and complexity of an airport or airline, daily quality control includes several tasks to be completed hourly, daily, weekly, monthly, annually, or triennially. Triennially is the outer limit since a triennial audit process is required to conform to regulatory requirements. When running an SMS, both airlines and airports need to conduct system analyses and convert these analyses into operations plans. A daily quality control system includes operations plans for each

safety critical areas and safety critical functions in their operations. Safety critical areas and safety critical functions are designated hazard classifications in a hazard register and analyzed for trends and compliance with SPC control charts.

SMS is reliability between operators. It is an airport operator’s role to ensure aircraft ground deicing systems are available and it is the captain’s responsibility to apply it. It is an airport operator’s role to ensure that a runway is clear of contamination, and it is the captain’s responsibility to assess runway suitability. It is an airport operator’s role to ensure they have personnel available during their published hours of operations, and it is the captain’s responsibility to learn what those hours are. It is an airport operator’s role to publish in the aeronautical publications, and it is the captain’s responsibility to review publications. It is an airport operator’s role to conduct runway friction testing for turbojet aircraft, and it is the captain’s responsivity to verity NOTAM. The list could co on and on forever. Times are long gone when airport operators could wash their hands by NOTAM non-conformances to comply with standards or regulations. However, airport operators are still publishing NOTAM for their lack of compliance and are unreliable service providers when operating an airport that is incompatible with the safe operation of an airport or aircraft.

A prime example is a true story about an SMS airport that went back to their prescriptive compliance operations since their performance-based operations were extremely successful. In 2016 this airport totally failed a regulatory inspection. The regulator demanded they return their certificate but the airport operator negotiated an agreement with the regulator to invalidate the 2016 inspection and for the airport to start over again with a “clean slate”. The regulator agreed and over the years their new systems and processes became so successful that they forgot the past and reverted back to their pre-2016 processes. Their justification to revert to pre-2016 SMS was that they are at a stage which have ultimately made the airport a safer and more resilient airport, and able to maintain its federal airport certification. Not only did the airport revert, but they also abandoned their post-2016 compliance system.

As in the true example above, and until airports and airlines accept their different roles and responsibilities and comprehend their performance-oriented compliance processes, their safety management system is its own worst enemy.

OffRoadPilots

SMS Performance Evaluation

 SMS Performance Evaluation

By OffRoadPilots

Every operator with a safety management system (SMS) conducts regular SMS performance assessments of their systems. In a healthy performance environment, assessments are conducted daily within a quality control system. As a businesslike approach to safety, an SMS enterprise has an obligation to learn how their systems are performing, in the same manner as a business assess their cashflow daily. In a business the cashflow is the leftover after cash is received and cash paid. If a business's cash acquired exceeds its cash spent, it has a positive cash flow. A positive cash flow means more cash is coming in than going out, which is essential for a business to sustain long-term growth. This same principle goes for a healthy and sustainable long-term growth of a safety management system.

Conventional wisdom is that a healthy safety management system for airports or airlines, is a system without incidents, or a system with reduction of incidents over time. The question to answer with this approach is what is the next approach, or goal after the zero incidents goal is achieved. A goal must be attainable to be a valid goal, and an attainable goal is a goal with a proven tracking record. A zero incidents goal does not come with a proven tracking record. A major global air carrier states that they are committed to the highest standard of safety, and they expect the same standard of all their suppliers. Since there are no industry standards for the aviation industry of what the highest standard is, an airline or airport has an obligation to develop their own proprietary testing methods and stringent internal guidelines above and beyond regulatory requirements to ensure their services are held to the highest standard in safety.

Without a definition, or expectations of what the highest standard is, the standard could be anything an opinion desire and contracts terminated for any suppliers who are unable to comply with these opinions. When applying this approach, a safety management system may conform to regulatory compliance, but a contract agreement may be terminated if unable to comply with expectations. This was also the first approach taken by the regulator to make findings against expectations with the justification that an expectation was linked to a regulatory requirement.

When the safety management system was first implemented, a list of six components, seventeen elements and about 95 expectations were developed to assist operators to design processes that conformed to regulatory requirements. All elements of the SMS were linked to at least one regulation and divided into the following 17 elements.

Elements:

• a safety policy,

• non-punitive reporting policy,

• roles and responsibilities & employee involvement,

• communication,

• safety planning,

• performance measurement,

• management review,

• identification and maintenance of applicable regulations,

• SMS documentation,

• records management,

• reactive processes, 

• proactive processes,

• investigation and analysis,

• risk management,

• training, awareness & competence,

• quality assurance, and

• emergency preparedness & response.

  Within these 17 elements are about 95 expectations as guidance tools to maintain a successful SMS. Depending on size and complexity, a total of 154 expectations were later designed for airport operators to maintain a sustainable and long-term growth SMS.

  Expectations, policies and goals are non-action items, but are tools available in the toolbox for airport or airline operators to run a successful SMS. Without placing expectations as the first item of tools available for purchase, policies, goals, objectives and processes becomes unattainable. Since running an SMS is a businesslike approach to safety, expectations become units available for purchase (example: an expectation unit available is the purchase of a safety policy). Expectation purchased is the cost of the unit (example: the cost of a safety policy is to design and built goals). Developing, maintaining, maintenance and quality control of expectations purchased are the cost of associated actions (example: the cost of a goal is strategic planning of objectives to reach the goal). Conversion of objectives into processes is the value, or return on investment by an operator (example: the conversion of a goal to operate with a bare and dry runway during winterseasons is the value of an objective). Conversion of an objective into processes are goods and services provided to customers (example: a process to maintain a bare and dry runway is to design the 5-Ws + How; What, When, Where, Why, Who and How). As an advertising and marketing tool, processes are converted into procedures and acceptable work practices for targeting leads (example: Assign roles and responsibilities, and action items to operational personnel). When applying a businesslike approach to the SMS and working with a group of leads, the safety management system becomes a blueprint to design and develop sales presentations to convert leads into buyers.

A reduction in incidents or accidents is not a performance improvement or measurement of a safety management system since the outcome of an occurrence cannot be changed. That the number of operational incidents were decreasing annually is only a report stating that there was fewer incident during the last 12- month period than the previous period. Interpreting this information is not an interpretation of an SMS performance but is merely an emotional assessment of events.

The challenge when operating with a safety management system is to convert abstract information into tangible facts, or expectations into cash value. The only known performance measurement tool to measure businesses performance is to measure in cash value. The goal for a business is that there is more incoming cash than money going out, with an objective to operate with processes to make this goal attainable.

An SMS enterprise is obligated to perform SMS performance evaluation daily of their safety policy, their goalsetting process and attainment of these goals, a hazard identification process and evaluation of associated hazard, their process for training of personnel and evaluation of their competency, their internal reporting process, their hazard analysis process, their corrective action plan process, their SMS manual and communication processes to all personnel, including the accountable executive, a process for making personnel and associated contractors aware of their roles and responsibilities, a quality assurance program, an audit program of their safety management system with defined schedules, and any additional requirements for the safety management system that are prescribed under the regulations. One reason for daily observation of performance evaluations is to detect drift, or non-conforming process at an early stage.

When an SMS enterprise has established this platform they have established their foundation to convert SMS performance into cash-performance. More money spent on safety does not necessarily equate to a safer operational environment. It is how cash is allocated and distributed that makes the difference. The first step when evaluating their SMS performance is to operate with a daily quality control system. This system is the cash-register of a safety management system. Items sold and items purchased are registered in this system and assigned to an account with a regulatory requirement.

An SMS enterprise assessing their SMS system within a cashflow system, the complete purchase and cost of the entire SMS system is the annual cost of the SMS manager. The assumption is that the blueprints are completed, SMS designed and built to its completion. Incoming cash, such as landing fees, ticket purchases or freight revenue, can then be allocated to the SMS system and distributed to one of the expectation accounts. Over time a cashflow statement will generate an income or loss cashflow statement allocated to an expectation account. Operating with an SMS businesslike approach is no different than operating a general store where cashflow is allocated to separate accounts.

Establishing an SMS accounting system is the key to a successful SMS and opens the doors to evaluate SMS performance cashflow in SPC control chart environment. SPC control charts is a tool to make changes as needed to out of control processes, to make changes to the average in processes that are in control, or to allocate or remove expectations to the vital few processes that require attention.

OffRoadPilots

The Inverted Iceberg

 The Inverted Iceberg

By IceRoadPilots

The iceberg effect is a variant of the Heinrich Pyramid developed in the 1930’s. Herbert W Heinrich put forward the following concept that, in a workplace, for every accident that causes a major injury, there are 29 accidents that cause minor injuries and 300 accidents that cause no injuries. The Heinrich Law was widely accepted by the global aviation industry as a risk analysis tool and incorporated into the safety management system.

The Heinrich Pyramid is commonly known as the safety pyramid, or the safety triangle, and indicates a relationship between major injuries, minor injuries, and near-misses. The Heinrich Pyramid concludes that injuries and incidents are caused by a human decision to perform an unsafe action, and that by lowering the number of minor injuries, businesses could reduce the total number of major injuries and incidents. While the most often cited figure would suggest an emphasis on human errors, Heinrich also suggested that workplaces focus on hazards, not just worker behavior.

Commonly known as the iceberg ignorance, a variant of the Heinrich Pyramid principle, was later designed as the Iceberg Model. In the iceberg model, only four out of every hundred operational problems are known to the accountable executive (AE), and 96% of issues, or hazards, are hidden from the AE. The foundation of the iceberg theory is that 100%, or all problems, are known to the frontline workers, being flight crew, mechanics, or airport personnel, while only 4% are known to the AE.

The iceberg concept principle is that all serious problems are the result of several smaller problems that went unnoticed or unmanaged. For every serious incident in the iceberg model there were 59 smaller incidents, and 600 minor conditions. Conventional wisdom is that an accountable executive, as the decision authority, needs to be more aware of minor issues and conditions and initiate actions to stop these issues or conditions before they lead to a serious incident. Preventing minor event to escalate into a full-scale disaster is one reason why the iceberg of ignorance matters to the AE, directors, supervisors, and frontline workers. An accountable executive need to make a concerted efforts to be aware of minor issues and conditions, and overcoming this issue happens primarily through changing the hazard reporting system requirements. There are several valid theories and conditions to the iceberg principles, but there is a major flaw, or finding, in the system when the system is relaying on that knowledge of every minor event or preventing minor events will stop future disasters from occurring.

When applying the near- miss principles from the Heinrich Pyramid, the Safety Pyramid, or the Iceberg Model, an SMS enterprise builds their SMS platform on a misconception that major accidents only occurs after several minor incidents or near-misses are identified. When operating within a human factors system, organizational system, supervision system and environmental system, each individual person within each system performs independent of the other persons. Tasks are performed individually in a 3D environment with tasks measured in time (speed), space (location), and compass (direction). A robot would complete the task within the same timeframe each time, it would initiate the task at the exact same location, and it would follow the exact same process every time. Within a mechanical production system, the outcome of a process produces the same outcome without learning from past errors. When relying on these two principles to establish SMS reliability, an SMS enterprise is placing themselves in a box that is very difficult to crawl out of. Several years ago, a flaw was discovered in compressor turbine disk by applying non-destructive testing of the CT disk. This test was the first test after the final production stage and after it had left the production line. The test discovered a flaw in the material and was reported. There was no safety management system reporting avenue at that time, just an inspection report. The time was also before the iceberg model was widely known and understood. This material flaw was the very first flaw in a CT disk that management new of. Applying the safety pyramid as the accepted standard, 300,000 unsafe acts would be required, followed by 600 reports, or near misses before 30 incidents would occur, then 10 serious incidents, and finally one major accident. There have been several CT disk failures in turbine engines, with Sioux City IA as a high-profile accident. This does not imply that the same or similar flaw was the cause, but that applying the safety pyramid principles keeps an SMS enterprise inside the box. Another question to answer is how does the production of a CT disk crate an unsafe act. Since the safety triangle is based on 300,000 unsafe acts, there must be an astronomical number of unsafe acts daily. 300,000 hours equals 12,500 days, or just over 34 years of systematic undetected unsafe acts. For unsafe acts to go undetected, they must be few and hidden within a system that only the front-line workers knows about. An example could be the speed at which large trucks travel. The opinion of an unsafe speed varies from person to person, and it vary between jurisdictions. Each jurisdiction has their own speed limit set for large trucks, with a justification for safety. Applying the logic in the safety pyramid, the lower the speed limit is, the faster number 300,000 is reached and time between major trucing accidents shortens.

Another example is the publishing of airport NOTAM (notice to airmen). An airport operator has a tool in their toolbox to publish NOTAM when there are issues, construction, or events at an airport. When a NOTAM is published, the airport must implement a counteraction to remain within the airport standards. I have seen this time after time, that airports publish NOTAM, but does not action their operations to remain in compliance with the regulations, airport standards, operational processes, or their SMS safety policy. Just recently, this winter, an airport operator was operating with a NOTAM for four days that the runway was 100% ice-covered, with 1/8 inch of dry snow on top of the ice. Their next NOTAM was 3 inches of snow on top of compacted snow. Since the temperature was well below freezing for several days, and the ice was not removed, ice was still the base surface condition, but went unreported. In addition to the ice and snow, the runway was not cleared to full width, but left 10 feet wide windrows, and 4 feet tall on each side of the runway. The runway remained open day and night with these conditions present. Again, applying the safety pyramid principle, an unsafe act four days out of the 180 days of winterseason, gives an airport operator unlimited opportunities to operate with unsafe conditions. An airport general operating limitation is one-half statute mile visibility. An airport operations manual is a legal reference document between the airport operator and the regulator with respect to level of service. When the visibility is below one-half statute mile, the airport must close to remain compliant with their legal document. Another example of unsafe airport operations condition is that this same airport remained open, day and night, with visibility below legal limit. When operating within a safety management system, an airport operator is required to ensure that their airport is suitable for the operations of an aircraft, and sometimes this may include closing of the airport. Eventually the evidence goes away, and the airport operator continues as nothing ever happened. Both examples are current and true stories. True stories are good examples to learn from but applying safety triangle principles keeps an airport operator together with the regulator inside a box that suits their comfort level and they have no reasons to crawl out of.

When applied by an SMS enterprise operating with processes to action unsafe conditions, non-regulatory compliance processes, and hazards identified as an immediate threat to aviation safety, the Heinrich Pyramid, the Frank Bird Safety Pyramid, or the Edward T. Hall Iceberg Principle are tools to maintain conformance with safety management system principles. When actions are applied to unsafe conditions, or non-conforming processes, these three triangles are turned 

inverted, since each unsafe, or non-conforming conditions become learning experiences to be addressed. When the triangles are inverted, other unsafe conditions and non-conformances, or conditions below the waterline also become visible. The reason why the global aviation industry needs their SMS, is to start chipping away of safety concerns that became visible when the triangles turned.

A compliance guidance document states it beautifully that an SMS enterprise needs to complete a review of the finding, or observation, and clearly identify what happened, how widespread it is within its own organization, where it occurred in the system and if it was a policy, process, procedure, or culture issue. It is not the intent for an SMS

enterprise to reiterate the finding or observation, but rather that they do a factual review of the observations as it applies to their own organization. Their review includes a description of relevant factual information related to the non- compliance, identification of the enterprise system that led to the non-compliance, and they identify policy, processes, procedures, practices or organizational culture involved.

Other operators may learn from observations or findings that are shared with them, but corrective action of events is an internal performance task applicable to a single, and specific organization only. In other words, sharing information is sharing information only, and it is not sharing of corrective action plans.

OffRoadPilots