Saturday, January 21, 2023

Line-Item Audits

 Line-Item Audits

By OffRoadPilots

Airports and airlines are required to conduct a triennial audit of the entire quality assurance program, calculated from the initial audit, or a series of audits conducted at intervals set out in their manual. There are two phases to a line-item audit. The first phase is a static regulatory compliance, which includes manuals, documents and records. A regulatory compliance audit is in a static environment, without movements or operational tasks. An example could be the runway surface inspection required by an airport stating that this will be done whenever there are changes to the runway conditions, such as snow, slush or standing water etc. This statement conforms to regulatory requirements. However, the important part is that processes in an operational environment also conforms to regulatory compliance. The second phase is process compliance.

A process defined in the safety management system (SMS) manual, are processes that are consistent with regulatory compliance. An example could be the runway surface condition, where the manual conforms, the process described in the operations plan conforms, and records verify that the process was completed, and result submitted. Records requirement for an airline or airport is that their recording systems which do not comprise entries on paper, including computer records, may be used to comply with the record-keeping requirements if measures are taken to ensure that the records contained in the recording systems are protected, by electronic or other means, against inadvertent loss or destruction and against tampering, and a copy of the records contained in the recording systems can be printed on paper.

Conducting a line-item audit begins with an audit of how an SMS enterprise recording systems are protected against inadvertent loss or destruction and against tampering, and if a copy of the records contained in the recording systems can be printed on paper. Paperformat systems written in ink maintain compliance to what level they conform to pre-established process for production sequence verification and the legibility, and includes production date and time. An electronic spreadsheet may be tampered with and may not be compliant unless there is an ongoing hourly or daily, depending on size and complexity, quality control of the system. Electronic systems stored and managed by a general internet technology manager, may have issues interfacing with operations in a secure environment and open the doors for tampering. Several SMS Enterprises make statement to the effect that nobody in their organization will tamper with SMS documentation. While this is true, two reasons to include tampering in the regulation is to preserve the integrity of SMS and in a representative sample of the population tampering happens, and titles or positions are not excluded from the population sample. A cloudbased third-party managing the SMS is the most reliable document and records storage and retrieval process, as long as the third-party is authorized by the accountable executive (AE) and included in their operations manual. Otherwise, it is possible for an operator to lose all data in the blink of an eye and operate with a non-conforming SMS. Non-conforming documents and records processes does not affect the rest of the audit, since an internal audit, or third- party independent audit are not regulatory findings, but are observations or opinions of non-conformances. The regulator is the only body with the authority to issues regulatory findings. An accountable executive may elect to temporarily pause the audit until a satisfactory result of the process integrity has been established.

An accountable executive is a person appointed to be responsible to the regulator for meeting the requirements of the regulations on behalf of the certificate holder. It is not an SMS manager, QA manager or airport manager who is responsible, it is the accountable executive. There is no personal liability associated with the position of an accountable executive as this individual represents the certificate holder. The certificate holder retains all liability for non-compliance with the regulations. At airports where the airport manager is the certificate holder, the airport manager may have accepted this liability. The appointment of an AE does not create an additional burden for operators, as the certificate holder has always been responsible for compliance. The appointment of an AE is primarily a matter of identifying the senior individual who will discharge the certificate holder’s responsibilities, and particular, lead the necessary cultural change.

With the appointment of an AE, a line-item audit becomes an audit of the accountable executive as opposed to an audit of the certificate holder. The outcome remains the same, but with an AE there is one person who is required to answer to internal audits, or regulatory findings. A third- party internal audit are observations and opinions only, while upon sharing an internal audit with the regulatory authority they become findings since the regulator is required to address any safety concerns reported to them.

When the SMS was first implemented by a regulatory requirement, an SMS enterprise was informed that the only responsibility for an accountable executive was to have control of the financial and human resources that are necessary for the activities and operations authorized under the certificate. However, control of financial and human resources are available resources to an accountable executive and are conditions a certificate holder must include in the AE job description to meet the requirements of the regulations.

The line-item audit tool is a tool available to verify that an accountable executive has their systems in place to ensure regulatory compliance. The first part of a line- item audit is to audit manuals for compliance. A manual makes references to standards, policies, processes, procedures, or acceptable practices, which also are audited by a line-item audit. A line-item audit is the most comprehensive and detailed audit available. Manuals and related references are audited in a static environment where an audit match manual text to regulatory references. The next step is the process audit, or to establish what level of regulatory compliance an SMS enterprise operates at. These levels are not scaled levels, but are levels to what deviation from expectations their processes produces. Visual levels of conformance may be published by SPC control charts. A successful SMS includes expectations of outputs defined by the operator. A process without an expectation is only a wish for anything to come true.

A daily quality control system is a requirement for the accountable executive to meet the requirements of the regulation. Without daily knowledge of processes and systems performance, the AE does not have a tool to verify compliance. A simple way to look a this, is to compare SMS performance to cashflow performance, which is closed out daily. A daily quality control system include links in processes to conform to regulatory requirements. With this link, and when a process performs as expected, the regulatory requirement is met. This does not imply that the process cannot be changed but is to monitor if current processes are conforming.

A line-item audit of documents and processes is a supreme tool to ensure that the AE maintain compliance. The beauty of a line-item audit system is that future audits focus on changes in documents and processes. System compliance and monitoring are key factors to maintain a healthy safety management system.


Saturday, January 7, 2023

When an Enterprise Quits SMS

 When an Enterprise Quits SMS

By OffRoadPilots

There are several ways to quit a safety management system (SMS) and an SMS enterprise may unintentionally or unknowingly have quit their SMS. A safety management system is an expensive system, requires hard work and the benefits are unknown, assumed, or abstract benefits. Benefits, if any, remain unknown since an SMS cannot tell the future, or make predications to what, where, where, why, who and how an incident will occur.

When a justification is presented to an SMS enterprise, a CAO, a CEO, or municipality, that SMS is expensive and without tangible results there is a strong temptation to accept these facts. There is no evidence that the SMS will cause a reduction of future accidents, incidents or hazards, there is no evidence of higher return on investment, and there is no evidence that an SMS has

produced better qualified flight crew, maintenance crew or airport personnel. That an enterprise quits SMS does not imply that they abolish their SMS program, but that it is possible to operate an ineffective SMS by while producing desired results.

A crucial question to answer for an airline or airport to operate with a successful SMS is “Why does the Global Aviation Industry, being Airlines or Airports, need a Safety Management System (SMS) today, when they were safe yesterday without an SMS?” The simple answer is that an SMS is needed to generate system analyses specific applicable to an airline or airport, and to have a road map when arriving at the fork in the road. An SMS enterprise is operating with defined processes to conform to regulatory requirements and each task within a system analysis is applied to a regulatory requirement and followed up with a quality control system.

An airline or airport operating with an SMS has at the least an SMS Manual in place that includes multiple processes conforming to regulatory requirements. An SMS manual contains at a minimum a safety policy, a process for setting goals and for measuring the attainment of those goals, a process for identifying hazards to aviation, a process for ensuring

that personnel are trained and competent to perform their duties, a process for analyzing of hazards, incidents and accidents and for taking corrective actions to prevent their recurrence, a document containing all safety management system processes and a process for making personnel aware of their responsibilities with respect to them, a quality assurance program, a process for periodic reviews or audits, and any additional requirements for the safety management system.

In addition, the SMS manual contains specific roles and responsibilities for the person managing the safety management system. These responsibilities are to maintain a reporting system for collecting information related to hazards, incidents and accidents, identify hazards and carry out risk management analyses of those hazards, investigate, analyze and identify the cause or probable cause of all hazards, incidents and accidents, maintain a safety data system by electronic means to monitor and analyze trends in hazards, incidents and accidents, and monitors, at defined intervals, and evaluate the results of corrective actions. An SMS manager also monitor the concerns of the civil aviation industry in respect of safety and their perceived effect on the Certificate Holder (CH), and determine the adequacy of the training for the person managing the safety management system and for personnel assigned duties under the safety management system. 

An SMS manager determines what, if any, corrective actions are required and carry out those actions, keeps records of any determination made, and the reason for it. The responsibility of an SMS manager closes the Plan-Do-Check-Act cycle by notifying the CH of any systemic deficiency and of the corrective action taken. A systemic deficiency includes the implementation of a new system to manage the safety management system.

A Certificate Holder lay their foundation from a blueprint of regulatory requirements and builds their SMS system on top of their foundation. The system must be a stable system, where minor deviations are detected as drift, and special cause variations are analyzed within the context of the SMS system with corrective action plans. If a change leading to an identified special cause variations were intended by the airline or airport, the corrective action also includes a safety case for change. A change could be a policy change, process change, or a change in acceptable practices. When drift, or deviations goes unattended, or unmonitored, an AE would have a difficult time to capture that their SMS had fallen into noncompliance. Conventional wisdom is that a previous accepted SMS manual conforms to regulatory requirements, and that new changes to the system does not affect SMS compliance.

An accountable executive is responsible for operations or activities authorized under the certificate and accountable on behalf of the certificate holder for meeting the requirements of the regulations. Without in-depth knowledge of applicable regulations and how operational processes affects these requirements, an AE may take a non-conforming turn at the fork in the road. Generally speaking, a sole proprietor business owner, or a CEO of a corporation review their financial statements regularly. At some point a demand is placed on personnel to assess expenses and find methods and areas to reduce expenses. When analyzing the safety management system, there were zero hazard reports, zero incident reports, zero accident reports and zero concerns raised by personnel about safety in operations. For an untrained eye, when analyzing the SMS with zero results, the cost of operating with an SMS system that does not produce results should be reduced, or eliminated. Since the elimination mitigation of an SMS is unavailable due to regulatory requirements, the prior step backwards is to mitigate the SMS. There are several ways to mitigate an SMS, but a common business solutions are to eliminate non-essential tasks expenses and eliminate tasks that are producing zero results. By eliminating task, such as the task conformance matrix, a new SMS system is linked to the SMS manual. This system is a conforming system, but since it is a totally new system put in place, all prior data, corrective actions and system analyses are invalidated. When a new system is put in place of how to operate an SMS, an airline or airport starts their SMS process all over again.

Simply said, when processes are removed, when a cloudbased SMS service provider is changed, the data collation system is changed, the analysis system is changed or when operational processes are changed for other than identified improvement changes, another first-time gap analysis is needed. Since results are abstract, it is an ongoing uphill battle to raise support for an effective safety management system that does not identify occurrences. An SMS requires hazards, incidents, and accidents to earn this support.


Saturday, December 10, 2022

Santa’s Just Culture

Santa’s Just Culture

By OffRoadPilots

On 15 March, 1960, Santa Claus was on a reconnaissance trip to review his last delivery trip, and verification that he had not forgotten or left someone out this time. Since Santa implemented the SMS (Streamlined Mission Service), he would do a verification trip, or a quality assurance of his deliveries to learn from the past and improve for next deliveries. Since the SMS was implemented, Santa also does triennial travel audits of the operations during the month of March. On this day in March, the reindeers suddenly lost all their flying powers. Santa had been so busy with the quality assurance program and audit preparations that he had forgotten to tell the elves to feed the reindeers that morning, and they ran out of power. This was an embarrassing moment for Santa. He was known all over the world for timely deliveries, safe transportation, except for a few roof-top crashes, and with a unique quality to know what presents people in different areas of the world wants. When the power ran out, all electrical systems also failed, including Rudolph’s red nose landing light. All Santa know at that time was that he was somewhere where it was very cold, very flat and many lakes, so he established a nice three degrees straight in approach into the unknown. 

It was a dark night, but Santa had faith in the reindeer autonomous landing system, which was powered by an onboard emergency-elf who generated power by running on a treadmill. Without lights, without a glideslope and without any visible clues on the ground, Santa prepared for the worst. One hazard Santa had identified in his SMS, was that a total power failure was a real probability and gliding to roof-tops without power was implemented in his new training program. It was the SMS Director, Mrs. Santa who discovered in her accident reviews that many of the roof tops incidents were due to lack of power for the hoof-reversals to assist braking a higher speeds. After many glide-approaches, the reindeers became very proficient in hitting their landing spots. But this time it was dak, which they had not experienced before. Santa and Mrs. Santa conducted their no-red-nose power approach scenarios by fist establishing s risk classification number, a risk analysis for a safety risk level, a root cause analysis and at the conclusion, a system analysis of the approach. 


Santa could now see the ground, but he had no manual control of the autonomous landing system, so he buckled up the best he could and prepared for a crash landing. The system worked well and commanded the reindeers to flare at exactly the right time and it was a smooth landing. After all the snow was cleared and Santa was looking around, he could see nothing else by snow covered land. He knew he had landed on a lake, and with his extensive knowledge of world geography, he knew exactly what lake it was. It was actually his favourite lakes for summer fishing, and he had been there several times. He had landed in his favorite narrows fishing spot at 466212.95E, 6089107.56N, 48U. Some years earlier a bush pilot flew him in a float plane to this spot, and the pilot did the worst water landing ever. Santa’s friend who came on the fishing trip was a senator and named the landing the Norwegian landing, since he had spent time in the Norwegian Sea with high waves. Santa jumped out of his sleigh and thanked Rudolph and the other reindeers for their smooth landing. After he had looked over the crash site, he got angry and wanted to punish the responsible elf. Luckily, Santa had a direct elf-to-elf telephone and could instantly communicate with Mrs. Santa. When Santa asked her for the name of the responsible elf, she answered that this is not how we run our Streamlined Mission Service (SMS) in a just culture. She also informed Santa that he was the Accountable Elf, and needed to follow the same process as everyone else when conducting root cause analyses. Santa then understood that he could not change a risk level just by the stroke of a pen, but it required hard work. 


Santa has struggled with the just culture principles since they implemented SMS. Just culture is a different behavioral concept and must become lasting habits to achieve positive, sustainable change. Generally speaking, there are two types of organizational cultures. The old way is the blame culture, and the new way is the just culture. The old way blame culture is simply to blame the last link in the chain for the occurrence, lack of competency, incompetent to follow procedures and the root cause for the catastrophic evens. A simple old-way example is when a Santa in training was blamed for a crash when both Santa and the Santa trainee were focused on a hoof-down and locked light malfunction, and failed to stay in the air. The blame culture is simple and easy, but human errors or other negatives are not useful for intervention to improve safety. 

In Santa’s SMS there is a just culture and a place where there is trust, learning, accountability, and information sharing. Santa comprehends the principles that elves nature is to resist changes and that Santa and Mr. Santa must take the very fist step, which starts with an action and not words, text messages or social media shows. A just culture change is to move from known into unknown. Some of the senior elves are in opposition to Santa’s SMS because they do not see their own benefits by changing. There is also uncertainty and insecurity when moving into unknown territories and there is opposition to the way changes to a just culture was presented. Santa’s objective is to instill trust since trust is a key ingredient for a successful change. Trust must be earned, and Santa realized that it cannot be implemented organizational wide supported by any other platform. 


Santa implemented four just culture platforms in his streamlined missions service (SMS) system. 



·      Believe in reliability

o   Without trust there cannot be expectations to perform


·      Self improvements

o   Organizations conduct training, but an individual can only improve by learning

§  Without trust, learning becomes difficult


·      Forwardlooking accountability

o   With trust and learning, accountability to tasks becomes possible

Information sharing

·      Learn from others and the past

o   Without trust, learning and accountability there is no valuable information to be shared


The sun was rising in the East and finally help arrived for Santa. The rescue crew brought food, dry clothes (a used Santa suit) and tools to repair the broken systems. Santa enjoyed the company talked about the old days and all the landing he did over many centuries on the roof of their homes. This was the first time anyone had actually seen Santa and his reindeers. When the repair was done, the snow was cleared off the ice, Santa wanted to do a taxi-run to feel the condition of the ice surface. However, the repair was so well done, that when Santa reach 70 elves-steps per hour, the reindeers lifted off and he was on his way back to the new secret location since the pandemic. 

Santa arrived at home and had a long conversation with Mrs. Clause, who is the SMS Manager and elves, including their process coordinator. Santa did a root cause analysis, since the incident was a special cause variation, and his opinion was to site electrical failure as the root cause, since that caused the reindeer autonomous landing system. Mrs. Santa opposed strongly, but Santa demanded he had the right as the Accountable Elf to decide the root cause by the stroke of a pen. When the elves heard about the root cause, one of the elves came forward and admitted that he had forgotten to feed the reindeers and that is the reason they ran out of power. With Mrs. Santa’s support to the elves, they proposed the root cause to be the reindeer feeding system processes itself and 40% contributed by organizational factors, when compared to elves-factors (15%), supervision factors (25%) and environmental factors (20%). 


Santa reviewed his observations and was glad that he knew the area without relying on the GPS (Genuine Path for Santa) for travel routes. Over several centuries Santa had travelled the globe and visited every home, child and adult in the world and provided them with gift. There were no such thing as good kids or bad kids when Santa delivered. This year Santa had heard rumors that the GTS folks are changing the route to only include the good kids. However, within a Santa SMS system there is a just culture and he plan to turn off the GPS route, use his personal rout knowledge and visit all the kids in the world.  


Several years later Santa’s emergency landing on a remote and cold lake was published in the newspapers. The newspaper story was very different from the actual events and blamed the reindeers for the emergency. When Santa read the story, he smiled and felt good about living in an SMS just culture where issues can be resolved and improved.  





Saturday, November 26, 2022

Accepting or Rejecting Risks

 Accepting or Rejecting Risks

By OffRoadPilots

Accepting or rejecting risks is a fundamental principle in a successful safety management system (SMS). A person managing the safety management system is expected to maintain a process for identifying hazards to aviation safety and for evaluating and managing the associated risks and ensuring that personnel are trained and competent to perform their duties as they apply to the safety management system. This includes training for both the accountable executive and SMS manager, in addition to other airport and airline operations personnel.

A level of risk is an inherent element of aviation safety and there are several types of risks to consider when accepting or rejecting risks. One type of risk may take precedence over another type even if it is not directly associated with operations. Risk control strategies are beyond accepting or rejecting a risk, it is to justify control actions based on defined criteria. There are five categories of risks. The total risk is the sum of identified and unidentified risks. Identified risks are risks which has been determined through various analysis techniques. A task for the SMS manger is to identify all possible risks. Unidentified risks are risk not yet identified. Some unidentified risks are identified by occurrences, and some risk will never be known. Unacceptable risks are risks that are beyond a limit to what is acceptable to an SMS enterprise. Unacceptable risks may be controlled or eliminated. Acceptable risks are identified risks that is allowed by the SMS enterprise to persist without further engineering actions. Residual risks are the left-over risks after all other options has been fully explored. The residual risk is the sum of acceptable risks and unidentified risks and integrated in airport or airline operations. 


Conventional wisdom is that the safety management system is about safety, while the fact is that the SMS is about processes, and how things are done. The expected output of these processes is to eliminate harm and create prosperity. When decisions are based on emotional safety principles, rather than data points of facts, the end result may change risk levels to unknown risk level, or unmanageable risk levels.

The AE is the final decisionmaker to accept or reject risks, system analyses or predictive SMS operations plans. Accepting or rejecting risk is not an authority to deviate from any of safety risk management (SRM) processed, or to base accepting or rejecting on common sense and prior practices. In the past, several practices which were acceptable for an airport operator are unacceptable today within an SMS environment. Airport operators has a responsibility for their airport operations to be compatible with aircraft operations, which is the purpose of an airport. In the past, a NOTAM that a runway was covered with ice or snow contaminants were a sufficient action. However, today within an SMS-world, an airport operator must comply with the airport standards, which includes a friction index requirement, or close the runway. An AE may be the final authority, but when risk acceptances are based on prior practices, both safety in operations, and certificate compliance are jeopardized. Risk acceptance based on prior practices, with the justification that it was done before without incidents doesn’t hold water. In addition, data from prior practices applied to hazard classifications and risks may be outdated. 


An easy trap for an AE to fall into is to believe that they have the authority to change a risk level by the stroke of a pen. Nothing can be further from the truth. When an AE wishes to change a risk level, they must follow established processes for root cause analysis, risk assessment and system analysis, which include a signature page that they rejected a risk level advise from the SMS manager. In most organizations, an AE is the President of the company and the business management expert. An AE is not the data analysis expert but is still the person with final authority to change a risk level. Should an AE reject a recommended risk level, operations affected by the hazard in question is paused until an acceptable risk decision is made. On the other hand, an accountable executive has the prerogative to manipulate risk decisions after reviewing other apparent risks, or identified residual risks, and combined exceeds the effect of proposed risk control. 

The role of an SMS manager is not to lower a risk level due to pressure, but to assess mitigation options for assigned risk level, and options for processes to conform to regulatory requirements and acceptable to the AE. A trap for an SMS manager to fall into, is to change the risk level to the demand of an accountable executive. When an SMS manager is a non-employee at a remote location, temptations to manipulate risk levels are reduced. In a just culture there is no personal liability associated with the position of an AE as this individual represents the certificate holder. The certificate holder retains all liability for non-compliance with the regulations. It is crucial to the success of an SMS that an AE works within the just-culture principles of trust, learning, accountability and information sharing when considering recommended risks controls. 


A purpose of regulations is to establish operational limits acceptable to the interest of public safety as determined by the regulatory authority. Public safety may be a floating object and change with circumstances. In the aviation industry this became evident during the pandemic period, where regulatory aviation limits were changed to justify the cause of a greater threat to public safety. This makes risk control measures only applicable under the regulatory jurisdiction. Unless there are international agreements, a just culture, or non-punitive policy is not applicable beyond the regulatory jurisdiction. For airlines, an acceptable risk control within its own borders my be acceptable, while the same risk control internationally may be rejected, or in worst case a criminal action. A recent event occurred when a charter flight crew discovered an indication in the cockpit that something was wrong in the avionics bay. During an inspection of the bay, a duffel bags with illegal substances were discovered, and the flight crew reported this to the authorities. Since the crew was outside of the jurisdiction of their safety management system they were detained for seven months.


Accepting or rejecting risks is therefore more than just organizational related, it is also related to areas of operations, wherever that might take you. A principle of a successful SMS is that hazards are locally identified.  




Saturday, November 12, 2022

Predictive SMS

Predictive SMS

By OffRoadPilots 

Predictive SMS methods are applied research to entail the development of an expanded and well-organized safety database, as well as the use of predictive, or forecasting methods to identify potential and emerging hazards, trends and behaviour patterns. Using data analysis and predictive methods to identify latent hazards is a tool to prevent future adverse events in operations of any organization. SMS has generated wide support in the aviation community as an effective approach that can deliver real safety and financial benefits. SMS integrates safety concepts into repeatable, proactive processes in a single system. The structure of SMS provides organizations greater insight into their operational environment, including their reactive phase, proactive phase, and predictive phase. A prerequisite for a fully operational predictive safety management system are system analyses. 


There are several purposes to operate with a predictive safety management system, and one of these are to move special cause variations into common cause variation for specific operators and locations. A predictive analysis is forecasted expectations as opposed to special cause variations, where expectations are unknown. A predictive analysis is also different from a proactive approach, since the proactive approach is to assume potential hazards, and predictive approach is to analyze known hazards as facts. It is impossible to predict when a hazard will affect operations and cause an occurrence, but it is possible to predict that a hazard will appear in operations within a pre-established time, location, and direction. A predictive SMS does not predict accidents, incidents, or events since the affect of latent hazards are only available with reactive analyses. A predictive safety management system operates within a 3D system and in a virtual moment of the flight, taxi, vehicle operations or other movements. A 3D identification process is measured in time (speed), space (location), and compass (direction). When 3D thinking is applied in a safety management system, future scenarios can be designed with a defined exposure level to predictive hazards.



Root cause analyses of hazards for specific phase of operations and locations have already been conduced and accepted when operating with a predictive safety management. There is a requirement for the person managing the SMS to analyze and identify the cause or probable cause of all hazards, but this requirement does not extend to identify the cause of every hazard, or the same hazard multiple times. The cause of a hazard needs to be identified once, with subsequent same hazard classification numbers to be monitored in a control chart for pattern and frequency. Note that a predictive SMS is applicable to hazards of same classification number, and not of hazards with similar classifications. A successful SMS operates with a hazard classification system of safety critical areas and safety critical functions within identified areas. 


Analyzing birdstrike data in a predictive SMS generates control charts for reliability pattern and frequency. The outcome of this experiment unfolded as the post was written. Data applied in this scenario are from publicly available data for a specific airport between 2010 and 2022. Adding bird observations by airport personnel, tenants or users would enhance the analysis and improve predictive SMS operations. Data are reactive facts, since there are no expected, or assumed data applied in a predictive SMS analysis. 


The X-mR control chart is used with variables data - data that can be "measured" like time, density, weight, conversion, etc.  Like all control charts, the X-mR monitors variation over time.  The X-mR chart will tell if your process is in control (only common causes of variation present) or if there are special causes of variation.  You use the X-mR chart when you have only one data point to represent the situation at a given time.  For example, suppose your company is tracking accounts receivable each month.  You have limited data - one data point a month.  You can use the X-mR in these situations.  You plot the monthly result on the X chart.  You plot the moving range between consecutive months on the mR (for moving range) chart.”


An X-mR variable chart detects special cause variations. The X-mR chart below shows five spikes of special cause variations, or an out-of-control process, between 2010 and 2022. When a special cause variation is identified requires an SMS enterprise to conduct a full-scale Root Cause Analysis. 

When analyzing the out-of-control points, it is noticeable that they occurred during the summer seasons, with the last spike in 2017. What steps the airport took to eliminate special cause variations in 2018 is unknown. Since the main migratory bird routes through the area did not change overnight in 2018, it is assumed that the airport operator implemented changes. If operating with a proactive SMS, an operator would need to conduct a root cause analysis, system analysis and applied a predictive SMS approach to migratory bird behavior. With a predictive SMS approach to   migratory bird travel, systems may be put in place to direct the birds locally away from airport approaches. This particular airport is previously known for changing local bird travel routes by applying the principles of landuse in vicinity of airport, to divert, or eliminate bird activites. Such activities include diverting travel to and from landfills, water reservoirs, or removal of cereal crops in the area. Previous research has identified that bugs are attracted to the blacktop runway surfaces, which again attracts birds. Without any out-of-control points since 2018, it is assumed that a predictive SMS approached fulfilled its expectations.

A Pareto chart is a data-based approach to determine what the major problem or cause is.  All companies have lots and lots of problems on which to work.  There is not enough time in our day to work on everything.  The Pareto chart gives us a way to determine which problem to work on first – where we will get the most return for our investment.  And the Pareto chart is also a great communication technique as we shall see.

Vilfredo Pareto, an Italian economist, developed the Pareto chart in the late 1800s.  He discovered that 80% of Italy’s wealth was held by 20% of the people.  This has become known as the 80/20 rule or the Pareto principle.  It is at the heart of the Pareto chart.  The 80/20 rule applies in many places – 20% of our customers are responsible for 80% of the customer complaints; 20% of the workforce account for 80% of employee issues.  The Pareto chart is one method of separating that 20% - the vital few – from the 80% - the trivial many.  This allows us to focus our time, energy, and resources where we will get the most return for our investment.”


A pareto chart detects the frequency of hazard classifications. When frequencies are identified, an SMS enterprise may prioritize action plans for classifications with the highest frequencies. In a normal distribution, 20% of events are the cause of 80% of all hazards. 

This pareto chart identifies the months of July, August and September as the months when 73% of hazards are occurring during 25% of the months. For the airport operator, these three months now becomes the target focus area to manage bird activities. For airlines operating out of the airport, these three months become the target focus area for their predictive SMS. However, before jumping to a conclusion to apply these analyses to their predictive SMS, airline operators should approach the airport operator for detailed information about actions applied in their bird and wildlife control program. If no actions were taken by the airport operator, then other factors would have affected the bird activity process to reduce birdstrikes.  


In a search it was learned that the airport had implemented corrective actions, and revealed that in 2018 the airport implemented a new bird control system. Here is an excerpt of the news article (redacted): “The airport is pleased to welcome (company) to the airport. The company brings a specialty Falconry Bird Control Program to the airport which augments the airport’s existing wildlife management program. The company provides a service with trained falcons and other species of birds of prey to manage issues that are caused by wild birds in commercial and industrial environments. Bird control falconry is one of the only target specific methods of control which has the minimum impact of the environment and other non-evasive species within it.”


With the new bird control system, a new control chart analysis from 2018 was conducted that produced a similar special cause variation result. 

Migratory bird routes are common cause variations in the bird movement process. Their travel in the vicinity of airports or using airport lands as their feeding grounds is integrated into their process. The same birds come back year after year. For an airline or airport operator, this bird activity becomes a special cause variation when affecting the planned air travel or airport operations, since it is not an integrated part of their operations. When a common cause variation is manipulated, or controlled, the outcome may deviate from statistical expectations. As noted in this experiment, when the bird activity process is controlled by falconry, both the reliability pattern and frequency were slightly altered. 


The responsibility for improving a process in statistical control lies with management, while front-line personnel may have excellent suggestions on how to do this. Improving a process that is in control may mean changing the average or reducing variation. It is a never-ending process. The system must be changed to improve the process. From the birds’ point of view, their process may now be out-of-control, since a common cause variation was manipulated. The bird control system at this airport changed and monitoring the effect of implemented action verified that the birdstrike counts went down. This is a classical example of how simple, but effective, the concept of a safety management system is. 

With this new information an airline or airport has an opportunity to apply a predictive SMS to their operations. It is not the birdstrike that is predictive, but the bird activity. A root cause analysis can only be conducted of a hazard that the operator has control over, both control over data required for the analysis and control over the corrective action plan. In the bird experiment example, an airline operator has control over publicly available birdstrike data, and they have control over aircraft operations. A root cause analysis may have identified the migratory bird season as a root cause and their control measure may have been to accept the risk, reduce flights to this airport to mitigate aircraft damages, or pause operations during the hours when birds are present. Since an airline is in the business of generating money, it is impractical to reduce, or close down flights due to bird activities. Their own birdstrike data becomes their preferred tool to assess the likelihood and severity in their operations. At this particular airport, with approximately 1.3 mill movements and 5.2 birdstrikes annually, or one birdstrikes per 250,000 movements, any reasonable SMS manager would accept the risk. Zero birdstrike is an unacceptable goal. Both the airport and airline conducted their root cause analyses, their system analyses and is now ready to operate with a predictive SMS. At this particular airport, they continue to track the counts of birds, and birdstrikes, but a root cause analysis is not needed since it is already done for this hazard classification. 


A trap that is easy to fall into, when birdstrike numbers, or any hazards are low, is to reduce or eliminate current mitigation processes. The return of investment (ROI) in an SMS is inverted, with relatively higher investment and fewer occurrences returned. Most often justification for changing the mitigation process is due to cost and the low number of hazards. A Canadian airport voluntarily gave up their airport certificate a while ago since that allowed them to change their mitigation processes and eliminate their safety management system. Just this month they experienced what this trap could cause, by operating without an SMS, which also includes a plan of construction operations. “A privately registered Cessna P210N from (airport) to (airport) was taxiing on the hangar line and fell into an unmarked 3-foot wide strip where the pavement was taken away. The front wheel fell 4 to 5 inches into the construction area. There was propeller damage and engine damage to the aircraft.”





Line-Item Audits

  Line-Item Audits By OffRoadPilots A irports and airlines are required to conduct a triennial audit of the entire quality assurance program...