What To Expect From An Audit

What To Expect From An Audit

By Catalina9 

What we expect of an audit is that it is an unbiased and a neutral report of the facts. 

Everyone in the aviation industry needs to do audits for one reason or another. Audits might be done for regulatory compliance, for compliance with the enterprise’s safety policy, as a contract compliance agreement, at customer’s request as a satisfaction evaluation or after a major occurrence. An airport client must feel ensured that operating out of one specific airport does not cause interruptions to passengers due to inadequate maintenance of nav-aids, visual aids, markings, runways, taxiways, or aprons, or that are any surprises for aircraft, crew or passengers. 

Include in your SMS manual that audit
results are not automatically implemented

An airline or charter operator most often carefully research new airports they are planning to operate out of, and when there is a tie between two or more airports, the one with the best customer service wins the draw. A passenger on an airliner must feel ensured that the flight will be occurrence-free, or a shipper of goods must trust the carrier to ensure that their goods arrive at the destination airport in the same condition it was when first shipped. There is a million considerations and reasons why audits are needed. Since there are several reasons for audits, there are also several expectations of outcome of an audit. What these expectations are, depends on what side of the audit you are and the scope of the audit.

Let’s take a few minutes and reflect on these three different types of audits. The audits are the Regulatory compliance audit, the Safety Policy compliance audit, or the Customer Satisfaction compliance audit. 

The Regulatory compliance audit is a static audit, where no movements or processes are required for the audit. When an operator’s certificate is issued to an airline there are zero movements required for that certificate to be issued. However, there are conditions for operations attached to the certificate, which becomes the scope of regulatory audits. These conditions are management personnel, maintenance personnel and flight crew. All these positions for an air carrier are certificated positions and each person must to comply with their roles, responsibilities, and privileges of their licenses for the operating certificate to remain valid. For a new certificate holder, at the time the first aircraft leaves the gate for a flight, there is an expectation of an audit that pre-departure regulatory requirements are met and that all regulatory requirements are met at the closing of the flight upon arrival at their destination. When an audit of an airline is carried out, the first step is to review their operations manuals for regulatory compliance. At the time of issuance of the certificate they were compliant, but over time amendments are added and new regulatory requirements are implemented. One major implementation example is the Safety Management System (SMS), which had an enormous impact on airlines. Their compliance requirements went from a “job well done” to who did the job and how did they do it. After manuals are reviewed, their operational records are reviewed for compliance. Records for their very first flight, or first flight since last audit, to the most current records are reviewed. Regulatory compliance audits are audits of pre-flight compliance, in-flight compliance, and post-flight compliance. Training records, operations records, maintenance records or crew license records are all audited and assigned a compliance or non-compliance grade. The expectation of a regulatory audit is that any items audited are linked to a regulatory requirement. 

A Safety Policy compliance audit is an audit of an enterprise’s Safety Management System. The audit process is the same as for a regulatory compliance audit, with a difference the audit becomes a job-performance audit. A job-performance audit is about what the task was, when was the task performed, where in the operations was the task assigned, who did the task, why was the task necessary and how was these tasks performed. The “how” audit is an overarching audit for the other five questions: what, when, where, who, and why. A safety policy audit must answer how a decision was reached for each one of the five questions. E.g., how was a decision reach to select and airplane and crew, how was the timeline for crew-pairing selected, criteria for destinations and how was it decide who makes the final decision and why was this person selected.

A safety policy to be “safe” is a
policy with undefined destinations

A safety policy audit is the most comprehensive audit, since is involves all aspects of operations, each person in those operations and a complete timeline of that operations. An inflight example of a safety policy audit is the process for preparing for an emergency upon arrival. A person seated in the emergency exit is prior to takeoff asked if they are willing and able to assist the flight crew with opening the emergency exit. During the flight alcohol is also served to that person who could be intoxicated upon arrival as a temporary crew member with limited duties. A safety policy audit conducts interviews of operational personnel, crew members and maintenance personnel. During these interviews, an auditor may discover that intoxicated personnel are expected to be frontline crew members during an emergency. For each task required by regulatory requirements the same audit process is applied. Just one simple task may take hours to complete, and it becomes a resource impossibility and impracticability to conducts SMS audits of 100% of the requirements, 100% of the personnel and at100% of the times. An SMS audit must therefore apply random sampling and statistical process 

control (SPC) for a confidence level analysis. The industry standard is that there is a 95% confidence level for each element of an SMS to be present for an acceptable audit result. 

A customer satisfaction compliance audit is the simplest audit of all audits. A customer satisfaction audit is audited against opinions, or industry standard expectations. A customer may conduct an audit as an opinion of regulatory compliance, as an opinion of safety policy compliance audit or as an opinion of conforming to industry expectations. Customer satisfaction auditors is not required to be technical experts in regulatory interpretation, operational experts, or experts in airport operations, but are experts in providing opinions of their observations based on their operational experience in aviation. A customer satisfaction audit does not issue findings since the auditor is unqualified to issue findings against regulatory requirements, or operational recommendations. They issue opinions and suggestions for operational changes or implementations as viewed from a customer’s point of view and on behalf of a customer. An operator, being airline or airport, makes a decision if they should implement these changes and how these changes could affect their operations. The criterion for change may solely be based on a customer’s wish, public opinion, or social media trends. An enterprise without a clause in their SMS manual that any findings from any types of audits must first be assessed by the enterprise before accepted or rejected to be implemented in their operations, may be compelled to make changes without knowing the effect.        

An auditor has no responsibility for any occurrences an operator may experience by in their operations after implementing audit recommendations. A new regulatory requirement implemented may affect operational safety. A safety policy recommendation may affect safety and the implementation of a customer suggestion may affect safety in operations. In any case after an audit, being an airline or airport, must prior to implementation of changes conduct a safety case, or change management assessment, to evaluate the risk impact on their operations. Since there is an inherent hazard in aviation from the time an aircraft is moving under its own powers, an operator must monitor what direction the implementation of audit suggestions or requirements are taking and from their assessment continue the course or make operational changes to avoid or eliminate hazards on the horizon. 

Catalina9