Transport Canada’s System Level vs. Process Level Surveillance

Transport Canada’s System Level vs. Process Level Surveillance

All the following is excepted from Transport Canada’s own published internal surveillance document, SUR001. You can obtain a copy of the document by simply going to Transport Canada Website and search for SUR001. 

With the advent of Safety Management Systems and the reduction of resources for Transport Canada the need for a change in surveillance activity was inevitable. Transport Canada boiled down their surveillance activity into two broad categories:

1. System Level
2. Process Level

1. Systems level surveillance is conducted using assessments and/or PVIs. During these activities, a surveillance team reviews an enterprise’s systems to develop an understanding of how they comply with regulatory requirements. Based on this understanding, a sampling plan is developed and executed to determine whether an enterprise is in compliance with its regulatory requirements and the degree to which they can effectively continue to remain in compliance. The output of a systems level surveillance activity is findings of either compliance or non-compliance. Findings of non-compliance are at the systems level and are meant to have enterprises correct their systems in such a way that they return to compliance and maintain that state.
2. Process level surveillance is conducted using process inspections. During these activities, TCCA inspectors review an enterprise process to develop an understanding of the process. Inspectors can then trace an output from the process to determine if it functions yes or no (i.e. does the process achieve what it is intended to achieve). The output of a process level surveillance activity can be findings of compliance (and termination of the process inspection) or non-compliance or the initiation of a systems level surveillance activity. 

Systems approach to surveillance

TCCA oversees the safety of the civil aviation system within Canada through monitoring the compliance of civil aviation enterprises to regulatory requirements. TCCA takes a risk based approach to its surveillance activities such that enterprises undergo surveillance at a minimum once every five (5) years.  Therefore, it is necessary that enterprises establish systems to comply with their regulatory requirements and TCCA shall, through their surveillance program, verify that the systems enterprises have in place are adequate to ensure effective on-going compliance to their regulatory requirements.

Through this systems approach to surveillance, TCCA can obtain a degree of confidence that its regulated enterprises are capable of maintaining compliance to regulatory requirements on an on-going basis.

Note: Examples of systems which an enterprise may establish include, but are not limited to: Safety Management System, Quality Assurance Program or System, Training System, Task Control System, Maintenance Control System, Operational Control System, Weight and Balance System, Safety Oversight System, Documentation Control System, Emergency Preparedness System, etc.

Expectations

The basis for the TCCA surveillance program is regulatory requirements and the Minister has the authority to oversee compliance with those regulatory requirements.  

To assist in the task, the Minister has provided systems based guidance regarding the intent behind the regulatory requirements and what is expected in complying with the regulatory requirements.  Therefore, TCCA has created a series of expectations that provide an explanation of what would constitute an effective system for regulated enterprises to achieve and maintain compliance with regulatory requirements. 

These expectations are provided in two ways:

For enterprises required to have an SMS or QAP by regulation, expectations are provided for each component and element of the SMS framework (QAP is one such component).

1. For enterprises not required to have an SMS or QAP by regulation, expectations are provided for their specific type of organization (e.g. 406, 703/704, etc.).  

It should be noted that expectations define the intent of regulatory requirements.  Therefore, there may be several ways for an enterprise to meet an expectation in an effective manner.  As such, the expectations are meant to be used by surveillance teams as guidance for understanding an acceptable means of complying with regulatory requirements when developing sampling plans.

A complete listing of TCCA expectations is provided in Appendix C.

The systems level approach to surveillance is depicted below. It is important to understand this model as it serves as the underlying philosophy of the procedures for systems level surveillance described in the following sections. 

During systems based surveillance, the surveillance team must first develop an understanding of the enterprise’s systems. This understanding is derived from a review of:

Enterprise documentation (e.g. civil aviation documents, manuals, etc.);

Historical data from surveillance; and 

TCCA reference materials which delineate regulatory requirements and expectations to the enterprise (e.g. CARs, expectations provided in Appendix C, etc.).

Based on this understanding, a detailed surveillance plan is developed. Key to this surveillance plan is a sampling plan which outlines what enterprise records, people, products, etc. are to be sampled and why (i.e. the regulatory requirement to be verified through sampling). The surveillance team can then review the outputs of the enterprise’s systems through the execution of the sampling plan.

Based on the review of the systems outputs, the surveillance team can validate whether the systems the enterprise has in place are compliant and effective in continually complying with regulatory requirements. Following this, the surveillance team can make findings against systems compliance (i.e. findings of compliance or non-compliance) and report them to both TCCA and the enterprise.

Based on the findings made against system compliance, TCCA can take steps to ensure the enterprise mitigates any non-compliances (should any have been found).

Management controls the resources. The resources are used to complete the process or product. You complete the process and then you analyze the result. The analysis is then used by Management to adjust the resources for Continuous Improvement. 

Regulatory Oversight will examine outputs from processes. Examine data produced by the company’s SMS and QA system. Examine action by management as a result of the systems outputs and what management does to prevent hazards, incidents and accidents from happening again by affecting resources. 

Baking a cake is a process. It begins with the recipe which is the procedure. The procedure is mapped and then followed backwards, traced. 

Process Level Approach to Surveillance

Process level surveillance is far more focused than systems level surveillance.  It concentrates on a single enterprise process (e.g. technical or operational dispatch, airfield construction control, etc.) to determine the degree to which the process is functions at achieving its purpose and complying with regulations. 

Process level surveillance is intended to determine if the selected enterprise process functions or not.  In this regard, it provides information for decisions about the level of risk associated with an enterprise and potentially determine whether additional systems level surveillance is required.

Process level surveillance can be used as an unplanned surveillance activity to gather intelligence on an enterprise or in response to an unforeseen event or issue. However, it can also be used in support of planned surveillance activities both as part of a PVI or assessment or as its own scheduled activity. It should be noted, however, that process level surveillance cannot take the place of systems level surveillance. TCCA surveillance plans shall ensure that all enterprises undergo regular systems level surveillance; process level surveillance is merely a tool that can be used in support of this endeavour. 

The tool to be used for the conduct of process level surveillance shall be the process inspection.

Transport Canada's tool for using the Process approach is the Process Inspection or P.I. The PI is a tool intended to be used by one inspector and should be concluded within a day or two. The results of a PI could be findings or the request for a PVI or Assessment. In the PI the inspector will "Map" or "Flowchart" a process in your company. Then the inspector will "Trace" the process backwards to check all the inputs of each step of the process. 

It is clear that we need to know these approaches in order to satisfy the requirements of operating a safe enterprise. The reward for adhearing to the system approach is reduced oversight. To learn more about the system approach or process approach to regulatory surveillance, I would recommend getting our latest book: The SMS Memory Jogger by GoalQPC. To get a copy goto www.dtitraining.com or Amazon.com. Also consider the QA/SMS Symposium this coming Sept. Again goto: dtitraining.com click on symposium held at DisneyWorld Florida. 

Your thoughts........