Radar Vectors
Advisory Circular Current Issued Mar 13, 2015 AFS-200 49 pages Copy saved Nov 18, 2026

AC 120-35D Flightcrew Member Line Operational Simulations: Line-Oriented Flight Training, Special Purpose Operational Training, Line Operational Evaluation

This advisory circular (AC) presents guidelines for the design and implementation of Line Operational Simulations (LOS), including Line-Oriented Flight Training (LOFT), Special Purpose Operational Training (SPOT), and Line Operational Evaluation (LOE) for flightcrew members. This document does not interpret the regulations; interpretations are issued only under established agency guidelines. As operators develop LOSs, they should develop an interdependent relationship between their human factors, Crew Resource Management (CRM), flight operations, and safety initiatives, because they are linked to a common safety goal. This AC describes a means by which LOS scenarios are developed, scripted, tested, evaluated, and (in the case of LOFT and LOE) approved by the Administrator for use in an operator’s training program. The methodology set forth also achieves the Federal Aviation Administration’s (FAA) mandate to ensure that each certificate holder provides the highest level of safety in the public interest, while meeting the agency’s responsibility to reduce or eliminate the possibility or recurrence of accidents in air transportation.

Official source ↗ Open PDF

U.S. Department of Transportation Federal Aviation Administration

Advisory Circular

Subject: Flightcrew Member Line-Operational

Simulations: Line-Oriented Flight Training, Special Purpose Operational Training, Line Operational Evaluation

Date: 3/3/15 AC No: 120-35D Initiated by: AFS-200 Change:

This advisory circular (AC) presents guidelines for the design and implementation of Line-Operational Simulations (LOS), including Line-Oriented Flight Training (LOFT), Special Purpose Operational Training (SPOT), and Line Operational Evaluation (LOE) for flightcrew members. This document does not interpret the regulations; interpretations are issued only under established agency guidelines. As operators develop LOSs, they should develop an interdependent relationship between their human factors, Crew Resource Management (CRM), flight operations, and safety initiatives, because they are linked to a common safety goal.

This AC describes a means by which LOS scenarios are developed, scripted, tested, evaluated, and (in the case of LOFT and LOE) approved by the Administrator for use in an operator’s training program. The methodology set forth also achieves the Federal Aviation Administration’s (FAA) mandate to ensure that each certificate holder provides the highest level of safety in the public interest, while meeting the agency’s responsibility to reduce or eliminate the possibility or recurrence of accidents in air transportation. The contents of this document do not have the force and effect of law and are not meant to bind the public in any way. This document is intended only to provide clarity to the public regarding existing requirements under the law or agency policies.

John Barbagallo Deputy Director, Flight Standards Service

Page iii

CONTENTS Paragraph Page

CHAPTER 1. PURPOSE AND BACKGROUND INFORMATION … 1

1-1. Purpose … 1 1-2. Cancellation … 1 1-3. Related Regulations … 1 1-4. Definitions … 1 1-5. Related FAA Guidance … 4 1-6. Related References … 4 1-7. Background … 4 1-8. Requirements and Concepts … 5 1-9. Summary … 6

CHAPTER 2. BASIC ELEMENTS OF LINE-OPERATIONAL SIMULATION … 7

2-1. General … 7 2-2. Los Philosophy … 7 2-3. Crew Composition and Participation … 7 2-4. Real-World Situations … 7 2-5. No-Jeopardy Training … 8 2-6. Phases of LOS … 8 2-7. Training Hours, LOE, Recurrent LOFT, and Qualification LOFT … 9 2-8. LOS Scenarios … 9 2-9. Approval of Scenarios … 10 2-10. Pre-LOS Briefing … 11 2-11. CRM Component of the Pre-LOS Briefing … 12 2-12. LOS CRM Briefing and Crew Orientation … 14 2-13. Preflight Activities … 14 2-14. Crewmember Responsibilities … 15 2-15. Debriefing the LOS … 15 2-16. LOE … 17 2-17. Use of Audiovisual Equipment and Animation Software … 17 2-18. Additional Training/LOFT and Spot Completion … 17 2-19. Basic Elements of LOS Summarized … 18 2-20. FAA Philosophy … 18 2-21. Safety Management Systems (SMS) … 19 2-22. CRM/Threat and Error Management (TEM)/Risk and Resource Management (RRM) … 19 2-23. CRM Practical Applications … 19 2-24. CRM (Human Factors) Behavioral Markers … 20

CHAPTER 3. TYPES OF LINE-ORIENTED FLIGHT TRAINING … 23

3-1. General … 23 3-2. Recurrant LOFT … 23 3-3. Guidelines for Recurrent LOFT … 23 3-4. Qualification LOFT … 24

Page iv

CHAPTER 4. SPECIAL PURPOSE OPERATIONAL TRAINING … 27

4-1. General … 27 4-2. Guidelines for SPOT … 27 4-3. Specific Scenarios for Operators of All-Cargo Aircraft or Combination Cargo/Passenger (Combi) Aircraft … 27

CHAPTER 5. LINE OPERATIONAL EVALUATION … 29

5-1. General … 29 5-2. Elements Resembling Line-Oriented Flight Training (LOFT) … 29 5-3. Evaluation … 29 5-4. Evaluator … 29 5-5. LOE Media … 29

CHAPTER 6. THE ROLE OF LINE-OPERATIONAL SIMULATION FACILITATORS … 31

6-1. Minimum Qualifications … 31 6-2. Facilitator Resposibilities … 31

CHAPTER 7. LINE-OPERATIONAL SIMULATION SCENARIO DESIGN … 33

7-1. Scenario Design Methodologies … 33 7-2. Philosophy of LOS Design and Conduct … 33 7-3. Scenario Design Process … 34 7-4. Identification of Primary CRM Training Objectives … 37 7-5. Identification of Primary Technical Objectives … 37 7-6. Developing LOS Scenarios With Automation Themes … 38 7-7. Identification of Possible Incidents That Will Produce the Training Objectives … 39 7-8. Event Set Solution Versus Management and Event Trigger Selection … 40 7-9. The Event Set Matrix … 42 7-10. Level of Difficulty … 43 7-11. Integration and Validation af CRM Skills to Event Sets … 43 7-12. Scripting and Validating The LOS Scenario … 48 7-13. Scenario Flexibility … 50 7-14. Grading Event Sets/Grading Phases Of Flight … 50

LIST OF TABLES

Table 3-1. Recurrent LOFT Substitution Table … 24 Table 3-2. Qualification LOFT Substitution Table … 24 Table 7-1. Line-Operational Simulation Design Methodology … 36 Table 7-2. Selected Scenario Event Set Index With Phases of Flight and Proficiency Objectives … 42 Table 7-3. Selected Scenario Event Set Index With Phases of Flight and Crew Resource Management Behaviors … 45 Table 7-4. The Event Set Matrix … 46 Table 7-5. The Event Set Assessment/Grade Sheet … 47

1-4. DEFINITIONS. › a. Advanced Qualification Program (AQP). AQP provides an alternate…

CHAPTER 1. PURPOSE AND BACKGROUND INFORMATION

1-1. PURPOSE. This advisory circular (AC) presents guidelines for the design and implementation of Line-Operational Simulations (LOS), including Line-Oriented Flight Training (LOFT), Special Purpose Operational Training (SPOT), and Line Operational Evaluation (LOE) for flightcrew members. This document does not interpret the regulations; interpretations are issued only under established agency guidelines. As operators develop LOSs, they should develop an interdependent relationship between their human factors, Crew Resource Management (CRM), flight operations, and safety initiatives, because they are linked to a common safety goal. This AC describes a means by which LOS scenarios are developed, scripted, tested, evaluated and (in the case of LOFT and LOE) approved by the Administrator for use in an operator’s training program. The methodology set forth also achieves the FAA’s mandate to ensure that each certificate holder provides the highest level of safety in the public interest, while meeting the agency’s responsibility to reduce or eliminate the possibility or recurrence of accidents in air transportation. The contents of this document do not have the force and effect of law and are not meant to bind the public in any way. This document is intended only to provide clarity to the public regarding existing requirements under the law or agency policies.

1-2. CANCELLATION. AC 120-35C, Line Operational Simulations: Line Oriented Flight Training, Special Purpose Operational Training, Line Operational Evaluation, dated September 27, 2004, is canceled.

1-3. RELATED REGULATIONS. The current editions of the following Title 14 of the Code of Federal Regulations (14 CFR) parts can be located at: http://www.ecfr.gov.

• Part 60, Flight Simulation Training Device Initial and Continuing Qualification and Use. • Part 121, Operating Requirements: Domestic, Flag, and Supplemental Operations; Subpart N, Training Program; Subpart O, Crewmember Qualifications; Subpart Y, Advanced Qualification Program; Appendix E, Flight Training Requirements; Appendix F, Proficiency Check Requirements; Appendix H, Advanced Simulation. • Part 135, Operating Requirements: Commuter and On Demand Operations and Rules Governing Persons on Board Such Aircraft; Subpart G, Crewmember Testing Requirements; Subpart H, Training. • Part 142, Training Centers.

1-4. DEFINITIONS. The following terms are used throughout this AC and are defined as follows:

a. Advanced Qualification Program (AQP). AQP provides an alternate method of qualifying and certifying, if required, pilots, Flight Engineers (FE), flight attendants (F/A), aircraft dispatchers, instructors, evaluators, and other operations personnel subject to the training and evaluation requirements of parts 121 and 135. AQP is a systematic methodology for developing the content of training programs. AQP incorporates data-driven quality control (QC) processes for validating and maintaining the effectiveness of curriculum content. AQP

1-4. DEFINITIONS. › j. Flight Operations Quality Assurance (FOQA). A program to improve…

encourages innovation in the methods and technology that are used during instruction and evaluation, and efficient management of training systems.

b. Aviation Safety Action Program (ASAP). ASAP is a voluntary safety reporting program that serves to encourage employees of certificate holders or other operators to voluntarily report safety information that may be critical to identifying potential precursors to accidents. The FAA has determined that identifying these precursors is essential to further reducing the already low accident rate. Under an ASAP, safety issues are resolved through corrective action rather than through enforcement or disciplinary action.

c. Aviation Safety Reporting System (ASRS). ASRS is a National Aeronautics and Space Administration (NASA)-administered program that provides for the receipt, analysis, and deidentification of aviation safety reports; in addition, periodic reports of findings obtained through the reporting program are published and distributed to the public, the aviation community, and the FAA.

d. Behavioral Markers. Behavioral markers are very specific, observable, and measurable nontechnical practices that reflect the use of CRM knowledge and skill sets and contribute to safe flightcrew member performance. Operators are encouraged to develop their own classifications of behavioral markers that will function effectively within their unique safety culture.

e. Conditions. Existing circumstances that affect flightcrew performance. The circumstances may be external to the aircraft (e.g., weather, runway conditions, or airport area) or internal to the aircraft (e.g., equipment malfunctions or Weight and Balance (W&B) issues).

f. Crew Resource Management (CRM). The effective use of all available resources (e.g., human resources, hardware, and information) to achieve safe and efficient flight.

g. Evaluator. A person who assesses or judges the performance of crewmembers, instructors, other evaluators, aircraft dispatchers, and other operations personnel.

h. Event. An integral part of training or evaluation that is task oriented and requires the use of specific procedures. In an AQP, an event is a specific instance of a training or evaluation objective (with its range of conditions, behavior, and standards) in the context of a curriculum. Once the specific conditions for presenting an objective are selected, that objective becomes an event.

i. Event Set. A relatively independent segment of a scenario made up of several events, including an event trigger, possible distracters, and supporting events.

j. Flight Operations Quality Assurance (FOQA). A program to improve flight safety by providing more information about, and greater insight into, the total flight operations environment through routine recording and analysis of digital flight data generated during flight operations. Analysis of FOQA data can reveal situations that require improved operating, training, and maintenance procedures, practices, equipment, and infrastructure.

1-4. DEFINITIONS. › s. Risk and Resource Management (RRM). Like Threat and Error Management…

k. Flight Simulation Training Device (FSTD). A full flight simulator (FFS) or a flight training device (FTD).

l. Line Familiar. Describes a flightcrew member who is familiar with current line operations. This familiarity may be acquired and maintained either by conducting line operations or by completing an approved line observation program.

m. Line Operational Evaluation (LOE). LOE means a simulated line environment, the scenario content of which is designed to test integrating technical and CRM skills.

n. Line-Operational Simulation (LOS). LOS is a training or evaluation session conducted in a simulated “line environment” setting. LOS includes LOFT, SPOT, and LOE. Instruction and training is based on learning objectives, behavioral observation, and assessment of performance progress and instructor or check pilot/check FE debriefing or critique (feedback). The training objectives under AQP are proficiency objectives that include both technical and CRM issues identified by a task analysis.

o. Line-Operations Safety Audits (LOSA). LOSA is a formal flight deck auditing process that requires expert and highly trained observers conducting observations from the forward observer’s seat during regularly scheduled flights to collect safety-related data on environmental conditions, operational complexity, and flightcrew performance. Confidential data collected in a non-jeopardy environment is used to enhance safety at the operator.

p. Line-Oriented Flight Training (LOFT). Training conducted in an FSTD with a complete flightcrew using representative flight segments that contain procedures that may be expected in line operations. The LOFT includes real-time scenarios that address normal, non-normal, abnormal, or emergency procedures and provides training in CRM. There are two types of LOFT:

(1) Qualification LOFT. Qualification LOFT is conducted to facilitate the transition from a structured flight training environment to line operations. Qualification LOFT is required by part 121 appendix H and must be conducted in an FFS.

(2) Recurrent LOFT. Recurrent LOFT must be conducted in an FFS and may be used to meet recurrent flight training requirements in accordance with part 121, § 121.441.

q. LOS Facilitator. For LOFT or SPOT, an instructor who administers the training session. For an AQP LOE, a check pilot/check FE who administers the evaluation session.

r. Proficiency Objective. An objective containing the criteria for a required level of performance. A proficiency objective contains a range of conditions that may be encountered by the student, while an event contains the specific conditions selected for a particular training or testing opportunity.

s. Risk and Resource Management (RRM). Like Threat and Error Management (TEM), RRM provides a framework or strategy for employing the traditional CRM skill sets, such as communication, workload management, and decision making, to flight operations. Unlike TEM,

1-7. BACKGROUND. › a. LOFT. The use of gate-to-gate FFS scenarios, known as LOFT, began in…

it provides a more complex series of strategies and toolsets and integrates more directly to a certificate holder’s Safety Management System (SMS).

t. Safety Management System (SMS). SMS is the formal, top-down business approach to managing safety risk. It includes a systematic approach to risk management, including the necessary organizational structures, accountabilities, policies and procedures.

u. Special Purpose Operational Training (SPOT). SPOT is an FSTD training session designed to address specific training objectives. Training objectives are based on technical and CRM requirements, and include specific training objectives to be critiqued and debriefed on both technical and CRM performance. SPOT may consist of full or partial flight segments, depending on the training objectives for the flight.

v. Task Familiar. Describes a flightcrew member who is familiar with and can satisfactorily accomplish the duties of a particular flightcrew member duty position, though not qualified for that duty position. For example, a second in command (SIC) candidate who performs the duties of the pilot in command (PIC) during FSTD training.

w. Threat and Error Management (TEM). Like RRM, TEM is a framework or strategy for employing the traditional CRM skill sets. TEM, like RRM, promotes safe operations through the continuous process of identifying, avoiding, mitigating, or managing “threats” (external errors, conditions, or situations), internal errors, or undesired safety states.

1-5. RELATED FAA GUIDANCE. The current editions of the following ACs can be located at: http://www.faa.gov/regulations_policies/advisory_circulars/.

• AC 120-51, Crew Resource Management Training. • AC 120-54, Advanced Qualification Program. • AC 120-66, Aviation Safety Action Program (ASAP). • AC 120-82, Flight Operational Quality Assurance. • AC 120-90, Line Operations Safety Audits. • AC 120-92, Safety Management Systems for Aviation Service Providers. • AC 120-109, Stall Prevention and Recovery Training. • AC 121-42, Leadership and Command Training for Pilots in Command.

1-6. RELATED REFERENCES. Facilitating LOS Debriefings: A Training Manual, NASA Technical Memorandum 112192 DOT/FAA/AR-97/6 (http://humansystems.arc.nasa.gov/flightcognition/Publications/Final_Training_TM.pdf).

1-7. BACKGROUND.

a. LOFT. The use of gate-to-gate FFS scenarios, known as LOFT, began in the mid-1970s as a means to provide pilot training that is more representative of actual flight operations than maneuver-based training alone. LOFT was soon recognized as a highly effective means of developing and practicing CRM skills. Due to the role of CRM issues in accident causation, it has become evident that training curriculums must develop pilot proficiency in both technical and CRM skills. While LOFT is designed to include all phases of flight, scenario-based training

a. LOFT. LOFT is a useful training method because it gives flightcrew… › (2) Part 121 Appendix H.

may also include limited portions of flight designed to focus on specific operational training needs, known as SPOT. Air carriers with an approved AQP must also conduct evaluated LOFTs, known as LOE, for jeopardy grading purposes. These three methodologies, LOFT, SPOT, and LOE, are now grouped under the general heading of flightcrew member LOS.

b. CRM. The introductory CRM training that many flightcrew members have experienced is similar to the foundation of a building: it is an essential structural part, but by itself the foundation has limited operational use. If CRM training is to be operationally effective, it must be built into other training steps and activities in a systematic way. A structured LOS design process is employed to specify and integrate the required CRM and technical skills into LOS scenarios.

c. LOS. LOS is an environment that is structured to allow and encourage the application of technical and CRM concepts to a situation that enables conceptual knowledge to become working knowledge. Instead of being programmed with a solution, the students can manage the operational environment and process available information to learn its limits, properties, and operational relevance. LOS can be conducted in an FFS or FTD, depending on whether the LOS is for training or evaluation, and the requisite fidelity of the training/evaluation media.

d. Basis. Much of the information in this AC was taken from a working paper developed by Airlines for America (A4A) (formerly known as the Air Transport Association of America (ATA)) Training Committee, AQP Subcommittee, LOFT Design Focus Group. This AC provides a structured design process for LOS design and implementation and builds upon air carrier experience with developing and implementing scenarios to provide guidelines for LOS programs. The scientific literature on the event set methodology is available from the Air Carrier Training Systems and Voluntary Safety Programs Branch.

1-8. REQUIREMENTS AND CONCEPTS.

a. LOFT. LOFT is a useful training method because it gives flightcrew members the opportunity to practice line operations (e.g., maneuvers, operating skills, systems operations, and the operator’s procedures) with a full flightcrew in a realistic environment. Flightcrew members learn to handle a variety of real-time scenarios that include routine, abnormal, and emergency situations. They also learn and practice CRM by way of operator-developed behavioral markers that may include, but are not limited to, essential elements such as situational awareness, communication, decision making, workload management, and automation management skills. The overall objective of LOFT is to provide training that improves total flightcrew performance, and thereby preventing incidents and accidents during operational flying.

(1) Section 121.409. Section 121.409(b) delineates the requirements of recurrent LOFT, which may be used to meet recurrent flight training requirements in accordance with § 121.441. Section 121.409(b) requires a complete flightcrew to be used in recurrent LOFT, but does not provide detail on what constitutes a complete flightcrew. The guidance provided in this AC recognizes a complete flightcrew as one that is task familiar and line familiar.

(2) Part 121 Appendix H. Appendix H contains requirements for qualification LOFT for certificate holders who choose to provide flightcrew member training in advanced airplane

CHAPTER 1. Purpose and Background information › 1-9. SUMMARY.

simulators. This AC presents guidelines for implementing qualification LOFT as required under appendix H or as may be used within any other approved training program. This AC discusses how qualification LOFT is designed to help flightcrew members transition from a training environment to operational flying.

(3) AQP. AQP requires the use of LOS in both qualification and continuing qualification (recurrent) curriculums and encourages the use of SPOT and LOFT.

b. SPOT. New training concepts and training media have provided the opportunity to creatively tailor training sessions to address specific training objectives. Training objectives are based on technical and CRM requirements. SPOT may consist of full or partial flight segments, depending on the training objectives for the flight. This AC presents guidelines in conducting SPOT.

c. LOE. The LOE is the primary means of proficiency evaluation under an AQP. This evaluation addresses the individual’s ability to demonstrate technical and CRM skills appropriate to fulfilling job requirements in a full mission-scenario environment. The intent of an LOE is to evaluate and verify that an individual’s job knowledge, technical skills, and CRM skills are commensurate with AQP qualification standards. The LOE is conducted in an FSTD qualified in accordance with part 60 and approved for its intended use in the AQP.

1-9. SUMMARY. This AC identifies four types of LOSs: (1) Recurrent LOFT (in reference to §§ 121.409, 121.427, 121.433, and 121.441); (2) Qualification LOFT (in reference to part 121 appendix H); (3) SPOT, which is line-oriented training that addresses specific training objectives; and (4) LOE, which is a line-oriented evaluation designed for flightcrew members qualifying using an AQP. It defines the terms used in describing LOS. It provides guidance for conducting LOFT, SPOT, and LOE. It defines the role of instructors and evaluators, and provides guidance for designing LOS scenarios.

CHAPTER 2. BASIC ELEMENTS OF LINE-OPERATIONAL SIMULATION › 2-4. REAL-WORLD SITUATIONS.

CHAPTER 2. BASIC ELEMENTS OF LINE-OPERATIONAL SIMULATION

2-1. GENERAL. Certain elements about Line-Operational Simulation (LOS) must be understood to ensure that its primary objective, to provide realistic line-oriented training and evaluation, are met. These elements apply to both recurrent and qualification Line-Oriented Flight Training (LOFT), and are described in this chapter.

2-2. LOS PHILOSOPHY.

a. LOS. The overall objective of LOS is to improve total flightcrew performance by integrating Crew Resource Management (CRM) and technical skills. CRM skills include techniques that allow and encourage crewmembers and aircraft dispatchers to become better problem solvers and resource managers. The LOS context must be structured to enable CRM behaviors to emerge and the crew to become aware of them; that is, the scenario must last long enough for crew traits to become evident and should require CRM skills to be displayed in response to specific circumstances. Similarly, scenario construction should focus on the CRM and technical objectives integrated into a training program.

b. LOFT and Special Purpose Operational Training (SPOT). LOFT and SPOT are training events in which flightcrews can enhance their CRM and technical skills. Line Operational Evaluation (LOE) contains events in which CRM and technical skills are assessed. LOS learning and assessment should not be artificially stress free; crewmembers should maintain performance parameters applicable to their phase of training. This will require crews to handle failures and their consequences. If the LOS facilitator identifies flightcrew member performance deficiencies, additional training should be provided.

2-3. CREW COMPOSITION AND PARTICIPATION. LOSs should take place in a line operational environment with a complete flightcrew. A complete flightcrew concept allows crewmembers to use their full resources and creativity to support a complete learning experience. During LOS, each flightcrew member performs both as an individual and as a member of a team, as is expected during line operations. Line crewmembers must be scheduled and paired together, as much as practical, in a standard crew configuration (e.g., line captain with line first officer). Circumstances will occur where the initial composition of the schedule cannot be maintained. Hiring requirements, high first-officer-to-captain ratios, illness, or failure of a crewmember to progress are all situations that would necessitate providing a seat substitute to complete the training. In all cases, the seat substitute must be task familiar with the flightcrew member duty position.

2-4. REAL-WORLD SITUATIONS. LOS should contain scenarios of real-world, line operational situations, which progress in real time. These scenarios should be representative of flight segments where an entire en route operation is completed. In cases of flights with flight segments involving repetitive events, the en route segments may be compressed. However, if the scenario is compressed, enough time should be allotted for crewmembers to resume or restart the scenario without confusion. In addition, the scenario compression method must be designed and executed to minimize degradation of scenario realism and enhance training or evaluation. It is common practice for scenarios to be based upon actual events reported through the Aviation Safety Action Program (ASAP), Aviation Safety Reporting System (ASRS), flight operations

2-6. PHASES OF LOS. LOS scenarios should contain the following phases: › d. Debriefing. Debriefing should include feedback to crewmembers on…

quality assurance (FOQA), Line Operations Safety Audit (LOSA), and other event-reporting programs.

2-5. NO-JEOPARDY TRAINING. LOFT and SPOT are “no-jeopardy” training (i.e., not pass/fail). If the LOFT or SPOT instructor identifies flightcrew member performance deficiencies, additional training will be provided. This training should be in a form most appropriate to correct the specific performance deficiency, including additional LOFT. Before the crewmember may return to line operations, the performance deficiencies must be corrected, and the instructor must document the training as satisfactorily completed. The no-jeopardy concept allows crewmembers to use their full resources and creativity. At the end of a LOFT session and after debriefing, if the flightcrew member has demonstrated satisfactory performance, the instructor must certify as to the proficiency and knowledge of the crewmember. This includes situations where the instructor provided additional training following the LOFT in order to correct any performance deficiencies. The certification must be made in accordance with part 121, § 121.401(c) or part 135, § 135.323(c), as applicable. However, if the flightcrew member was not able to demonstrate satisfactory performance, even after additional training following the LOFT, the instructor may not certify the flightcrew member’s proficiency and an additional LOFT session should be scheduled.

2-6. PHASES OF LOS. LOS scenarios should contain the following phases:

a. Briefing. Before the flight begins, the LOS facilitator should brief crewmembers on the LOS scenario, including the training objectives, and the role of the LOS facilitator (i.e., the LOS facilitator is considered not present, except as to perform other roles in support of the scenario, such as flight attendant (F/A), aircraft dispatcher, or air traffic control (ATC)). The role of the flightcrew should be discussed in the briefing (i.e., flightcrew members should perform their duties just as they would in line operations). Information about the environmental setting of the scenario, when appropriate, should also be discussed.

b. Preflight Planning Documents and Activities. Preflight planning documents (e.g., weather reports and flight plans) should be prepared with the operator’s particular training objectives in mind. For example, the operator may choose to have crewmembers learn how to handle unfavorable weather conditions or how to correct improper fuel loads. Preflight activities include flight deck setup and computation of takeoff data.

c. Flight Segment. The flight segment includes taxi-out, takeoff, climb, cruise, descent, landing, and taxi-in, as appropriate. It should also include the time in which communication with ATC and other ground agencies takes place.

d. Debriefing. Debriefing should include feedback to crewmembers on their performance. The FAA recommends use of the facilitated or reverse debrief, wherein the LOS facilitator first allows the students to debrief and provide their observations. Positive comments regarding crew performance should be emphasized in the debriefing, as well as crew performance that needs improvement. The debriefing involves LOS facilitator critiques of individual crewmembers and of the crew as a team. Also, it is important that crewmembers be given the opportunity to critique and analyze their own performance and review key points of the recording, if used. The National Aeronautics and Space Administration (NASA) Technical Memorandum 112192, Facilitating

b. Scenario Construction. A variety of scenarios can be constructed by… › (5) Alternate operation of flight management systems (FMS).

LOS Debriefings: A Training Manual, provides guidance on facilitated debrief techniques. This document is available at: http://humansystems.arc.nasa.gov/flightcognition/Publications/Final_Training_TM.pdf.

2-7. TRAINING HOURS, LOE, RECURRENT LOFT, AND QUALIFICATION LOFT. The Administrator will approve the content and duration of LOFT/LOE scenarios. By regulation, recurrent LOFT and qualification LOFT must provide at least 4 hours of training. The 4 hours of full flight simulator (FFS) training should include flight deck preparation, preflight activities, crew briefings, and interactions with flight dispatch and other ground agencies. Reasonable amounts of time should be allowed for problem solving (e.g., consulting minimum equipment lists (MEL) and operations manuals, preparing takeoff data, as well as other crew actions that are occasioned by the training scenario). All crewmembers participating in a LOFT session are credited with 4 hours of training time.

2-8. LOS SCENARIOS. LOS scenarios should be developed using the following guidelines:

a. Objectives. The operator should assign specific training and/or evaluation objectives to each scenario, as applicable. These objectives should be based on the particular needs of the operator. For example, if an operator is experiencing an unusual frequency of a specific operational problem, such as wet or icy runways, the scenarios should be designed to include exposure to that particular operational problem. Other specific objectives may include winter operations training, unusual airport or runway operations, alternate operation of automated systems, etc. LOS continuously trains and/or evaluates the development of the essential human factor elements of crew performance in the accomplishment of their duties. LOS scripting must ensure that both the training and/or evaluation process support a wide range of both technical and CRM considerations. In addition to the objectives established by the operator, the FAA may also identify objectives based upon documented trends.

b. Scenario Construction. A variety of scenarios can be constructed by choosing different combinations of elements from the suggested categories listed below. Scenarios should be representative of the flight segment appropriate to the operations being conducted by the operator.

(1) Preflight activities might contain elements such as icing or cargo-loading anomalies that the crew must address during preflight planning and flight deck preparation.

(2) Taxi operations, including the concept of navigating the aircraft from the gate to the runway or from the runway to the gate, using available signage and charts to prevent runway incursions.

(3) Origin, routing, and destination (e.g., short versus long routes).

(4) Revised arrival procedures (e.g., an unexpected runway change).

(5) Alternate operation of flight management systems (FMS).

CHAPTER 2. BASIC ELEMENTS OF LINE-OPERATIONAL SIMULATION › 2-9. APPROVAL OF SCENARIOS.

(6) Abnormal and emergency conditions, including simple conditions (e.g., a potential hot start) and complex conditions that continue for the entire flight (e.g., a failed essential alternating current bus).

(7) Adverse weather conditions.

(8) Partial or full loss of integrated FMS.

c. Scenario Cues. In general, the scenario should provide all students the same cues and distractions they would encounter in a line environment. Other than the case of the SPOT, no cueing should come from the LOS facilitator other than what has been built into the script of the scenario in advance. During an LOS scenario many pilots are hypervigilant in their search for cues, fully expecting the sort of event that would surprise them on the line, but which is expected during training and evaluation. Some scenarios should introduce cues that would indicate grossly incorrect pitch trim setting.

d. Scenario Timing. Scenarios should run in real time, to the extent possible. This may include inactive time to realistically resemble actual operations.

e. Scenario Development. Scenarios should contain realistic circumstances such as incoming and outgoing messages from ATC or the F/A(s), as well as routine interruptions. Scenarios should also be developed to observe checklist management procedures, standard callouts, leadership qualities, assertiveness, crew coordination, communication, and ground movement operations. Operators may use these elements to design full-length, real-time scenarios, as well as shorter scenarios that teach specific skills (e.g., wind shear, special navigation equipment, Traffic Alert and Collision Avoidance System (TCAS), incorrect pitch trim settings, and incorrect cargo loading.).

f. Scenario Updates. Scenarios used in recurrent or continuing qualification training should be updated periodically (at least annually) to help ensure they continue to meet evolving training objectives. Each update of an LOFT or LOE must be approved by the Principal Operations Inspector (POI). Just as crewmembers cannot anticipate all flight operational situations, operators should try to prevent crewmembers from anticipating the content of the scenarios. Under normal circumstances, no student should ever experience an identical scenario more than once. (See Chapter 7, Line-Operational Simulation Scenario Design, for a more detailed description of the scenario development process. This chapter includes techniques for updating and altering scenarios to keep them fresh.)

2-9. APPROVAL OF SCENARIOS. The Administrator will approve LOFT and LOE scenarios. When submitting LOFT or LOE scenarios for approval, operators should state what training and/or evaluation objectives they expect to achieve. Operators must submit specific LOFT or LOE scenarios to the POI for approval. Scenarios that comply with the elements provided in this advisory circular (AC) and meet the operator’s stated training and/or evaluation objectives may be approved. Detailed scripts of the scenarios may be submitted to the FAA, but are not required for FAA approval. Scenarios must be scripted to the extent that each student is given a standardized scenario experience that allows the LOFT or LOE to be used for training and/or evaluation purposes. Nonstandard application of evaluation scenarios is unfair to the

b. Conduct. The following LOS conduct principles should be presented in… › (8) Equipment will be placarded according to MEL procedure and noted in…

student, who may experience an advantage or disadvantage due to the different execution of the evaluation event.

2-10. PRE-LOS BRIEFING.

a. Philosophy. The philosophy underlying the particular LOS being administered should be thoroughly explained before the crew begins to plan for the flight. Inadequate LOS briefings often set the stage for problems that later interfere with LOS realism. One common difficulty is failure to convince the crew that the LOS facilitator is functionally not present during the event—that he or she will not be available for communication except in roles as scenario support, such as ATC, company, or maintenance.

b. Conduct. The following LOS conduct principles should be presented in the pre-event briefing:

(1) Except for the LOE, LOS is designed as a pure learning experience.

(2) The facilitator’s role in LOS is to manage the event to maximize learning. This does not include scenario interruptions to teach right solutions, or to test the trainees, except for SPOT. The opportunity for full self-analysis is provided during the debriefing. The LOS facilitator will take notes only to assist in this debriefing.

(3) LOS is a training concept designed to accent CRM skills, as well as the integration of CRM and technical skills. Other training and evaluation events already focus exclusively on technical skills and knowledge. Therefore, technical proficiency, while important, is not the driving force behind the addition of LOS scenarios to an otherwise technical curriculum. Realism is maintained to the greatest extent possible.

(4) All phases of flight will be sequenced in real time. Center-stored routing will be followed unless the crew or ATC requests rerouting. This guidance does not apply to SPOT.

(5) Mistakes may be made, just as they sometimes occur on the line, and the flightcrew is expected to continue the operation. The ability to detect and correct errors is an important safety skill, so opportunities to practice error management strategies in the safety of the simulated setting should not be discouraged.

(6) Frequently, there is no book solution to an LOS exercise—there may be no one correct solution. For example, the crew may decide that a diversion is more prudent than landing at the filed destination. Scenarios should be written to offer several operational choices. This is discussed in more detail in Chapter 7.

(7) All abnormal or emergency situations will be handled in the appropriate manner. These situations will last throughout the flight, unless they can be corrected by the use of alternate operations or any line resources normally at the crew’s disposal.

(8) Equipment will be placarded according to MEL procedure and noted in the logbook. The flightcrew will consider placarded items legal per the aircraft’s MEL. The LOS facilitator

c. Communication. Questions that encourage crews to consider… › (1) How does the crew view inquiry and advocacy?

has the responsibility to determine whether effective training/evaluation can take place with the inoperative equipment.

(9) During an Advanced Qualification Program (AQP) LOE, the crew will be given realistic situations to address as a part of the evaluation. The crew will be expected to perform to standards in both technical and CRM skills that have been previously trained as part of the AQP curriculum. The evaluator will be assessing outcomes of event sets that have been designed with specific documented success criteria unique to that scenario. In addition, the evaluator will assess the technical and CRM skills of the flightcrew against relatively standard global criteria that apply to all evaluation events (e.g., the aircraft must land safely).

(10) All crewmembers will use headsets and emergency breathing equipment as required in line operations.

2-11. CRM COMPONENT OF THE PRE-LOS BRIEFING.

a. Reverse Briefing. In addition to establishing the rules for the conduct of LOS, the LOS briefing should include a reverse briefing on CRM factors affecting crew performance. The concept of reverse briefing is to elicit information from the crew by encouraging them to brief themselves, helping to determine their level of expertise. Reverse briefing makes the crew active participants rather than passive recipients of briefings on issues already understood. Questions using the CRM behavioral markers (refer to the current edition of AC 120-51, Crew Resource Management Training, Appendix 1), mentioned in subparagraphs b, c, and d below, can effectively elicit the level of understanding that the crew has about human factors and technical proficiency issues.

b. Crew-Oriented Briefing. The crew could be asked to discuss the conduct and quality of an effective crew-oriented briefing. Crew performance is highly associated with the quality of the initial crew briefing. The following are examples of opening questions that may help to initiate the discussion:

(1) What can be said to create an atmosphere for establishing the team concept and open environment within the flight deck and with the cabin crew?

(2) What are the components of a briefing that is operationally thorough and interesting and addresses coordination, planning, and problems?

(3) Overall responsibility is primarily a captain’s function; what are the responsibilities of the other crewmembers and how can they add significantly to planning and definition of potential problem areas?

(4) What can be done to make cabin crewmembers feel they are part of the team? The importance of the crew briefing cannot be overstated.

c. Communication. Questions that encourage crews to consider communication issues include:

(1) How does the crew view inquiry and advocacy?

2-11. CRM COMPONENT OF THE PRE-LOS BRIEFING. › g. Summary. These are just a few of the issues that can be addressed in…

(2) To what extent should crewmembers advocate a course of action they feel is best, even when it involves conflict and disagreements with others?

(3) What is their feeling towards the relationship between inquiry and advocacy and the captain’s authority?

(4) How do they define the proper balance between authority and assertiveness?

(5) What are the indications that a crew is concerned with the effective accomplishment of necessary tasks?

(6) Can they give examples where poor workload management or the lack of situational awareness has contributed to accidents or incidents?

(7) Does casual social conversation during periods of low workload indicate a lack of vigilance?

(8) What can be done to avoid overloading individual crewmembers?

d. Technical Proficiency. The relationship between CRM and technical proficiency is a rich area for crew discussion. Some important questions to ask include:

(1) What is their understanding of the relationship between technical proficiency and CRM?

(2) Can CRM overcome a lack of technical proficiency or vice versa?

e. Crew Coordination. LOS assumes knowledge of systems and an understanding of, and proficiency in, skills involving procedures and techniques. Training programs have always been concerned with developing the specialized skills required to be technically proficient crewmembers. However, how well the entire crew discharges the technical aspects of the flight reflects awareness that a high degree of technical proficiency is essential for safe and efficient operations. In the briefing, it must be made very clear that demonstrated mastery of CRM concepts cannot overcome a lack of proficiency, but just as importantly, high technical proficiency cannot guarantee safe operations in the absence of effective crew coordination.

f. Self-Critique. Also useful is a discussion of the crew attitudes toward self-critique. What is their understanding of critique? Do they see any benefit in reviewing positive behavior? Have the crewmembers used critique on line operations? When do they feel critique is appropriate?

g. Summary. These are just a few of the issues that can be addressed in an LOS briefing. The proper briefing will reinforce CRM principles and technical procedures learned during initial and recurrent training and qualification and continuing qualification training. Without a proper briefing, LOS becomes a full mission simulation without a focus. Though it can be a positive learning experience, it may easily become centered on individual technical proficiency and abnormal checklist usage.

2-13. PREFLIGHT ACTIVITIES. › a. Documentation. LOS facilitators will provide the crew with complete…

2-12. LOS CRM BRIEFING AND CREW ORIENTATION.

a. Briefing and Orientation. A thorough LOS CRM briefing provides the following:

(1) Establishes an environment for open and interactive communication (e.g., the facilitator calls for questions or comments, answers questions directly, listens with patience, does not interrupt or talk over crewmembers, does not rush through the briefing, and makes eye contact as appropriate).

(2) Is interactive and two-way, and emphasizes the importance of questions, critique, and the offering of information.

(3) Sets the agenda, outlines expectations, and establishes a team concept.

(4) Covers pertinent safety and operational issues.

(5) Identifies potential problems such as weather, delays, and abnormal system operations.

(6) Provides guidelines for crew actions; addresses division of labor and crew workload.

(7) Sets expectations for how deviations in flight training simulation device (FSTD) performance and mechanical problems are to be handled.

b. The LOS Facilitator. The briefing that includes CRM issues will also give direction to the LOS facilitator’s conduct of the LOS. It will help to focus the facilitator’s observations on the operator’s tailored CRM behavioral markers (e.g., situational awareness, communication, decision making, workload management, automation management, and team-building) that will later be highlighted in debriefing. This may lower the LOS facilitator’s workload by allowing attention to be directed to a few key CRM categories, rather than having to monitor for all categories continuously.

c. Summary. The briefing should prepare the crew for an effective training experience. A good briefing is operationally thorough and interesting, and will provide an overview of the overall LOS. Effective facilitators create the appropriate training environment and demonstrate their own commitment to LOS. The crew will be prepared to participate in an authentic simulation of the line operations and the crew debriefing following the training or evaluation.

2-13. PREFLIGHT ACTIVITIES.

a. Documentation. LOS facilitators will provide the crew with complete flight planning documentation. An effort should be made to duplicate, as closely as possible, the preflight and dispatch process. The weather reports, Weight and Balance (W&B), and other documents should be the same as those provided prior to line flights, and include the following LOS planning and preparation documents:

2-15. DEBRIEFING THE LOS. › a. Debriefing Manner. After the LOS is completed, the manner in which…

• Dispatch or flight release with center-stored flight plan and flight plan analysis. • W&B, loading, and fuel loading instructions. • Weather reports and forecasts. • Notices to Airmen (NOTAM). • Performance data sheets. • Inbound maintenance log sheets signed off. • MEL placards. • Company documents.

b. Flightcrew Preparation. The flightcrew should be in the FSTD early enough to allow adequate time for the crew to perform a normal flight deck preflight setup. If it is customary for the Flight Engineer (FE) to enter the flight deck before the captain and first officer, that sequence should be adhered to. However, in the interest of saving time, it is possible to modify the scenario to provide shorter ground times, as on a through flight. A planned departure time toward which all preparations can be directed helps to ensure that these activities are performed efficiently, and also helps to enhance the realism of an LOS scenario.

c. FSTD Interference. Certain FSTD problems that cause interference with the realism associated with LOS can occur. If a component required for a given scenario is inoperative, that scenario should not be flown. However, minor FSTD malfunctions can be placarded, in accordance with MEL procedures, just as the maintenance crew would do on the line. If an actual equipment failure occurs in flight, and it is consistent with failures that could occur in an aircraft, the scenario can proceed, with modification if necessary.

2-14. CREWMEMBER RESPONSIBILITIES. Crewmembers’ duties include:

• Performing their normal flight duties. • Using avionics equipment as they would normally do during flight. Radio frequencies must be changed as required. • Being natural in character and operation. They should not be inhibited or try to operate in a manner calculated to give the Academy solution or to please the LOS facilitator. • Planning the flight as one would a real line flight, with any service the company or ATC normally provides available to the crew. • Performing all normal procedures and communications, such as final weight checks, departure reports, and in-range reports. • Using headsets and emergency breathing equipment, to the same extent as required in line operations.

2-15. DEBRIEFING THE LOS.

a. Debriefing Manner. After the LOS is completed, the manner in which the debriefing is handled by the facilitator is of key importance if CRM skills are to be reinforced and improved. The facilitator should not handle the debrief in a teacher-tell manner. Instead, the facilitator should operate as a resource to crewmembers by highlighting different portions of the LOS that may be suitable for review, critique, and discussion. The discussion should be led by the crewmembers themselves, using the facilitator and, if the scenario is recorded, the recording as

c. Self-Criticism. Frequently, crews are more critical of themselves… › (9) Ask the crew, and individual crewmembers, for specific feedback on…

resources during their critique. Handled in this way, crew-led debriefs may occur with increasing frequency on the line after a difficult segment, or in other cases where crew critique and review is appropriate.

b. Debriefing Topics. Because the focus of LOS is on the integration of CRM skills into the technical skills normally assessed in flight training, the LOS debriefing session will concentrate on this area. Key items for discussion include workload management, crew coordination, crew communications, and technical proficiency. The use of systems and other resources are other areas for attention. The discussion should include the crew’s use of ATC and company communications; manuals, charts, and software; the use of other crewmembers; and the use of autopilot, autothrottle, and other potential workload-reducing devices. It is the facilitator’s responsibility to ensure that these items are fully explored during the debriefing sessions.

c. Self-Criticism. Frequently, crews are more critical of themselves than the facilitator would ever be. Self-criticism and self-examination are almost always present in these situations, and in many cases they are much more effective than facilitator criticism. Thus, the facilitator should do everything possible to foster this sort of self-analysis. In the role of moderator, the facilitator can guide the discussion to areas that he or she has noted. Questions about certain procedures, decisions, and mistakes should be asked. However, the facilitator should minimize lectures about what is right and wrong. Obviously, the facilitator should avoid embarrassing the crewmembers. In order to be effective, the facilitator should use the following guidelines:

(1) Actively state the debriefing and critique agenda and solicit agenda topics from the crew on items they would like to cover; set time limits.

(2) Ask the crew for their overall self-appraisal of the flight.

(3) State own reaction to the LOS in an objective and performance-oriented way. Actively guard against making the crew defensive.

(4) Highlight key incidents and examples from the recording that include technical as well as CRM performance examples. Select material for discussion that illustrates key behaviors using the company’s behavioral markers. When using a recording, show only enough material to make the point.

(5) Effectively integrate technical and CRM feedback into the debriefing. Do not preach to the flightcrew, and do not gloss over items worthy of discussion.

(6) Exercise patience, and do not be reluctant to probe into key areas where individual and/or crew improvement is needed.

(7) Ensure that all crewmembers participate in the discussion, and effectively draw out quiet or hostile crewmembers.

(8) Provide a clear summary and recap of key learning points.

(9) Ask the crew, and individual crewmembers, for specific feedback on their performance.

CHAPTER 2. BASIC ELEMENTS OF LINE-OPERATIONAL SIMULATION › 2-18. ADDITIONAL TRAINING/LOFT AND SPOT COMPLETION.

(10) Be effective in both technical and CRM debriefing.

d. Crew and Individual Debriefing. During debriefing, the facilitator should openly discuss and assess crew performance and individual performances. The facilitator can mention constructive assessment of an individual in the presence of the full crew.

e. Debriefing Goals. One of the goals of LOS is to enable crewmembers to gain a greater understanding of their behaviors and their consequences, and be able to explore new behavioral strategies in an LOS training environment where formal, mandated evaluation is explicitly omitted. The debriefing should respect this goal and build on it to provide a positive learning experience.

f. Summarize the Debriefing. At the appropriate time, the facilitator should summarize the debriefing. In the summary, every effort should be made to relate the training experience to line operations. It is most desirable if the crewmembers recognize, for themselves, behaviors used in the LOS that they can carry back to the line, as feedback or critique is seldom used on the line. The LOS debriefing can help reinforce the importance of feedback even on routine line flights. Just a few minutes are needed to reinforce what went well or to discuss ways to improve crew performance at appropriate times during or at the conclusion of the flight.

g. Summary. In summary, the effective LOS facilitator will lead the crewmembers through self-critique of their performance. The debriefing and crew analysis period will include both technical and CRM discussion items. Positive points of crew performance as well as areas for improvement will be discussed. At the conclusion of the session, key learning points will be summarized.

2-16. LOE. The LOS briefing/debriefing guide must be modified for the LOE administered under an approved AQP. In the LOE, the facilitator is now an evaluator and must perform a different role. This role is to evaluate the standard performance of the proficiency objectives assigned to the event sets. Although the briefing will set the stage for the LOE, most carriers use this period to perform an oral review of crew knowledge concerning the operational issues presented in the LOE. For example, the briefing might cover issues such as takeoff visibility and required alternates based on operations specifications (OpSpecs). The debriefing is used to review the event sets and compare the success criteria assigned to these sets versus actual crew performance. However, in this review, there still will be many opportunities for crewmembers to discuss their CRM and technical performance.

2-17. USE OF AUDIOVISUAL EQUIPMENT AND ANIMATION SOFTWARE. Although not required, recorded audiovisual/animation feedback is very useful as a debriefing aid for most types of LOS. It allows crewmembers to view themselves from a third-person perspective. This feedback helps crewmembers to better understand their performance, identify and accept their weak areas, and build upon their strong areas, thereby encouraging positive changes in attitudes and behavior. Recorded audiovisual feedback should be destroyed at completion of the debriefing.

2-18. ADDITIONAL TRAINING/LOFT AND SPOT COMPLETION. Decisions that produce unwanted results do not indicate a training failure, but serve as a learning experience

2-20. FAA PHILOSOPHY. › c. Substitutions. The training value of LOS can be seriously diminished…

that may indicate need for additional training. The additional training could come in many forms, including additional LOS. In any case, required additional training shall be provided and documented as satisfactorily completed prior to the crewmember’s return to line operations.

2-19. BASIC ELEMENTS OF LOS SUMMARIZED. LOS is defined by the following basic concepts:

• It takes place in a simulated line operational environment. • It uses a complete crew with total participation. • It contains real-world incidents, unfolding in real time. • LOFT and SPOT are used for no-jeopardy training, while LOE is used for evaluation. • LOFT and LOE contain scenarios and segments that run uninterrupted; SPOT can be interrupted and segments interactively accomplished. • It contains scenarios tailored to the operator’s learning objectives. • It incorporates CRM skills. • It provides critique of individual and crew performance.

2-20. FAA PHILOSOPHY.

a. Effectiveness. The effectiveness of LOS is dependent on four important aspects.

(1) Use of the most appropriate FSTD.

(2) Ensuring that training and evaluation are conducted to the maximum extent feasible using a full crew complement, consisting of a pilot in command (PIC), a second in command (SIC), and, where applicable, an FE.

(3) LOFT or LOE scenarios must run their full, uninterrupted course.

(4) A variety of scenarios, fully compatible with training objectives, are available and periodically updated to ensure that the LOS experience does not become repetitive or stale.

b. Choice of FSTD. An operator who has an available range of training media will conduct LOS in the FSTD that provides the appropriate simulation fidelity for the training/evaluation objectives. A major consideration for public safety is the maintenance of flightcrew member skills, particularly for tasks that, due to the nature of a given fleet’s operations or to their low probability of occurrence, are rarely executed in normal flight operations. Providing interim practice on such skills using lower-level devices may be one way to maintain these skills. Although the focus of LOS has been on the development of a methodology and tool set to generate valid and reliable LOS scenarios for training and evaluation in full flight simulators (FFS), the methodology also has important capabilities for use in scenario-based training using equipment of lower physical fidelity.

c. Substitutions. The training value of LOS can be seriously diminished when inappropriate crew substitutions are made. Operators should not schedule any person other than line-qualified crewmembers for recurrent LOFT. For qualification LOFT, operators should schedule only line-qualified crewmembers or those crewmembers that are in training for a

CHAPTER 2. BASIC ELEMENTS OF LINE-OPERATIONAL SIMULATION › 2-23. CRM PRACTICAL APPLICATIONS.

particular flightcrew member duty position. In both cases, operators should make every reasonable effort to meet these scheduling guidelines. When, due to reasons beyond the control of the operator, the need for substitution arises, the substitution tables in this AC may be used. However, these tables should be used only after the operator has made all reasonable efforts to provide a substitute crewmember of equal status to the person originally scheduled. LOE substitution tables will be included in the approved AQP documentation.

d. Interruptions. Interruption of LOFT scenarios is detrimental to the learning experience. Arbitrary interruption of LOFT or LOE is not acceptable. LOFT and LOE event sets should be allowed to continue to their logical conclusion. In qualification LOFT, if the instructor is certain that negative training is occurring, the scenario may be interrupted, but only in extreme situations. The FAA believes that well-thought-out and properly developed scenarios will not often lead to situations that require interruption.

e. Summary. Proper planning, development, approval, and execution of LOS scenarios are essential to ensure that training objectives are met in promotion of safety and the public interest. Training objectives must be carefully determined in a manner that gives consideration to operator- and/or FAA-identified hazards and provides training in controlling the resultant risks. Moreover, the promotion of safe operations requires both the training and evaluation of all crewmembers to continually identify, avoid, and manage operational risks. These are critical characteristics of any LOS program and a requirement of Safety Management Systems (SMS). Training value is diminished when students become familiar with scenarios. Therefore, a variety and a sufficient number of LOS scenarios are required to guard against crewmembers experiencing repetitious situations. In addition, operators should regularly update, at least annually, all LOS scenarios currently in use, thereby ensuring that crewmembers are exposed to new technology, procedures, and current operational problems.

2-21. SAFETY MANAGEMENT SYSTEMS (SMS). SMSs provide a formal, top-down business approach to managing safety risks, which includes a systemic approach to managing safety. SMSs set forth all necessary organizational structures, accountabilities, policies, and procedures (refer to the current edition of AC 120-92, Safety Management Systems for Aviation Service Providers). In the execution of an SMS, the general design expectations for any participating air operator requires that training be accomplished as an integral part of the complete scope and life cycle of the organization’s safety system.

2-22. CRM/THREAT AND ERROR MANAGEMENT (TEM)/RISK AND RESOURCE MANAGEMENT (RRM). Conceptual frameworks for integrating the traditional CRM markers and skill sets, such as TEM and RRM, provide an additional layer of safety by providing strategies and toolkits for the appropriate deployment of CRM skills in the operational environment. They may also be used as classifications to record and respond to risks in the operational environment, in direct support of an air operator’s SMS. As such, they bring risk assessment and management tools to the flight deck, the cabin, and the dispatch desk.

2-23. CRM PRACTICAL APPLICATIONS. The following is a sampling of threats/compounding conditions that flightcrews may encounter:

a. Behavioral Markers. Behavioral markers are CRM-observable,… › (6) The following is a sampling of operator-developed, CRM human…

a. Expected Risks/Threats/Compounding Conditions. Thorough briefings are used for managing expected risks. They include a planned course of action for anticipating how the crewmembers will mitigate each risk.

b. Unexpected Risks/Threats/Compounding Conditions. Crewmembers react to these types of risks as they occur, using their CRM skills acquired from past experiences and training. The goal is to give crewmembers a CRM process to proactively handle these threats and manage associated errors.

c. Time Risks/Threats/Compounding Conditions. The majority of risks during flight are time risks. Examples include, but are not limited to, most aircraft system malfunctions, deteriorating weather conditions, or runway changes. Misdiagnosing a time risk and/or rushing through a plan to manage time risks are among the most common errors that crewmembers make.

d. No-Time Risks/Threats/Compounding Conditions. These include unexpected risks that are time critical and usually require the crew to land as soon as safely possible. These types of risks are best managed when the PIC has an alternative plan of action (plan B) in mind before an actual event occurs. No-time risks are sometimes aircraft specific. Examples of no-time risks may include, but are not limited to, fire of unknown origin, smoke from an unknown source, or a security issue. Having a plan B suggests that each PIC should have a plan of action in mind before an actual no-time event occurs.

2-24. CRM (HUMAN FACTORS) BEHAVIORAL MARKERS.

a. Behavioral Markers. Behavioral markers are CRM-observable, nontechnical behaviors that contribute to flightcrew and cabin crew performance.

(1) Each CRM behavioral marker is specifically developed by the operator, as derived from data analysis and structured into a set of human factor categories.

(2) CRM behavioral marker lists are used to plan, script, train, and evaluate human factor proficiency requirements.

(3) CRM behavioral markers are utilized during training, LOS, Line Operations Safety Audits (LOSA), and other line-related auditing activities, as they pertain to nontechnical, human factor CRM skills.

(4) Behavioral markers reduce instructor workload by allowing attention to be directed on a few key CRM categories, rather than having to monitor all aspects of human factors continually.

(5) Once developed, each operator should maintain a behavioral marker consistency throughout the CRM training, evaluating, and auditing processes.

(6) The following is a sampling of operator-developed, CRM human factors observable behavioral markers:

b. Observable Skills. The practical application and evaluation of CRM… › (5) Automation Management.

Par 2-24 Page 21 (and 22)

• Situational awareness. • Communication. • Decision making. • Workload management. • Automation management.

b. Observable Skills. The practical application and evaluation of CRM behavioral observable skills may be demonstrated by crewmembers, as scripted around events, event sets, and LOS themes.

(1) Situational Awareness. The closer the crewmember’s mental model is to reality, the better his or her situational awareness. Accessing a variety of resources and using good communication skills can raise situational awareness.

(2) Communication. Good communication includes the use of inquiry, advocacy, and assertion.

(3) Decision Making. In the decision-making process, crewmembers expand their CRM team (deadhead crew, dispatch, maintenance, etc.) and use onboard resources (e.g., quick reference handbooks (QRH) and manuals), to assist in formulating a plan. The plan is communicated, as necessary. Decision making includes determining the criticality of the risk (time or no-time).

(4) Workload Management. In essence, workload management is time management. Crewmembers can create time to reduce task saturation by slowing down (e.g., requesting vectors and holding). Fuel remaining must be considered during this process. A crewmember should not risk task saturation by attempting to multitask (e.g., flying the aircraft, responding to ATC, and accomplishing non-normal checklists). Use of effective workload management divides crewmember duties. Examples are:

(a) Pilot Flying (PF). Uses an appropriate level of automation to reduce workload in times of task saturation. Situational awareness is maintained by periodic communication with the pilot monitoring (PM). The PM is involved if flying difficulties are encountered.

(b) Pilot Monitoring (PM). Maintains constant situational awareness. Prioritizes risks. Manages the highest risk by expanding the CRM team (if needed) for inputs. A plan is formulated, communicated, and tested for reception.

(5) Automation Management. Automation management is the use of appropriate flight guidance to manage crew workloads and maintain situational awareness. Automation management is consistent with an operator’s policies and procedures.

CHAPTER 3. TYPES OF LINE-ORIENTED FLIGHT TRAINING

3-3. GUIDELINES FOR RECURRENT LOFT. › f. Training Media. The highest fidelity full flight simulator (FFS)…

3-1. GENERAL. There are two types of Line-Oriented Flight Training (LOFT): recurrent LOFT and qualification LOFT. Guidelines for designing and conducting these types of LOFT are presented below.

3-2. RECURRENT LOFT. Recurrent LOFT is designed to ensure that each crewmember maintains proficiency in the type of aircraft and flightcrew member duty position involved. Recurrent LOFT is intended for flightcrew members who are presently qualified in a particular make, model, and series (M/M/S) aircraft. (Refer to part 121, §§ 121.409, 121.427, 121.433, and 121.441.)

3-3. GUIDELINES FOR RECURRENT LOFT. Recurrent LOFT should meet the following guidelines:

a. Instruction or Interruption. Recurrent LOFT does not permit direct instruction and does not permit interruption of the scenario by the instructor, except for the nondisruptive acceleration of uneventful en route segments.

b. Crew Substitutes. Recurrent LOFT must utilize a complete flightcrew, who should be line qualified. The use of substitutes is discouraged, and substitution should be rare. When the composition of the scheduled line-qualified flightcrew cannot be maintained, the operator may use substitutions based on the guidelines in Table 3-1, Recurrent LOFT Substitution Table. However, the operator will first attempt to substitute with another line-qualified flightcrew member. This table should be used only as a last resort, to prevent interruption of scheduled training.

c. Number of Segments. Recurrent LOFT must include at least two flight segments that are representative of the operations conducted by the operator. However, two gate-to-gate operating cycles are not necessary. The flightcrew completing recurrent LOFT must perform at least one taxi-out and one taxi-in. Additional segments need only consist of takeoff, climb, en route, descent, and landing.

d. Stall Prevention Training (Approach to Stall). Recurrent LOFT must include scenario-based or maneuver-based stall prevention training (approach to stall) before, during, or after the LOFT scenario for each pilot.

e. Scenario. Recurrent LOFT must provide an opportunity for each pilot to demonstrate workload management and pilot monitoring skills. Recurrent LOFT must also include at least the maneuvers and procedures (abnormal and emergency) that may be expected during line operations (refer to AC 121-42, Leadership and Command Training for Pilots in Command, for additional requirements for recurrent LOFT).

f. Training Media. The highest fidelity full flight simulator (FFS) available should be scheduled for recurrent LOFT.

a. Crew Composition. Qualification LOFT requires a complete crew… › 2An instructor (aircraft or simulator) as provided for under part 121,…

TABLE 3-1. RECURRENT LOFT SUBSTITUTION TABLE

Pilot in Command (PIC)

Position

Second in Command

(SIC) Position

Flight Engineer (FE)

Position 1. PIC1 SIC1 FE1 2. SIC2 PIC1 FE Instructor3 3. Pilot Instructor3 Pilot Instructor3

1Includes those who are either line qualified or in training and are line and task familiar with the position in which they are substituting.

2An SIC may be substituted for this position if the pilot holds a type rating in the aircraft the flight simulation training device (FSTD) replicates.

3An instructor (aircraft or simulator) as provided for under part 121, § 121.412 or part 135, § 135.338, as applicable. The instructor should not have previous knowledge of the scenario; however, when this is unavoidable, the instructor should not use that knowledge to influence or direct the scenario.

NOTE: The instructor conducting the Line-Operational Simulation (LOS) session will not act as a substitute crewmember.

3-4. QUALIFICATION LOFT. Qualification LOFT is designed for crewmembers whose training has been provided using advanced simulation. Qualification LOFT provides training that facilitates the transition from FSTD training to operational flying. Scenarios are designed to represent typical flight segments (refer to part 121 appendix H).

a. Crew Composition. Qualification LOFT requires a complete crew complement. It is preferable to schedule a crewmember who is qualifying with other crewmembers who are fully line qualified. As a minimum, LOFT crewmembers will be task familiar with their assigned flightcrew member duty position. The use of substitutes is highly discouraged and substitution should be implemented rarely. When the composition of the scheduled crew cannot be maintained, the operator may substitute crewmembers using Table 3-2, Qualification LOFT Substitution Table.

TABLE 3-2. QUALIFICATION LOFT SUBSTITUTION TABLE

Pilot in Command (PIC)

Position

Second in Command

(SIC) Position

Flight Engineer (FE)

Position 1. PIC1 SIC1 FE1 2. SIC1 PIC1 FE Instructor2 3. Pilot Instructor2 Pilot Instructor2

1Includes those who are either line qualified or in training and are task familiar with the position in which they are substituting.

2An instructor (aircraft or simulator) as provided for under part 121, § 121.412 or part 135, § 135.338, as applicable. The instructor should not have previous knowledge of the scenario; however, when this is unavoidable, the instructor should not use that knowledge to influence or direct the scenario.

3-4. QUALIFICATION LOFT. › c. Training Media. Qualification LOFT will be conducted in a Level C or…

Par 3-4 Page 25 (and 26)

NOTE: The instructor conducting the LOS session may not act as a substitute crewmember.

b. Scenario and Number of Segments. Qualification LOFT must consist of at least two flight segments representative of the operations conducted by the operator. One segment must contain strictly normal operating procedures from pushback at one airport until arrival at the gate at another airport. Another flight segment must contain appropriate abnormal and emergency operations. Qualification LOFT must provide an opportunity for each pilot to demonstrate workload management and pilot monitoring skills.

c. Training Media. Qualification LOFT will be conducted in a Level C or higher FFS (refer to part 121 appendix H).

CHAPTER 4. SPECIAL Purpose OPERATIONAL TRAINING › 4-3.

Par 4-1 Page 27 (and 28)

CHAPTER 4. SPECIAL PURPOSE OPERATIONAL TRAINING

4-1. GENERAL. Special Purpose Operational Training (SPOT) is designed for training crewmembers in a flight simulation training device (FSTD). SPOT is useful whenever coordinated crew performance is required. It may not be substituted for recurrent Line-Oriented Flight Training (LOFT) or qualification LOFT. SPOT may include training which:

• Focuses on Crew Resource Management (CRM) skills. • Provides differences training on variant aircraft. • Provides specific phase of flight training. • Trains in special aircraft equipment (e.g., navigational equipment and flight management system (FMS)).

4-2. GUIDELINES FOR SPOT. The components of SPOT vary, depending on the purpose or objective of the training. Therefore, the following provides only general guidelines for SPOT.

a. Instruction and Interruption. SPOT permits direct instruction and allows for interruption of the scenario by the instructor.

b. Crew Composition. SPOT may include use of a complete or partial crew, depending upon the training objectives.

c. Crew Substitutes. The use of crew substitutes in SPOT depends upon the type of training being provided.

d. Number and Type of Segments. SPOT may contain any number of full or partial flight segments, depending upon the training objectives.

e. Training Media. SPOT may use a wide range of FSTDs, depending upon the training objectives.

4-3. SPECIFIC SCENARIOS FOR OPERATORS OF ALL-CARGO AIRCRAFT OR COMBINATION CARGO/PASSENGER (COMBI) AIRCRAFT. Operators of all-cargo or combi aircraft should conduct SPOT to provide pilots training on situations unique to all-cargo and combi operations, such as:

• Indications of improper loading during preflight, taxi, and initial takeoff roll. • Indications of cargo shifting during taxi and initial takeoff roll, resulting in an aborted takeoff. • Indications of cargo shifting, leading to excessive pitch changes during takeoff and initial climb. • Unanticipated pitch mistrim during takeoff, resulting in an aborted takeoff.

CHAPTER 5. LINE OPERATIONAL EVALUATION › 5-5. LOE MEDIA.

Par 5-1 Page 29 (and 30)

CHAPTER 5. LINE OPERATIONAL EVALUATION

5-1. GENERAL. The Line Operational Evaluation (LOE) is the primary means of proficiency evaluation in an Advanced Qualification Program (AQP). This evaluation addresses the individual’s ability to demonstrate technical and Crew Resource Management (CRM) skills appropriate to fulfilling job requirements in a full mission-scenario environment. The intent of an LOE is to evaluate and verify that an individual’s job knowledge, technical skills, and CRM skills are commensurate with AQP qualification standards. The LOE is conducted in a flight simulation training device (FSTD) qualified in accordance with part 60 and approved for its intended use in the AQP. LOE development methodology, substitution tables, success criteria, and remediation strategy will be included in the carrier’s approved AQP documentation.

5-2. ELEMENTS RESEMBLING LINE-ORIENTED FLIGHT TRAINING (LOFT). LOE contains elements similar to those in LOFT (e.g., line environment, complete crew, real-world scenarios, real time, and must run uninterrupted). A complete crew complement should be scheduled and maintained. Flightcrew member substitution is highly discouraged. If crew substitutions are necessary, the substitute crewmember will be either another line-qualified crewmember or a task-familiar crewmember in a training status comparable to the person being evaluated. Evaluators conducting the LOE may not serve as a substitute crewmember. The LOE substitution table/matrix will be part of the carrier’s approved AQP documentation.

5-3. EVALUATION. The LOE addresses the individual’s ability to demonstrate technical and CRM skills appropriate to fulfilling job requirements in an operational environment. The intent of an LOE is to evaluate and verify that an individual’s job knowledge, technical skills, and CRM skills are commensurate with AQP qualification standards. Part 121, § 121.907 requires that the LOE evaluate both CRM and technical skills in a scenario which is designed to test both. While encouraged and widely practiced, it is not mandatory that CRM be graded on a pass/fail basis. The alternative is to assume that failed CRM performance will, in almost all cases, drive a technical failure. That technical failure will then be recorded with CRM annotated as a contributing factor. This still allows CRM shortfalls to be documented and remediated.

5-4. EVALUATOR. An evaluator must have satisfactorily completed the certificate holder’s AQP evaluator training. The Administrator must approve all LOE evaluators. It is essential that LOE evaluator training include specific exercises to achieve and verify standardization among such personnel in grading performance.

5-5. LOE MEDIA. Operators conducting LOE may be approved to use any level of FSTD, depending on the objective of the evaluation, the capability of the FSTD, and compliance with part 60.

6-2. FACILITATOR RESPONSIBILITIES. › c. Simulation. Of vital importance to the effectiveness of LOS is the…

CHAPTER 6. THE ROLE OF LINE-OPERATIONAL SIMULATION FACILITATORS

6-1. MINIMUM QUALIFICATIONS. Line-Operational Simulation (LOS) facilitators/instructors/evaluators should be trained in the philosophy, skills, and conduct of LOS and Crew Resource Management (CRM). They should be able to effectively observe and critique both individual and crew performance during the scenario. To do this, they should meet the minimum requirements discussed in the following paragraphs.

a. Line Familiar. Facilitators should be line familiar. This will ensure that facilitators accurately perceive and evaluate situations as they arise.

b. Qualified as Instructors/Evaluators. To conduct Line-Oriented Flight Training (LOFT) or Special Purpose Operational Training (SPOT), facilitators should be qualified as defined in part 121, §§ 121.412 and 121.414 and appendix H; or part 135, §§ 135.338 and 135.340, as applicable, and as required by the approved Advanced Qualification Program (AQP), if appropriate. To conduct Line Operational Evaluation (LOE), facilitators should be qualified as defined in §§ 121.411 and 121.413 and part 121 appendix H; or §§ 135.337 and 135.339, as applicable, and as required by the approved AQP.

c. Trained in CRM Skills. Facilitators will receive training in CRM skills in order to observe and critique these areas in LOS. Refer to the current edition of Advisory Circular (AC) 120-51, Crew Resource Management Training, for further information.

d. Trained in Methods for Briefing, Debriefing, and Critique. Facilitators should be trained to conduct the briefing and debriefing/critique phases of LOS, including how to provide effective feedback.

6-2. FACILITATOR RESPONSIBILITIES. The following is a description of the roles and responsibilities of facilitators:

a. Briefing and Preparation. Facilitators should be able to effectively convey the purpose of the LOS and how it is representative of line operations. Facilitators should also explain their role as observers during the training/evaluation, in that they are not considered present unless playing a non-flightcrew member role in the scenario (e.g., air traffic control (ATC), flight attendant (F/A), or dispatch). This does not apply to SPOT, which allows for an interactive role on the part of the facilitator.

b. Flight Segment. Facilitators should be able to observe and perform ancillary roles. They should be trained in observing and assessing technical and CRM skills. Facilitators should also be trained in proper pacing, proper introduction of abnormal/emergency procedures, and methods of handling unforeseen crew actions.

c. Simulation. Of vital importance to the effectiveness of LOS is the creation of a strong illusion of reality in the simulated flights. This requirement dictates that many routine activities, such as flight paperwork, manuals, and communications should be carefully prepared. Previous experience with LOFT and LOE has shown that overlooking these activities can destroy this illusion.

6-2. FACILITATOR RESPONSIBILITIES. › h. Debriefing and Critique. Facilitators should provide both positive…

d. Resources. The facilitator’s goal is to produce crew performance and behavior that is typical for an actual line flight in the same set of circumstances as those developed in the scenario. In keeping with this goal, it is essential that crews have access to all the resources they would have during an actual line flight. The briefing should include mention of the roleplaying aspect of LOS and its importance to overall LOS effectiveness.

e. Facilitator Role. The role of the facilitator in LOFT or LOE should be viewed as that of communicator, observer, and moderator in the debriefing process. He or she is not an instructor in the traditional sense during the flight simulation training device (FSTD) period. He or she is the facilitator or manager of the flight, using appropriate radio calls or responses to direct the flight along the desired path. The facilitator must be prepared to accept and manage alternate courses of action that the crew may wish to follow. The facilitator should remain as unobtrusive as possible within the physical limitations of the FSTD. He or she should resist the temptation to instruct, and must not intrude in any way into the situation.

f. Communication. All communications must be conducted in the manner normally found on a line flight; that is, via radio from outside the “aircraft;” via interphone or normal conversations between flight deck crewmembers; or, in the case of communication between the flight deck and cabin, via the usual aircraft equipment for this purpose. All external communications (e.g., ATC and ground crew) must be credible and realistic.

g. Recorded Feedback. The entire FSTD phase of the flight, including initial flight deck setup, should be recorded, if the equipment is available. The importance of the correct use of recording cannot be overstated: LOS with recording is one of the most powerful tools we have for reinforcing desirable behavior in CRM. During debriefing, the recording should be reviewed and discussed by the flightcrew with emphasis being placed upon crew performance, including their use of CRM behavioral markers and skills. When crewmembers have learned and can appreciate the importance of open and direct critique for purposes of operational review and analysis, a platform is in place for effective post-LOS discussion that reviews more than stick-and-rudder skills or systems knowledge. Following review of the recording, it may be erased.

h. Debriefing and Critique. Facilitators should provide both positive and negative feedback during critiques of individual and crew performance. Prior to the facilitator’s critiques, crewmembers should be encouraged to critique themselves. Facilitators will provide feedback to the crew to encourage the changes needed for improved performance. Facilitators should also provide specific recommendations to improve individual crewmembers’ performance.

7-2. PHILOSOPHY OF LOS DESIGN AND CONDUCT. › d. Level of Difficulty. One misconception is the belief that LOS…

CHAPTER 7. LINE-OPERATIONAL SIMULATION SCENARIO DESIGN

7-1. SCENARIO DESIGN METHODOLOGIES. This chapter provides a Line-Operational Simulation (LOS) scenario design methodology that has proven effective. There may be other, equally effective LOS development methods. The important aspect of LOS development is that a disciplined methodology is employed.

7-2. PHILOSOPHY OF LOS DESIGN AND CONDUCT.

a. Scenarios. LOS scenarios are best designed to be operationally relevant, believable, and good tests of the crew’s technical and Crew Resource Management (CRM) skills. LOS training is systematic and is intended to simulate actual problem situations on the line that require good crew skills for effective resolution and decision making.

b. Line Realism. Because LOS requires as much realism as possible, LOS design and evaluation guidelines should maximize line realism. Preflight activities and detailed review of flight paperwork, manuals, and conduct of communications should be included in the scenario. This requirement does not preclude employing scenarios that use short segments beginning or ending in an en route environment, if the objectives of the LOS can be met. If the scenario is designed to begin in an en route environment, enough quiet time should be present for the crew to become acclimated to the flight routine. These en route segments are identified as Special Purpose Operational Training (SPOT). Line Operations Safety Audit (LOSA) data demonstrate that a flightcrew addresses 40 percent of the risks encountered during a flight prior to takeoff. Therefore, SPOT should not be employed simply to shortcut this phase of flight, unless the objectives of the scenario warrant it. An operator should not change the name of a Line-Oriented Flight Training (LOFT) to a SPOT simply to avoid the requirement for FAA approval.

c. Open Communication. LOS scenarios should be designed to foster an environment where free and open communication is practiced. This encourages crewmembers to provide necessary information at the appropriate time (e.g., initiating checklists, advocating positions, and problem definition). Furthermore, the LOS design should encourage active participation in the decision-making process and questioning of actions and decisions by all crewmembers.

d. Level of Difficulty. One misconception is the belief that LOS training should continuously increase crew operational workload until the crew becomes overloaded. This is not the purpose or intent of LOS and can actually help to defeat its effectiveness. The difficulty of the LOS should not be designed to saturate a flightcrew or impose an unrealistic level of difficulty or complexity. On the other hand, the LOS must provide enough difficulty to adequately test the flightcrew’s skills and capabilities. LOS scenarios are most effective if they are straightforward. For example, choosing a departure airport that requires an effective preflight briefing might be one way to begin. A scenario that allows the crew to choose from different options is very useful. One scenario can have a wide variety of outcomes and choices, depending on the decision and course of action that a crew undertakes. Again, the scenario should be realistic, and the situation should be one where crewmembers live with whatever problems they have until the situation is either resolved or the aircraft (flight simulation training device (FSTD)) is back on the ground.

7-3. SCENARIO DESIGN PROCESS. › d. Training Objectives. In LOS scenario design, the CRM and technical…

e. Briefing and Debriefing. The effective LOS experience begins with a briefing to discuss the LOS objectives and expectations. To be complete, the LOS must include a debriefing to examine the crew performance demonstrated during the LOS. The facilitator draws on personal experience and training in CRM and technical issues to elicit discussions of points of interest and operational relevance. Positive comments regarding crew performance should be emphasized in the debriefing, as well as comments regarding areas of crew performance that need improvement. Crewmembers must be given the opportunity to critique and analyze their own performance and review key points.

7-3. SCENARIO DESIGN PROCESS.

a. Developing the Scenario. The framework described here for developing LOS scenarios is based on the concept of an event set, a group of related tasks and conditions that are part of the scenario and are integrated into the LOS session with specific CRM and technical training objectives. Included in the framework is a new method for identifying specific CRM skills appropriate for the event sets and a tool for developing the CRM category profile of LOS scenarios. The result is an approach that makes LOS sessions more manageable and easier to assess by allowing the facilitator to concentrate on a few CRM categories within any given event set. It also limits the specific crew behavior observations to very specific behavioral markers and indicator behaviors that are either observed or not observed. Not only is the facilitator focusing on a single CRM category, such as “Communication,” but he or she is also looking for a particular, predetermined result driven by the details of the event, such as “Second in command (SIC) communicates Weight and Balance (W&B) for passenger count.”

b. Minimizing Subjectivity. One of the major goals of the use of the event set methodology is to remove as much subjectivity from the grading process as possible, particularly for CRM skills. By engineering the event in such a way that there is a clear indicator of the desired performance, the performance issue is focused down to, for example, whether the SIC either did or did not communicate W&B at the appropriate time and in the appropriate manner. That is what is being graded, not some subjective notion of “communication.” The assessment is not based on the general adequacy of the SIC’s communication skills, but instead on the adequacy of a very specific behavior. This technique has allowed evaluators to be trained to grade CRM skills with the same consistency with which they grade technical skills.

c. The Event Set. The primary unit of both LOS design and CRM assessment is the event set. The event set is made up of one or more events, including an event trigger, distracters, and supporting events. The event trigger is the condition or conditions under which the event is fully activated. The distractors are conditions inserted within the event set timeframe that are designed to divert the crew’s attention from other events that are occurring or are about to occur. Event sets with extremely disruptive triggers do not necessarily require a distractor. Finally, supporting events are other events taking place within the event set designed to further CRM and technical training objectives, usually the normal events of that phase of activity.

d. Training Objectives. In LOS scenario design, the CRM and technical training objectives should be integrated into the event sets. This event set framework allows the design team to present the appropriate degree of realism in the LOS. Instead of focusing on a single technical issue, the event set integrates the entire complex line environment (e.g., terrain, air

7-3. SCENARIO DESIGN PROCESS. › h. Summary. This design methodology provides a rigorous validation…

traffic control (ATC), or weather issues) to facilitate and maximize the crew’s performance in response to specified CRM and technical issues. With the LOS scenario now defined by event sets, scenario validation is performed at the event set level rather than limiting validation to the overall LOS. This allows, over time, for a sufficiently large enough library of effective scenarios to build up, such that the operator can begin building new scenarios from the event sets of previously developed scenarios. Rather than building each scenario from scratch, they can begin to mix and match event sets from their event set libraries to build new entire scenarios.

e. Developing Scenarios from the Event Set. The event set framework supports the development of LOS scenarios based on complex events as well as simple events. Simple events have no further consequences on the conduct of the flight once they have been diagnosed and corrected. The hung start in Table 7-2, Selected Scenario Event Set Index With Phases of Flight and Proficiency Objectives, is an example of a simple problem. It is effectively addressed within a single phase of flight. Overuse of simple problems or events detracts from LOS realism. Routine prestart problems, followed by a start problem, followed by a taxi problem, intrude on the crew’s perception that the LOS is an actual flight. However, one or two of these events can be useful for setting a proper environment to facilitate a CRM LOS when the objectives are stated properly within the event set framework.

f. Complex Events. Complex events have ongoing consequences that must be dealt with during the flight and cannot be solved by simply selecting and executing an abnormal checklist. Event-set-based scenarios require the coordinated actions of all crewmembers for successful completion. The compressor stall in Table 7-2 is an example of a complex problem. It is never actually solved, but instead it is managed over multiple phases of flight. Complex event set problems tend to be ambiguous, with no simple corrective checklist solution. The properly designed event set does not necessarily have a single solution. Rather, it may have a number of possible and reasonable solutions. Thus, the well-designed event promotes the management of a complex situation.

g. The Development Process. Table 7-1, Line Operational Simulation Design Methodology, describes an LOS development process. The overall purpose of the LOS development methodology is to build event sets that allow for the examination of the crew’s CRM and technical skills.

h. Summary. This design methodology provides a rigorous validation process to assure training and evaluation of all critical technical and CRM tasks identified by a training program. In addition, it allows for adaptation based on the operational environment of a particular organization. The remainder of this chapter analyzes each component of the design methodology in detail.

4) Evaluation, Modification, and, if Appropriate, FAA Approval of the… › 4.1 Represent the LOS scenario showing the event sets, event trigger,…

TABLE 7-1. LINE-OPERATIONAL SIMULATION DESIGN METHODOLOGY

1) Identification of Primary Crew Resource Management (CRM)/Technical Training Objectives.

1.1 Identify the primary CRM categories (e.g., decision making, communication, workload management) and integrate these with the primary technical training objectives.

1.2 Identify the related skills for the CRM categories identified in 1.1.

1.3 Identify the primary technical training objectives.

2) Identification of Possible Incidents That Will Produce the Training Objectives.

2.1 Identify incidents through a search of the Aviation Safety Reporting System (ASRS) database, own-carrier incident reporting, and own-carrier flight safety programs, such as Aviation Safety Action Program (ASAP) and flight operations quality assurance (FOQA).

2.2 Develop a preliminary list of relevant incidents and events.

2.3 Refine the listing of incidents and events, and correlate with the CRM categories and observable behaviors.

3) Specification and Development of LOS Scenario Event Sets.

3.1 Specify LOS scenario objectives, related proficiency objectives, primary and secondary CRM categories, and observable crew behaviors for each scenario event set.

3.2 Translate incidents and situations into scenario event sets by identifying the event trigger, distracters, and supporting events, and specify the phase of flight.

3.3 Integrate the individual scenario event sets into the overall scenario.

3.4 Administer the LOS validation instrument to ensure event sets are specified and organized consistent with the CRM and technical training objectives or proficiency objectives.

4) Evaluation, Modification, and, if Appropriate, FAA Approval of the LOS Scenario.

4.1 Represent the LOS scenario showing the event sets, event trigger, ATC communications, and the related CRM categories (e.g., use the CRM category

CHAPTER 7. LINE-OPERATIONAL SIMULATION SCENARIO DESIGN › 7-5. IDENTIFICATION OF PRIMARY TECHNICAL OBJECTIVES.

profile and matrix methods to represent the frequency of CRM categories over the entire scenario).

4.2 Fly the LOS scenario using at least two different crews. Invite the Principal Operations Inspector (POI)/Aircrew Program Manager (APM) to participate. Consider recording some of these sessions for use in developing facilitator training materials.

4.3 Administer the LOS validation instrument form to crews and facilitators instructors that fly the scenario.

4.4 Make required modifications to the revised LOS scenario.

4.5 Submit LOFT or Line Operational Evaluation (LOE) scenario to the POI for approval.

5) Instructor Training Implementation and Evaluation of LOS Scenarios.

5.1 Develop the final representation of LOS for facilitators, with the emphasis on event sets.

5.2 Develop the training plan and materials for recurrent training of facilitators and train the instructors/evaluators.

5.3 Implement the LOS scenario at the fleet level, and evaluate using actual facilitator and crew feedback.

7-4. IDENTIFICATION OF PRIMARY CRM TRAINING OBJECTIVES.

a. Identify Concepts and Definitions. Before an operator can develop a meaningful LOS, it must identify the CRM concepts and definitions that are meaningful within its own culture (refer to the current edition of Advisory Circular (AC) 120-51, Crew Resource Management Training). Several different organizations of CRM concepts can be used, but all of them follow a similar structure:

(1) High-level categories or elements, such as situational awareness and workload management.

(2) Supporting each category are knowledge and skills that should be trained. For example, in workload management, the knowledge would include how to prioritize tasks, while the skill would be the clear assignment of tasks that are understood by all crewmembers.

b. Measure Proficiency. To measure proficiency in the LOS, there should be a set of observable behaviors to look for. For example, in workload management, communicating task priorities is an observable behavior.

7-5. IDENTIFICATION OF PRIMARY TECHNICAL OBJECTIVES. Each carrier’s flight operations department develops technical training objectives. Identification of these objectives,

in the form of proficiency objectives, is a specific requirement of Advanced Qualification Program (AQP). A subset of these objectives can be selected as the technical objectives for each LOS. These objectives will serve an important role in selecting the event sets that will comprise the LOS scenario. Technical objectives might include:

7-6. DEVELOPING LOS SCENARIOS WITH AUTOMATION THEMES. › d. Scenario Design. Scenario designs should be guided by the skills…

• Origin, routing, and destination. • Revised departure or arrival procedures. • Alternate operation of flight management systems (FMS). • Partial or full loss of integrated FMSs. • Abnormal and emergency events. • Adverse weather and environmental conditions.

7-6. DEVELOPING LOS SCENARIOS WITH AUTOMATION THEMES.

a. Automation. Automation is the replacement of a human function, either manual or cognitive, with a machine function. This definition applies to all levels of automation in all aircraft. Effective use of automation means using that level most appropriate to support the priorities of safety, economy, and stated flight operations policies of the individual air carrier.

b. Automation Proficiency. Flightcrew members must be proficient in operating their aircraft in all levels of automation. They must be knowledgeable in the selection of the appropriate degree of automation, and must have the skills needed to move from one level of automation to another.

Continue reading ↓ 120 of 157 passages shown