Radar Vectors
Advisory Circular Current Issued Jun 23, 2006 AFS-280 150 pages Copy saved Nov 18, 2026

AC 120-54A Advanced Qualification Program - (Change 1)

This advisory circular (AC) provides the Federal Aviation Administration (FAA) guidance for approval of an Advanced Qualification Program (AQP) under AQP regulatory guidance. (AQP guidance is provided in Title 14 of the Code of Federal Regulations (14 CFR) part 121 subpart Y, or in a subsequent regulation that supersedes it.) AQP is a systematic methodology for developing the content of training programs for air carrier crewmembers and dispatchers. It replaces programmed hours with proficiency-based training and evaluation derived from a detailed job task analysis that includes Crew Resource Management (CRM). AQP incorporates data-driven quality control processes for validating and maintaining the effectiveness of curriculum content. AQP provides an alternate method of qualifying and certifying, if required, pilots, flight engineers, flight attendants, aircraft dispatchers, instructors, evaluators, and other operations personnel subject to the training and evaluation requirements of 14 CFR parts 121 and 135. The AQP encourages innovation in the methods and technology that are used during instruction and evaluation, and efficient management of training systems. The goal of AQP is to achieve the highest possible standard of individual and crew performance. A leading objective of AQP is to provide effective training that will enhance professional qualifications to a level above the present standards that are provided in parts 121 and 135.

Official source ↗ Open PDF

U.S. Department of Transportation Federal Aviation Administration

Advisory Circular

Subject: Advanced Qualification Program Date: 1/31/17 AC No: 120-54A Initiated by: AFS-200 Change: 1

1. PURPOSE. This advisory circular (AC) provides the Federal Aviation Administration (FAA) guidance for approval of an Advanced Qualification Program (AQP) under AQP regulatory guidance. (AQP guidance is provided in Title 14 of the Code of Federal Regulations (14 CFR) part 121 subpart Y, or in a subsequent regulation that supersedes it. AQP is a systematic methodology for developing the content of training programs for air carrier crewmembers and dispatchers. It replaces programmed hours with proficiency-based training and evaluation derived from a detailed job task analysis that includes Crew Resource Management (CRM). AQP incorporates data-driven quality control processes for validating and maintaining the effectiveness of curriculum content. AQP provides an alternate method of qualifying and certifying, if required, pilots, Flight Engineers (FE), flight attendants (F/A), aircraft dispatchers, instructors, evaluators, and other operations personnel subject to the training and evaluation requirements of 14 CFR parts 121 and 135.

The AQP encourages innovation in the methods and technology that are used during instruction and evaluation, and efficient management of training systems. The goal of AQP is to achieve the highest possible standard of individual and crew performance. A leading objective of AQP is to provide effective training that will enhance professional qualifications to a level above the present standards that are provided in parts 121 and 135. The contents of this document do not have the force and effect of law and are not meant to bind the public in any way. This document is intended only to provide clarity to the public regarding existing requirements under the law or agency policies.

2. PRINCIPAL CHANGES. This change provides clarification to certificate holders using AQP and the responsible Flight Standards offices conducting oversight of those certificate holders. It also updates references and terminology where needed.

PAGE CONTROL CHART

Remove Pages Dated Insert Pages Dated

Page i thru iii 6/23/06 Pages i thru iii 1/31/17

Pages v thru ix (and x) 6/23/06 Pages v thru ix (and x) 1/31/17

Page 1 6/23/06 Page 1 1/31/17

Pages 4 thru 6 6/23/06 Pages 4 thru 6 1/31/17

Page 11 6/23/06 Page 11 1/31/17

Page 18 6/23/06 Page 18 1/31/17

Pages 26 and 27 6/23/06 Pages 26 and 27 1/31/17

Page 35 6/23/06 Page 35 1/31/17

Page 60 6/23/06 Page 60 1/31/17

Page 63 6/23/06 Page 63 1/31/17

Page 67 6/23/06 Page 66 1/31/17

Pages 68 and 69 6/23/06 Pages 68 and 69 1/31/17

Pages 75 and 76 6/23/06 Pages 76 and 77 1/31/17

Page 77 6/23/06 Page 79 1/31/17

Page 79 6/23/06 Page 81 1/31/17

Pages 82 and 83 6/23/06 Pages 84 and 85 1/31/17

Pages 87 and 88 6/23/06 Pages 87 and 88 1/31/17

Page 90 6/23/06 Page 90 1/31/17

Page 95 6/23/06 Page 95 1/31/17

Appendix 1, Page 4 6/23/06 Appendix 1, Page 4 1/31/17

Appendix 2, Page 1 6/23/06 Appendix 2, Page 1 1/31/17

Appendix 3, Pages 1 and 2 6/23/06 Appendix 3, Pages 1 and 2 1/31/17

Appendix 4, Pages 2, 14, and 20 6/23/06 Appendix 4, Pages 2, 14, and 20 1/31/17

John Barbagallo Director, Flight Standards Service

3. RELATED 14 CFR PARTS. › e. Part 142, Training Centers.

U.S. Department of Transportation Federal Aviation Administration

Advisory Circular

Subject: Advanced Qualification Program Date: 6/23/06 AC No: 120-54A Initiated by: AFS-230 Change:

1. PURPOSE. This advisory circular (AC) provides the Federal Aviation Administration (FAA) guidance for approval of an Advanced Qualification Program (AQP) under AQP regulatory guidance (AQP guidance is provided in Title 14 of the Code of Federal Regulations (14 CFR) part 121 subpart Y, or in a subsequent regulation that supersedes it). AQP is a systematic methodology for developing the content of training programs for air carrier crewmembers and dispatchers. It replaces programmed hours with proficiency-based training and evaluation derived from a detailed job task analysis that includes Crew Resource Management (CRM). AQP incorporates data-driven quality control processes for validating and maintaining the effectiveness of curriculum content. AQP provides an alternate method of qualifying and certifying, if required, pilots, flight engineers (FE), flight attendants (F/A), aircraft dispatchers, instructors, evaluators, and other operations personnel subject to the training and evaluation requirements of 14 CFR parts 121 and 135.

The AQP encourages innovation in the methods and technology that are used during instruction and evaluation, and efficient management of training systems. The goal of AQP is to achieve the highest possible standard of individual and crew performance. A leading objective of AQP is to provide effective training that will enhance professional qualifications to a level above the present standards that are provided in parts 121 and 135. The contents of this document do not have the force and effect of law and are not meant to bind the public in any way. This document is intended only to provide clarity to the public regarding existing requirements under the law or agency policies.

NOTE: The methods and procedures in this AC describe one acceptable means of compliance. Alternate means are encouraged and will be evaluated on their merit.

2. CANCELLATION. AC 120-54, Advanced Qualification Program, dated August 9, 1991, is canceled.

3. RELATED 14 CFR PARTS.

a. Part 119, Certification: Air Carriers and Commercial Operators.

b. Part 121, Operating Requirements: Domestic, Flag, and Supplemental Operations.

c. Part 121 Subpart Y, Advanced Qualification Program.

d. Part 135, Operating Requirements: Commuter and On Demand Operations and Rules Governing Persons on Board Such Aircraft.

e. Part 142, Training Centers.

b. Joint Government-Industry Task Force. On August 27, 1987, FAA… › (3) Establish an FAA national air carrier training program office that…

Page ii

4. BACKGROUND. The capabilities and use of simulators and other computer-based training devices in training and qualification activities have changed dramatically. AQP regulatory guidance and this AC allow certificate holders that are subject to the training and evaluation requirements of part 121 and part 135 to develop innovative training and qualification programs that incorporate the most recent advances in training methods and techniques. AQP regulatory guidance and this AC also apply to training centers under 14 CFR part 142 that intend to provide training for eligible certificate holders. AQP emphasizes crew-oriented training and evaluation. These training and evaluation applications are now grouped under the general term of Line-Operational Simulations (LOS), including Line-Oriented Flight Training (LOFT), Special Purpose Operational Training (SPOT), and Line Operational Evaluation (LOE). Due to the role of CRM issues in fatal accidents, it has become evident that training curriculums should develop and evaluate both technical and CRM skills. In AQP, a structured LOS design process is employed to specify and integrate the required CRM and technical skills into LOS scenarios.

a. History. In 1975, the FAA began to deal with two issues: hardware requirements needed for total simulation, and the redesign of training programs to deal with increasingly complex human factors problems. At the urging of the air transportation industry, the FAA addressed the hardware issue first. This effort culminated in 1980 in the development of the Advanced Simulation Program. Since then, the FAA has continued to pursue approaches for the redesign of training programs to increase the benefits of Advanced Simulation and to deal with the increasing complexity of cockpit human factors.

b. Joint Government-Industry Task Force. On August 27, 1987, FAA Administrator T. Alan McArtor addressed representatives from major air carriers and air carrier associations, flightcrew member associations, commuter air carriers and regional airline associations, manufacturers, and government organizations. One of the issues discussed at the meeting focused on flightcrew member performance. This meeting led to the creation of the Joint Government-Industry Task Force on Flightcrew Performance. On September 10, 1987, the task force met at the Air Transport Association’s (ATA), currently known as Airlines for America (A4A), headquarters to identify and discuss flightcrew member performance issues. Working groups in three major areas were formed: (1) man/machine interface; (2) flightcrew member training; and (3) operating environment. Each working group submitted a report and recommendations to the joint task force. On June 8, 1988, the recommendations of the joint task force were presented to Administrator McArtor. The major substantive recommendations to the Administrator from the flightcrew member training group were the following:

(1) Require part 135 commuters (whose airplane operations require two pilots) to comply with part 121 training, evaluation, qualification, and recordkeeping requirements (part 119).

(2) Provide for an Special Federal Aviation Regulation (SFAR) and AC to permit development of innovative training programs (SFAR 58).

(3) Establish an FAA national air carrier training program office that provides training program oversight at the national level (Air Carrier Training Systems and Voluntary Safety Programs Branch).

5. BENEFITS OF AQP.

Page iii

(4) Require seconds in command (SIC) to satisfactorily perform their duties under the supervision of check airmen during operating experience (§ 121.434(c)(2)).

(5) Require all training to be accomplished through a certificate holder’s training program.

(6) Provide for approval of training programs based on course content and training aids rather than specified programmed hours (SFAR 58).

(7) Require Crew Resource Management (§ 121.404) (SFAR 58) training and encourage greater use of LOFT (§ 121.409) (SFAR 58).

c. National Transportation Safety Board (NTSB). In June of 1988, the NTSB issued a Safety Recommendation (A-88-071) on the subject of CRM training. The recommendation was that all part 121 carriers review initial and recurrent flightcrew training programs. The purpose of this review was to ensure that the training programs include simulator or aircraft training exercises that involve crew resource management and active coordination of all crewmember trainees, and which permits evaluation of crew performance and adherence to those crew coordination procedures.

d. Part 121 Subpart Y. In response to the recommendations from the joint task force and from the NTSB, the FAA, on October 2, 1990, published SFAR 58, which addresses the majority of the above recommendations. Part 121 subpart Y was released on September 16, 2005. AQP was also established to permit a greater degree of regulatory flexibility in the approval of innovative pilot training programs. Based on a documented analysis of operational requirements, a certificate-holder under AQP may propose to depart from traditional practices with respect to what, how, when, and where training and testing is conducted. This is subject to FAA approval of the specific content of each proposed program. Part 121 subpart Y requires that all departures from traditional regulatory requirements be documented and based upon an approved continuing data collection process sufficient to establish at least an equivalent level of safety. AQP provides a systematic basis for matching technology to training requirements and for approving a training program with content based on relevance to operational performance. An applicant may propose to replace certain requirements of 14 CFR part 61, 63, 65, 121, or 135, with an AQP curriculum, subject to FAA approval. An AQP may also employ substitutes for the practical test requirements. Each requirement of part 61, 63, 65, 121, 135 or the applicable Airman Certification Standards (ACS) or practical test standards (PTS) that is not specifically addressed in an approved AQP curriculum continues to apply to the certificate holder.

5. BENEFITS OF AQP. Although AQP is a voluntary program, the FAA Flight Standards Service (FS) encourages air carriers to participate. AQP provides for enhanced curriculum development and a data-driven approach to quality assurance along with the flexibility to target critical tasks during aircrew training. The AQP methodology directly supports the FAA’s goals for safety enhancement. The primary goal of AQP is to achieve the highest possible standard of individual and crew performance. In order to achieve this goal, AQP seeks to reduce the probability of crew-related errors by aligning training and evaluation requirements more closely with the known causes of human error. For example:

d. Additional Benefits. Added benefits that are expected for individual… › (6) Opportunity to achieve more efficient training.

Page iv

a. Crew Performance. Most accidents are attributed to crew error. Traditional training programs focus on individual training and evaluation. Under AQP, the focus is on crew and individual performance in both training and evaluation.

b. CRM. Most accidents are caused by errors of judgment, communication, and crew coordination. Traditional training programs focus primarily on flying skills and systems knowledge. Under AQP, competence in flying skills and systems knowledge are integrated with CRM skills in training and evaluation throughout the curriculum.

c. Scenario-Based Training and Evaluation. Most accidents are caused by a chain of errors that build up over the course of a flight and which, if undetected or unresolved, result in a final, fatal error. Traditional training programs, with their maneuver-based training and evaluation, artificially segment simulation events in such a way as to prevent the realistic build- up of the error chain. Under AQP, both training and evaluation are scenario-based, simulating more closely the actual flight conditions known to cause most fatal carrier accidents.

d. Additional Benefits. Added benefits that are expected for individual applicants will vary, but may include:

(1) The ability to modify training curricula, media, and intervals.

(2) Crew evaluation as well as individual assessment.

(3) Improved standardization across fleets and flight personnel.

(4) Shift from programmed hours to proficiency-based training.

(5) Access to innovative training ideas and research.

(6) Opportunity to achieve more efficient training.

Page v

CONTENTS Paragraph Page

CHAPTER 1. AQP OVERVIEW … 1

1-1. General … 1 1-2. AQP Objectives … 1 1-3. AQP Characteristics … 2 1-4. Equivalent Level of Safety and AQP Reqiurements … 2 1-5. Developmental Assistance and Support … 4 Figure 1-1. Other Resources … 6

CHAPTER 2. AQP DEVELOPMENT PHASES … 9

SECTION 1. INTRODUCTION … 9

2-1. Phases … 9 2-2. Instructional Systems Development (ISD) … 9 2-3. Training Systems … 10 Figure 2-1. AQP Phase Development Process … 10

SECTION 2. PHASE I: APPLICATION … 10

2-4. Joint Meeting … 10 2-5. The AQP Application … 11 2-6. Approval … 13

SECTION 3. PHASE II: CURRICULUM DEVELOPMENT … 13

2-7. Overview … 13 2-8. JTA … 14 Figure 2-2. Sample Pilot Job Task Listing … 16 2-9. Qualification Standards … 17 Figure 2-3. AQP Continuing Qualification Critical/Currency Chart … 19 Figure 2-4. Rating Scale for Procedures Validation, First-Look, Maneuvers

Validation, LOE, or Line Check … 23 2-10. Instructional Systems Development Methodology Document … 25 Figure 2-5. Curriculum Development … 26 2-11. Curriculum Outline Document … 27 Figure 2-6. Performance Difference Rating Scale … 28 Figure 2-7. Sample AQP Curriculum Outline … 29 2-12. Implementation and Operations Plan (I&O Plan) Document … 30 2-13. Approval … 32

SECTION 4. PHASE III: SMALL GROUP TRYOUT (IMPLEMENTATION) … 32

2-14. Overview … 32 2-15. Approval … 33

SECTION 5. PHASE IV: INITIAL OPERATIONS … 33

2-16. Overview … 33 2-17. Phase IV Activities … 33

Page vi

2-18. Revisions … 34 2-19. Approval … 34

SECTION 6. PHASE V: CONTINUING OPERATIONS … 34

2-20. Overview … 34 2-21. Data’s Impact on Continuing Operations … 34 2-22. Quality Assurance … 34

SECTION 7. DOCUMENT AND REPORTING REQUIREMENTS … 34

2-23. Overview … 34 Figure 2-8. Document Structures Chart … 37

CHAPTER 3. AQP CURRICULUMS AND CERTIFICATION … 39

SECTION 1. CURRICULUMS … 39

3-1. Background … 39 3-2. Indoctrination Curriculum … 39 3-3. Qualification Curriculum … 39 Figure 3-1. Flightcrew Member AQP Training, Validation, and Evaluation … 43 Figure 3-2. AQP Validation/Evaluation Table … 44 Figure 3-3. Example of AQP Flightcrew Qualification Curriculum Footprint … 45 3-4. Continuing Qualification Curriculum … 46 Figure 3-4. Example of An AQP Four-Day Flightcrew Continuing Qualification

Curriculum Footprint … 49 Figure 3-5. Continuing Qualification Cycle … 50 3-5. Secondary Curriculums … 52 Figure 3-6. AQP Secondary Curriculums … 52

SECTION 2. AIRMAN CERTIFICATION … 54

3-6. General … 54 3-7. AQP Training and Evaluation for Certification … 54 3-8. Completion of the Qualificatioin Curriculum … 54

CHAPTER 4. AQP APPROVAL AND DOCUMENTATION … 55

SECTION 1. DOCUMENTATION REVIEW AND APPROVAL … 55

4-1. Overiew … 55 4-2. Review Process … 55 4-3. Approval Process … 55 Figure 4-1. Five Phases of AQP Review and Approval … 56 4-4. Phase Approvals … 56 4-5. Method of Granting Approval … 59 4-6. Revision Control Process … 60

SECTION 2. WITHDRAWAL OF APPROVAL … 61

4-7. General … 61 4-8. Initial or Provisional … 61 4-9. Final Approval … 61

Page vii

4-10. Appeal Withdrawal of Final Approval … 61

CHAPTER 5. INSTRUCTORS AND EVALUATORS … 63

SECTION 1. INSTRUCTOR/EVALUATOR PROGRAMS … 63

5-1. General … 63 5-2. Instructor/Evaluator Curriculums … 63 5-3. Authorized Evaluation Personnel by AQP Event … 67 Figure 5-1. Authorized Evaluation Personnel by AQP Curriculum … 68

SECTION 2. CERTIFICATION PERSONNEL … 68

5-4. Training … 68 5-5. Authorized Certification Personnel … 69

SECTION 3. QUALITY ASSURANCE AND STANDARDIZATION … 69

5-6. Quality Assurance … 69 5-7. Standardization … 69

CHAPTER 6. TRAINING CENTERS … 71

SECTION 1. INTRODUCTION … 71

6-1. Background … 71 6-2. Training Centers Offering an AQP … 71

SECTION 2. PROVISIONAL APPROVAL: APPLICATION AND DEVELOPMENT … 71

6-3. Provisional AQP … 71

SECTION 3. REQUEST AND APPROVAL FOR USE OF TRAINING CENTERS … 72

6-4. Request … 72 6-5. Development … 72

SECTION 4. TRAINING CENTER’S INSTRUCTORS AND EVALUATORS … 73

6-6. Qualification … 73 6-7. Instructor/Evaluator Curriculum Approval … 73

SECTION 5. DATA COLLECTION AND RECORDKEEPING … 73

6-8. Data Collection … 73 6-9. Data Submission … 73 6-10. Recordkeeping … 73

CHAPTER 7. TRANSITION TO AQP … 75

SECTION 1. OVERVIEW … 75

7-1. General … 75 7-2. Entering AQP from an Established Training Program … 75 7-3. Transitioning to AQP Using the SVE … 75

Page viii

SECTION 2. THE PETITION FOR EXEMPTION … 81

7-4. General … 81 Figure 7-1. Single Visit Exemption (SVE) Process … 81 7-5. The Petition Process … 81 7-6. The Petition … 82 7-7. Granting of the Petition … 82 7-8. Renewal Requirements for the SVE … 83

SECTION 3. SVTP … 83

7-9. Overview … 83 7-10. Contents … 83 7-11. SVTP Proficiency Check … 83 7-12. Approval of the SVTP … 84 7-13. Revisions … 84 7-14. Implementation of the SVTP … 85 7-15. Data Submission and Reporting Requirements … 85 7-16. SVTP Review Meeting … 85 Figure 7-2. Sample of Proficiency Check Fixed and Variable Maneuvers … 86

CHAPTER 8. AQP DATA MANAGEMENT … 87

SECTION 1. INTRODUCTION … 87

8-1. Overview … 87 8-2. Definitions … 87

SECTION 2. DATA MANAGEMENT … 88

8-3. Overview … 88 8-4. Data Collection—SVTP and AQP … 88 8-5. Data Entry … 88 8-6. Data Submission … 88

SECTION 3. DATA ANALYSIS AND REPORTS … 90

8-7. Overview … 90 Figure 8-1. Data Submission Requirements … 91 Figure 8-2. Performance Data Report Table (PDRT) … 93 Figure 8-3. Skill/Reason Table (SKLRSN) … 96 Figure 8-4. Training Objectives Report Table (TORT) … 97

CHAPTER 9. CREW RESOURCE MANAGEMENT … 99

SECTION 1. INTRODUCTION … 99

9-1. Overview … 99

SECTION 2. INTEGRATING CRM INTO AN AQP … 99

9-2. Scope of Integration … 99 9-3. CRM/AQP Task List … 100 9-4. CRM Knowledge and Skills … 100 9-5. CRM and Proficency Objectives … 100

Page ix (and x)

9-6. Training Events … 101 9-7. Event Sets … 101 9-8. Source … 101 9-9. Curriculum Design … 102 9-10. Indoctrination Curriculum … 102 9-11. Qualification Curriculum … 102 9-12. Continuing Qualification Curriculum … 102 9-13. Line Opeartions and Proceduralized CRM … 103

SECTION 3. EVALUATING CRM … 103

9-14. Overview … 103 9-15. Observable Behaiors … 103 Figure 9-1. Sample CRM Skills … 104 Figure 9-2. Sample Event Worksheet … 106

APPENDIX 1. DEFINITIONS (6 pages) … 1 APPENDIX 2. GENERIC AIRLINES ADVANCED QUALIFICATION PROGRAM

APPLICATION (2 pages) … 1 APPENDIX 3. REGULATORY COMPARISON CHART/PART OF THE

QUALIFICATIONS STANDARDS DOCUMENT (6 pages) … 1 APPENDIX 4. ADVANCED QUALIFICATION PROGRAM DOCUMENTATION

CHECKLIST AND REVIEW JOB AID (21 pages) … 1 APPENDIX 5. ADVANCED QUALIFICATION PROGRAM QUALIFICATION

STANDARDS DOCUMENTATION (4 pages) … 1

1-2. AQP OBJECTIVES. › c. Remain responsive to continuing changes and best practices relative…

CHAPTER 1. AQP OVERVIEW

1-1. GENERAL. The Advanced Qualification Program (AQP) integrates a number of training and evaluation features aimed at improving performance. AQPs are systematically developed, continuously maintained, and empirically validated proficiency-based training systems. They allow for the systematic analysis, design, development, implementation, evaluation, and maintenance of self-correcting training programs.

a. Phases of Development. There are five phases for developing, implementing, and maintaining an AQP. A description of each phase is located in Chapter 2. Each phase must be Federal Aviation Administration (FAA)-approved before the applicant may proceed to the next phase. Each phase consists of specific activities, including the documentation of those activities. These documents must also be FAA-approved.

b. AQP Documentation. In addition to the supporting documents and manuals provided to the FAA in traditional training programs, there are six documents and an annual report requirement that is unique to AQP. The documents are instrumental in managing the AQP and must be maintained throughout the life of the program. The documents can be categorized in two groups, by their function and interrelationship: management and database. The documents, along with the annual report, comprise the Program Audit Database (PADB).

(1) Management Documents. These documents include the Application, Instructional Systems Development Methodology, and Implementation and Operations Plan. These documents are stand-alone in that a change in one will not necessitate a change in another.

(2) Database Documents. These documents include the Task Analysis, Qualification Standards, and Curriculum Outlines. Because a change to one often means a change to the others, it is recommended that they be maintained in an interactive database.

(3) Annual Report. The purpose of the report is to identify changes to the curriculums, training equipment upgrades, and the AQP Maintenance Strategy resulting from feedback and analysis of the information in the Performance/Proficiency Database (PPDB). Details of the report are found in Chapter 2, Section 7.

1-2. AQP OBJECTIVES.

a. Support safe operations by continuously improving training and evaluation.

b. Remain responsive to continuing changes in the industry, including new aircraft technology, changing operational environments, and new training methods and equipment.

c. Remain responsive to continuing changes and best practices relative to training and evaluation.

CHAPTER 1. AQP OVERVIEW › 1-4. EQUIVALENT LEVEL OF SAFETY AND AQP REQUIREMENTS.

1-3. AQP CHARACTERISTICS.

a. Participation is voluntary.

b. Employs innovative training and qualification concepts with the regulatory flexibility to tailor training to individual company circumstances.

c. Uses performance data to drive curriculum changes.

d. Qualification is based on individual and team performance, using progressive evaluations of proficiency objectives, and the structure and maintenance of all elements (curriculum, facilities, training equipment, instructors, evaluators, courseware, and quality assurance) of the program.

e. Data collection and analysis to empirically validate individual and team proficiency, and the AQP itself.

f. Training is systematically developed with an audit trail for all training and data requirements.

1-4. EQUIVALENT LEVEL OF SAFETY AND AQP REQUIREMENTS.

NOTE: AQP is a voluntary program that entails a strong commitment from the air carrier to exceed minimum training standards in the greater interest of safety. To determine an equivalent level of safety of an airline’s AQP, the entire training system must be examined as a comprehensive whole rather than considering any one component in isolation. To ensure that the AQP does establish an initial justification and a continuing process to show an equivalent level of safety, mandatory requirements have been established.

Under an AQP, the FAA monitors the process as well as the product. Instead of basing curriculums on prescribed generic maneuvers, procedures and knowledge items, AQP curriculums are based on a detailed analysis of the specific job tasks, knowledge, and skill requirements of each duty position for the individual airline. Compared to traditional training programs, the AQP process provides a systematic basis for establishing an audit trail between training requirements and training methodologies.

AQPs are systematically developed, continuously maintained, and empirically validated, proficiency-based training systems. They allow for the systematic analysis, design, development, implementation, progressive evaluation, and maintenance of self-correcting training programs that include integrated Crew Resource Management (CRM), improved instructor/evaluator standardization, scenario-based evaluation, and a comprehensive data-driven quality assurance system.

1-4. EQUIVALENT LEVEL OF SAFETY AND AQP REQUIREMENTS. › n. A list of, and text describing, supervised operating experience,…

a. Comply with all aspects of the approved AQP.

b. The processes used for development, implementation and maintenance of program operations will be continued throughout the life of the program.

c. Accommodate make, model, and series aircraft (or variant).

d. May build upon an existing training program or be completely new.

e. Provide three basic types of curriculums for every duty position; indoctrination (for new hires, new instructors, and new evaluators); qualification; and continuing qualification (see Figure 3-6). Specialty curriculums (transition, conversion, upgrade, requalification, refresher, etc.) will be derivatives from these basic types.

f. Duty positions to be covered must include all flightcrew member positions, instructors, and evaluators, and may include other positions, such as flight attendants (F/A), aircraft dispatchers, and other operations personnel.

g. Provide satisfactory justification to the FAA that the proficiency-based qualification of personnel under AQP meets or exceeds existing 14 CFR part 121 and/or part 135 standards.

h. Each applicable requirement of the Airman Certification Standards (ACS) or practical test standards (PTS) and 14 CFR part 61, 63, 65, 121, or 135 that is not specifically addressed in the AQP continues to apply to the certificate holder.

i. Document the requirements of the ACS or PTS and part 61, 63, 65, 121, or 135, as applicable, which would be replaced by an AQP curriculum.

j. The certificate holder must establish an initial justification and a continuing process to show how the AQP curriculum provides an equivalent level of safety for each requirement to be replaced.

k. Any training or evaluation that is satisfactorily completed in the calendar-month before or the calendar-month after the calendar-month in which it is due is considered to have been completed in the calendar-month it was due.

l. Curriculums must be based on an instructional systems development methodology. This methodology must incorporate a thorough analysis of the certificate holder’s operations, aircraft, line environment and job functions.

m. A list of, and text describing, the knowledge requirements, subject materials, job skills, and qualification standards of each task to be trained and evaluated.

n. A list of, and text describing, supervised operating experience, evaluation/remediation strategies, provisions for special tracking, and how recency of experience requirements will be accomplished.

1-5. DEVELOPMENTAL ASSISTANCE AND SUPPORT. › a. Flight Standards (FS). The appropriate Safety Standards office is…

o. All curriculums will include planned (not programmed) hours for ground training, flight training, evaluation, and operating experience.

p. Qualification and continuing qualification curriculums must integrate the training and evaluation of CRM/dispatch resource management (DRM) and technical skills and knowledge.

q. Include line operational training (e.g., full crew LOS).

r. Use LOE or an equivalent evaluation under an AQP acceptable to the FAA for proficiency evaluations.

NOTE: Regardless of number of events sets, a crash (i.e., unsafe individual or crew performance that would result in significant damage, hull loss, or loss of life) during an LOE constitutes a failure of the LOE.

s. Integrate appropriate advanced flight training equipment. Flight-training devices and simulators will be used to support scenario-based training as appropriate.

t. Develop data collection and analysis processes in order to obtain performance information on crewmembers, dispatchers, instructors, evaluators, and other operations personnel that will enable the certificate holder and the FAA to determine whether the form and content of training and evaluation activities are satisfactorily accomplishing the overall objectives of the curriculum.

NOTE: The FAA uses de-identified data for program monitoring not to monitor individual crewmember, dispatcher or other operations personnel.

u. Provide a Master AQP Transition Schedule (MATS) that provides a plan to transition from a traditional program to an AQP. In addition, the MATS should include a plan on how the applicant would return to a traditional program if that becomes necessary or desirable at some later date. This is not to imply that the traditional program must be kept current once AQP is entered, but only how the applicant would update the program and the timeframe needed for implementation.

1-5. DEVELOPMENTAL ASSISTANCE AND SUPPORT. Applicants may avail themselves of a wide range of support organizations, documents and services in the development of their AQPs. There is considerable support available from government and non-government sources. The total inventory of such support is constantly increasing, and the latest versions of all such support resources are always available by contacting the appropriate Safety Standards office. The following is available directly from the FAA:

a. Flight Standards (FS). The appropriate Safety Standards office is the primary support available to all AQP applicants and participants. This office includes a manager, aviation safety inspectors (ASI), instructional systems development specialists, and data management specialists dedicated to the support of AQP programs. This office works in conjunction with the Principal Operations Inspector (POI) and other FAA personnel to transition applicants into AQP and to maintain the quality of those programs following full implementation.

1-5. DEVELOPMENTAL ASSISTANCE AND SUPPORT. › c. Other Resources. The following list (Figure 1-1) provides samples of…

(1) Role of the Appropriate Safety Standards Office Manager. The FAA manager is the person responsible for ensuring that the regulatory requirements for AQP are met and approved AQP standards are maintained. In this capacity, the appropriate Safety Standards office manager provides oversight of the AQP process and documentation through all phases. The manager will also chair the AQP Extended Review Team (ERT). The manager will retain a quality assurance role and coordinate with the POI any modifications and subsequent approvals throughout the AQP’s lifetime.

(2) Role of the POI. The POI is responsible for ensuring that AQP regulatory requirements are met and, that through surveillance, the AQP provides a satisfactory level of safety and airman competence. The POI will provide oversight of the Instructor/Evaluator program and adherence to approved documentation for all AQP curricula. The POI will coordinate with the manager of the appropriate Safety Standards office any modifications and subsequent approvals throughout the AQP’s lifetime.

(3) ERT. The review and surveillance of an AQP prior to final approval is equivalent to a recertification of the applicants’ entire training program. To ensure maintaining an effective and standardized process, the appropriate Safety Standards office will lead a team to conduct the review and certification of the AQP. The ERT will also include (but not necessarily be limited to) the following:

(a) The POI (or a designated representative).

(b) Training Center Program Manager (TCPM) (if applicable).

(c) The assigned aircrew program manager (APM) (if any).

(d) An appropriate Safety Standards office ASI staff specialist.

(e) A cabin safety inspector (if required).

(f) A dispatcher (if required).

(g) An instructional system design specialist.

(h) A data management specialist.

NOTE: Additional technical personnel can be added to the ERT as required in specific areas of expertise. These would include the focal point, a member of the Flight Standardization Board (FSB) or Flight Operations Evaluation Board (FOEB), as appropriate.

b. Technology Support. The appropriate Safety Standards office includes a technology staff assigned to enhance the effectiveness of the program through the development of websites, databases, collaboration techniques, software training and installation, and other support.

c. Other Resources. The following list (Figure 1-1) provides samples of the types of resources available:

FIGURE 1-1. OTHER RESOURCES

AQP Websites: http://www.faa.gov/training_testing/training/aqp/ (public access) FAA Resources:

• Title 14 CFR Part 121 Subpart Y, Advanced Qualification Program FAA Advisory Circulars:

• AC 120-35, Flightcrew Member Line Operational Simulations: Line-Oriented Flight Training, Special Purpose Operational Training, Line Operational Evaluation (as amended) • AC 120-51, Crew Resource Management Training (as amended) • AC 121-42, Leadership and Command Training for Pilots in Command ATA AQP Working Group: Focus Group Reports:

• Line Operational Simulations: LOFT Scenario Design, Conduct and Validation • Advanced Qualification Program Instructor/Evaluator Task Analysis • Applied ISD in AQP Development: Front End Analysis • Advanced Qualification Program Data Management Guide, Second Release AQP Software Tools:

• RRLOE: Rapidly Reconfigurable Even Set-Based Line Operational Evaluation Software [LOS Scenario-building software toolkit] • Proteus Database [AQP curriculum & documentation-building software toolkit: AQP in a box] • Rater Calibration Software Tools [Measures and assesses reliability across evaluators] • Inter Rater Reliability Assessment Toolkit (workbook & diskette) • Instructor Evaluator Training & Calibration (IETC) Toolkit AQP Research Products:

• Developing Advanced Crew Resource Management (ACRM) Training: A Training manual • Developing Operating Documents: A Manual of Guidelines (on CD) • Facilitating LOS Debriefings: A Training Manual The Appropriate Safety Standards Office Products:

• Advanced Qualification Program Overview, Powerpoint briefing (as amended) • Advanced Qualification Program (AQP) Inspector Training (Student Guide, Leader Guide, Reference Book, Lab Book, Slides) • Overview of the Advanced Qualification Program (by T. Longridge) • Data Analysis and Reporting Tool (DART) • Data Cleaner (used in conjunction with DART) Generic Documents:

• Generic AQP Application • Generic Single Visit Training Program Application • Generic Single Visit Training Plan • Generic Pilot Job Task List • Generic Pilot Job Task Analysis • Generic F/A Job Task List • Generic Curriculum Development Methodology Document

Par 1-5 Page 7 (and 8)

FIGURE 1-1. OTHER RESOURCES (CONTINUED)

Research Papers:

• Scientific Evaluation of Aircrew Performance • Application of Psychometrics to the Calibration of Air Carrier Evaluators [with UNM] • Data Warehousing for AQP Databases • Prototype AQP Data Analysis Process • Training for Inter-Rater Reliability: Baselines & Benchmarks • Line Operational Evaluation (LOE) Air Carrier Scenario Based Evaluation • Performance Measurement Issues and the AQP Database Design • Skills Development and Assessment in the AQP Environment • The Importance of Quality Data In Evaluating Aircrew Performance • Questions Your Database Should Address • A Guide to the Evaluation Aspects of Entering AQP • Predicting Behavioral Grades from Physical Flight Data for Rejected Takeoff • Inferential Statistics and Sample Size: Applications to Air Carrier Training and Assessment • Initiatives Towards More Affordable Flight Simulators for U.S. Commuter Airline Training • Simulator Fidelity Requirements: The Case of Platform Motion • Simulator Platform Motion – The Need Revisited • Simulator Fidelity: The Effect of Platform Motion • Relationship Between Objective Measures of

Pilot Performance/Behavior and Instructor Grades • The Effect of Simulator Motion on Pilot Training and Evaluation • Realistic Radio communications in Pilot Simulator Training • Air Traffic Control in Airline Pilot Simulator • Rapidly Reconfigurable Event Set-Based Line Operational Evaluation Scenarios • Do Three Easy Tasks Make One Difficult One? Studying the Perceived Difficulty of Simulator Scenarios • Differences in General and Commercial Aviation Automation • An Information-Processing Model Approach to Identifying Training Issues for Advanced Aviation Automation • Do Pilots Know How They Manually Control the Vertical Flight Path? • CRM in the Model AQP: A Preview • The Use of ASRS Incident Reports in AQP CRM Training • The Model AQP Database Process: Challenges in Process and Design • AQPing CRM • Implementing the Model AQP Database: Lessons Learned • Data Collection and Analysis: The Next AQP Frontier • Reconceptualizing Leadership and Followership for Event-Based Training Research Integrations:

• Training Approaches and Considerations For Automated Aircraft: A Summary of Training Development Experiences

CHAPTER 2. AQP DEVELOPMENT PHASES

SECTION 1. INTRODUCTION

2-1. PHASES. The development of an Advanced Qualification Program (AQP) is a five-phase process. This process provides a structured building-block approach to program development. This approach results in a program with fully documented curriculums, supporting rationale, and development methodology. The use of this standardized AQP development process, along with its documentation requirements, allows an AQP applicant to develop training and evaluation programs based on analyzed and justifiable requirements. The documentation from this five-phase process will then serve as the regulatory basis for promulgating the AQP-developed training and evaluation programs. Each phase is approved sequentially. Each phase consists of specific activities, including the documentation of those activities. These documents must also be FAA-approved. An applicant cannot exercise any training and evaluation provisions permitted in Phases III, IV, or V, respectively, until each preceding phase is completed and approved by the FAA Extended Review Team (ERT) (see Figure 2-1).

2-2. INSTRUCTIONAL SYSTEMS DEVELOPMENT (ISD). › i. Measure student performance against proficiency objectives and…

2-2. INSTRUCTIONAL SYSTEMS DEVELOPMENT (ISD). The FAA has incorporated the use of the systematic methodology known as ISD into the AQP process. Applicants may employ any of a wide range of current ISD models or customize their own approaches to curriculum development. This chapter provides one acceptable methodology, but applicants are encouraged to be creative in tailoring their own ISD approach to their requirements, subject to FAA approval. Innovation and practical application may result in equally acceptable AQPs. Because some ISD models are far more complex than others, the FAA has found it useful to define its minimal ISD requirements. These are:

a. Develop a job task listing.

b. Analyze that listing to determine essential skill and knowledge requirements (either directly or by reference).

c. Determine which skill and knowledge requirements must be trained/tested.

d. Develop proficiency objectives that capture all training requirements.

e. Develop qualification standards that define acceptable operational performance levels.

f. Develop tests that measure proficiency in skill and knowledge areas.

g. Provide instructional programs that teach and test training requirements.

h. Establish and maintain an audit trail of explicit links between task requirements, training requirements, training and evaluation activities, and evaluation results.

i. Measure student performance against proficiency objectives and qualification standards for all curriculums.

Section 2. PHASE I: APPLICATION › 2-4. JOINT MEETING.

j. Revise the training program based on student performance levels on an ongoing basis. This de-identified data (stored in the Performance/Proficiency Database) will be collected and reported to the FAA on a regular basis.

2-3. TRAINING SYSTEMS. While all applicants must meet the minimum requirements of the ISD approach, the amount of effort each applicant should put into each step of their training analysis and development depends on a number of factors. Applicants may use the ISD process to build complete training systems from the ground up, to build a proficiency-based quality control shell around an existing training system, or to make minor modifications to existing proficiency-based curriculums. Building a training system from the ground up for a new aircraft will require far more depth of analysis and development than merely modifying a current curriculum for an existing aircraft.

FIGURE 2-1. AQP PHASE DEVELOPMENT PROCESS

SECTION 2. PHASE I: APPLICATION

2-4. JOINT MEETING. AQP begins with a letter from the applicant, to the Principal Operations Inspector (POI), a duplicate copy of which is forwarded to the FAA manager of the

appropriate Safety Standards office stating their intent to develop an AQP. The appropriate Safety Standards office manager will discuss the applicant’s plans with the POI. The appropriate Safety Standards office manager will respond to the certificate holder, through the POI, with a Letter of Acknowledgment requesting a meeting with representatives of the certificate holder, the appropriate Safety Standards office manager, and the responsible Flight Standards office inspectors responsible for oversight. The inspectors should include the POI or the inspector assigned to be the point of contact for AQP development. If the applicant is a training center, a Training Center Program Manager (TCPM) would represent the responsible Flight Standards office. The focus of this meeting is to ensure that all parties understand AQP, specifically the responsibilities and the commitment needed to complete the project. Discussions will include the anticipated benefits of AQP, AQP entry and exit strategies, the development process requirements, possible problem areas, and available tools. Notification of entry into AQP Phase I, Application Development, will be by letter from the appropriate Safety Standards office manager to the applicant through the POI.

2-5. THE AQP APPLICATION. › d. Training Equipment Description and Location. The application should…

2-5. THE AQP APPLICATION. The purpose of the application is to provide an overview of the applicant’s plan for developing an AQP for all of its fleets, instructors and evaluators, and for non-fleet specific curriculum (i.e., indoctrination). The application is submitted once and is updated as information in the application warrants changing (e.g., a change in the schedule, adding new aircraft, initiating flight attendant (F/A) or dispatcher AQP programs, etc.). To establish the applicant’s intent and approach for developing an AQP, the application should address thoroughly the following topics:

a. Letter of Application (See Appendix 2). The letter should clearly state the certificate holder’s intent to develop, implement, and operate an AQP.

b. Data Collection, Submission and Analysis Reporting. The AQP applicant must acknowledge their understanding and acceptance of the AQP Performance Data requirements as defined in Chapter 8 of this advisory circular (AC).

c. Operating Environment Description. Applicants should describe their operating environment, including the general physical environmental factors expected to be encountered in operations. Environmental factors are critical to development of Line-Operational Simulation (LOS) scenarios and meaningful proficiency objectives. Environmental factors include:

(1) Weather norms and extremes.

(2) Normal, abnormal, and emergency equipment operation.

(3) Geographic areas of operations.

d. Training Equipment Description and Location. The application should describe the training equipment, its location, and the organization responsible for its security and maintenance. If the type of training equipment for the AQP is not known at the time the application is submitted, the application will be updated when the training equipment requirements are identified. The FAA identification number assigned by the National Simulator Program manager and the level of qualification identifies flight simulators and/or flight training

g. AQP Organization. This section explains the organizational resources… › (5) Computer Support/Database Management.

devices. Other training devices will be listed and if qualification is required, the applicant should state when it intends to submit a test guide and a request for equipment qualification.

e. Facilities Description. Each AQP submission should describe the facilities the applicant intends to use. The description should include the location, type of facility, classrooms, training aids, courseware, and other features that contribute to creating and maintaining a positive learning environment.

f. Trainee Demographics. The application should provide a summary of demographic data on the experience level of the pilots, flight engineers (FE), F/As, and dispatchers that will be trained under AQP. This data includes the current ground and flight instructors and evaluators that are expected to continue these functions under AQP. For each fleet, this information would identify the:

(1) Entry requirements for ground and flight instructors and evaluators.

(2) Entry requirements for new hires.

(3) Students should be identified as a group in terms of previous experience.

(4) Current and anticipated need for replacement crewmembers by duty position.

g. AQP Organization. This section explains the organizational resources that the air carrier plans to employ for AQP development. Generally, an AQP applicant’s staff should support the following functions:

(1) AQP Coordination. The company focal point for the AQP development effort. Responsible for AQP leadership and is the primary contact with the FAA and any other external organizations that may impact the applicant’s AQP. This position should be filled with an individual who understands job task analysis and the training requirements of Title 14 of the Code of Federal Regulations (14 CFR).

(2) Subject Matter Expertise. Crewmembers, dispatchers and other operations personnel who are current and qualified in company operations and represent the population of professionals the AQP will address. The individuals may be called upon to act as liaison with operational support personnel.

(3) Document and Curriculum Development. Interfaces with the AQP Coordinator and the subject matter experts (SME) to develop the requisite AQP process, curriculum, and instructor/evaluator documents.

(4) Document Management. Ensures AQP document control and congruence with FAA requirements.

(5) Computer Support/Database Management. Oversight of the development and management of the performance/proficiency data acquisition and analysis system. In addition, this individual could be used for other computer-related issues related to the facilitation of an AQP, such as electronic document review.

Section 3. PHASE II: CURRICULUM DEVELOPMENT › The task analysis is the basic document that supports the development…

h. Master AQP Transition Schedule (MATS). The MATS will depict the projected transition/development schedule for all AQP curriculums. Since these schedules usually change, this section of the application must be updated accordingly. Transition from one type training program to another (traditional to Single Visit Training Program (SVTP) to AQP, or traditional to AQP) may include a period of overlap as one program is phased in and the other phased out. The following guidelines for transition are applicable:

(1) Currently qualified personnel may transition between traditional recurrent training curriculums and continuing qualification curriculums.

(2) Personnel who have completed a traditional training program initial, transition or upgrade curriculums may enter a continuing qualification curriculum.

(3) Partial MATS (incomplete) are not acceptable.

(4) The MATS may provide for incremental implementation of indoctrination, qualification, and continuing qualification curriculums for each AQP.

(5) The MATS must provide the time frame necessary to withdraw from AQP if it becomes necessary to revert to the traditional 14 CFR part 121 or 135 training program.

2-6. APPROVAL. Approval of the application marks the end of Phase I and the beginning of Phase II.

SECTION 3. PHASE II: CURRICULUM DEVELOPMENT

2-7. OVERVIEW. Phase II is the development of the training curricula that apply to any of the duty positions addressed in the Phase I application (e.g., pilots, F/As, dispatchers, instructor, evaluators). There are five general stages in this developmental process, each represented by the required document listed below:

• Job Task Analysis (JTA). • Qualification standards. • Instructional Systems Development (ISD) methodology. • Curriculum outlines. • Implementation and Operations Plan (I&O Plan).

The task analysis is the basic document that supports the development and analysis of the qualification standards. The ISD methodology defines how the task analysis and qualifications standards will be used to support the development of the curriculums. The last document is the I&O Plan, which explains how the AQP will be put into practice. The applicant must establish, demonstrate, and maintain a clear linkage between each of these steps. This linkage is provided by a systematic approach to the development of a complete instructional system. This section recommends a systematic approach and a methodology that is acceptable to the FAA, but innovation and practical application may result in equally acceptable variations.

Section 3. PHASE II: CURRICULUM DEVELOPMENT › JTA is the method or procedure used to reduce a unit of work to its…

a. Rationale. Under traditional training programs, the FAA evaluates finished training curriculums by comparing their contents, as described in a curriculum outline, to guidance in the regulations and FAA Order 8900.1. Under an AQP, the ERT evaluates these curriculums by monitoring their development and by approving a series of documents that are preliminary to the curriculum outline. These documents include the task analysis, the qualification standards, and the ISD Methodology. This continuing process allows the FAA greater insight into the rationale used by the applicant to develop each component of each curriculum.

b. Analysis-Based Approach. The analysis-based approach allows each applicant the opportunity to develop operator specific training programs. Consequently, AQP curriculums will be more sensitive to differences in aircraft, operating conditions, emergency and abnormal contingencies, student skill levels, and other operational variables, than are traditional programs. Under an AQP, the FAA monitors the process as well as the product. Instead of basing curriculums on prescribed generic maneuvers, procedures and knowledge items, AQP curriculums are based on a detailed analysis of the requirements of each duty position in each organization. To obtain FAA approval, these new AQP curriculums must be judged to be equal to, or provide better training than, a traditional training program. AQP curricula do not need to duplicate traditional curricula in order to satisfy this criterion, provided that they otherwise incorporate an alternative approach that addresses the required skills and knowledge to perform the job tasks.

2-8. JTA. To understand task analysis as it applies to AQP, it is necessary to build upon the following definitions:

• Job: A job is the summation of functions, identified as tasks and subtasks, performed by an individual at work. • Function: One of the major subdivisions of work activities performed by one individual. One or more functions constitute a job. Examples of functions applicable to AQP would be: phases of flight, such as takeoff, climb, cruise, etc. • Task: A task is a unit of work within a function having an identifiable beginning and ending point, which results in a measurable product. An example of a task applicable to AQP would be: perform a normal takeoff. • Subtask: Specific separate step or activity required in the accomplishment of a task. An example of a sub-task applicable to AQP would be: perform rotation and liftoff. • Element: A component of training analysis or design. In the case of task analysis, the element may be used as a level of analysis: phase of flight, task, subtask, element, sub-element (a further component of training analysis necessary in the accomplishment of a subtask (e.g., rotate aircraft at VR to 12 degrees of pitch)). In the case of curriculum design, the element may be used as a level of curriculum organization: curriculum, segment, module, lesson, lesson element, etc.

JTA is the method or procedure used to reduce a unit of work to its base components. The JTA document consists of a detailed, sequential listing of tasks, subtasks, and elements (if required) with the knowledge and skills (technical and Crew Resource Management (CRM)) that clearly define and completely describe the job. The following is a description of the above components:

a. Job Task List. The task analysis process begins with the development… › (3) Rationale.

a. Job Task List. The task analysis process begins with the development of a job task list that includes all of the major tasks performed by those who hold a particular duty or seat position. For example, a pilot-in-command (PIC) job task list includes all major activities involved in operating an aircraft, including conducting ground operations, performing takeoffs, etc. An F/A job task list includes all major activities involved in supporting passenger operations, such as performing check-in procedures, performing passenger boarding procedures, performing pre-takeoff cabin preparation procedures, etc. An instructor job task list includes all the major activities involved in teaching students, such as preparing training materials, managing the learning environment, operating training equipment, etc.

(1) Structure. For complex jobs, it is best to divide the job task into its subtasks and elements. Figure 2-2 illustrates how the pilot job task list can be divided, in this case, into 11 phases of operations. Each phase of operations can be then divided into job tasks, subtasks, elements, etc. For example, 2.0 Takeoff can be subdivided into 2.1 Perform Normal Takeoff, 2.2 Perform Instrument Takeoff, 2.3 Perform Engine Failure After V1 Takeoff, and 2.4 Perform Rejected Takeoff. Each of these job tasks can then be further divided into subtasks. For example 2.1 Perform Normal Takeoff, can be subdivided into 2.1.1 Assess Performance and Environmental Factors, 2.1.2 Perform Takeoff Roll, 2.1.3 Perform Rotation and Liftoff, and so on. Where necessary, these subtasks can be further subdivided into elements. For example, 2.1.3 Perform Rotation and Liftoff can be subdivided into 2.1.3.1 Rotate aircraft at VR to target pitch angle [PF], 2.1.3.2 Observe barometric/ADC altimeter increase [PF], 2.1.3.3 Call out positive rate [PM], etc.

(2) Hierarchy. By dividing tasks into subtasks, elements, and sub-elements, each job task listing produces a numbered hierarchy of job requirements for each duty position. These job requirements are essentially the graduation requirements for the courses from which they are developed. Through a series of additional analyses, these job requirements are translated into the training objectives of the various AQP curriculums: Indoctrination, Qualification, and Continuing Qualification. The tasks are translated into Terminal Proficiency Objectives (TPO) and the subtasks into Supporting Proficiency Objectives (SPO). Elements are translated into Enabling Objectives (EO). The hierarchical numbering system is retained as the basis of the audit trail that connects job requirements and performance with curriculum requirements and performance.

(3) Rationale. Proficiency-based training systems always begin with the development of a detailed task listing. This means that required job proficiency is the basis for designing, developing, operating and maintaining the training system. Task 1.1.1 will be taught in lesson 1.1.1, element 1.1.1, assessed using test item 1.1.1, and those test results used to validate that the individual/crew can perform task 1.1.1. This audit trail links job performance to training performance throughout every component of the training curriculum.

FIGURE 2-2. SAMPLE PILOT JOB TASK LISTING

1. Ground Operations

2.0 Takeoff

2.1 Perform Normal Takeoff 2.1.1 Assess Performance and Environmental Factors 2.1.2 Perform Takeoff Roll 2.1.3 Perform Rotation and Liftoff 2.1.3.1 Rotate aircraft at VR to target pitch angle [PF] 2.1.3.2 Observe barometric/ADC altimeter increase [PF] 2.1.3.3 Call out positive rate [PM] 2.1.3.4 Retract Gear [PF, PM] 2.1.3.5 Establish Climb Speed [PF] 2.2 Perform Instrument Takeoff 2.3 Perform Engine Failure After V1 Takeoff 2.4 Perform Rejected Takeoff

3. Climb Operations

4. Cruise Operations

5. Descent Operations

6. Approach Operations 6.1 Perform Approach 6.1.1 Perform Visual Approach 6.1.2 Perform Non-Precision Approach Procedures (VOR, NDB, LOC, LOC/BC, LDA, SDF, ASR, RNAV/FMS, GPS) 6.1.3 Perform CAT II ILS 6.1.4 Perform CAT IIIb ILS 6.1.5 Perform Coupled Autopilot Approach and Autoland Procedures 6.2 Perform One Engine Inoperative CAT I ILS Approach and Landing 6.3 Perform One Engine Inoperative Missed Approach 6.4 Perform Visual Approach and Rejected Landing

7. Landing Operations 7.1 Normal Configuration 7.2 Autoland 7.3 No-Flap

8. After Landing Operations

9. Aircraft Systems Operations

10. Abnormal and Emergency Procedures

11. Supplementary Procedures

a. Proficiency Objectives. A proficiency objective is the result of… › (1) TPOs.

b. Learning Analysis. The second part of the JTA is sometimes called competency analysis, skill analysis, knowledge skills and abilities (KSA) analysis, or hierarchical analysis. Here, those tasks, subtasks or elements that were selected for training are further analyzed into their more basic knowledge and skill level (attitudes are optional). This learning analysis will determine, to a finer level of detail, exactly what should be learned and the best approaches for teaching and testing what is to be learned. While the JTA adds greater specificity to the performance and training requirements of the tasks, the learning analysis defines in greater detail what should be taught and tested, and how it should be taught and tested, to ensure that the students acquire those job performance requirements.

c. JTA Document. As discussed in Chapter 1, the JTA document is one of six documents unique to AQP that is maintained in a current status throughout the program’s lifetime and must have an acceptable revision control methodology.

2-9. QUALIFICATION STANDARDS. A qualification standard is a job task proficiency objective (TPO or SPO) linked to an evaluation strategy. A certificate holder’s qualification standards define the requirements of mastery for specific duty positions and replace the applicable Airman Certification Standards (ACS) or practical test standards (PTS) for certification under AQP. The qualification standards document is the single most important part of any AQP. It provides the complete proficiency baseline for all duty positions and serves as the basis for the curriculum development for both the Qualification Curriculum and Continuing Qualification Curriculum. The first step in the development of qualification standards requires the development of proficiency objectives from the JTA.

a. Proficiency Objectives. A proficiency objective is the result of applying a performance statement, condition(s), and proficiency standard(s) to a task or a subtask. For each duty position, there are two types of proficiency objectives, both developed from the task analysis: TPOs, which are developed from tasks; and SPOs, which are developed from subtasks.

• A performance statement specifies precisely what behavior must be exhibited. • A condition statement identifies operational and equipment contingencies and environmental factors under which the behavior will be accomplished. • A standard or criterion statement establishes the parameters and tolerances that define satisfactory performance.

NOTE: All document references used in defining the performance, conditions, and standards for each proficiency objective must be listed by title and chapter in the documentation of the proficiency objectives in the qualification standards document.

(1) TPOs. TPOs are statements of performance, conditions, and standards established at the task level. A complete set of TPOs will fully describe a particular job in the applicant’s flight operation. TPOs are classified as critical and/or currency based on an operational assessment in the analysis process. This classification determines the frequency with which these tasks are evaluated during the continuing qualification cycle. TPOs also include the range of flight training equipment and the abnormal and emergency contingencies to be considered for training and evaluation.

a. Proficiency Objectives. A proficiency objective is the result of… › (4) Task Factors Analysis.

(2) SPOs. SPOs are statements of performance, conditions, and standards established at the subtask level. SPOs are used to develop training and evaluation curriculum lessons, modules, and segments. SPOs include the range of flight training equipment and the abnormal and emergency contingencies to be considered for training and evaluation.

(3) Enabling (Learning) Objectives (EO). EOs are used to prepare individuals and crews for subsequent training in an operational cockpit environment. An applicant may identify a certain knowledge factor, cognitive skill, motor skill, or CRM factor as an enabling proficiency objective. These are not normally carried forward in the supporting performance objective statement and, therefore, are not normally found on the qualifications standards document. However, performance of an SPO would depend on a student first acquiring the particular knowledge, skill, attitude, or CRM factor covered by an enabling objective.

NOTE: A learning objective (usually an enabling objective (EO)), which does not require operational conditions, can be demonstrated in a classroom or academic type setting. A performance objective (usually a TPO or SPO), which requires operational conditions, must be demonstrated in an environment equivalent to the operational environment.

(4) Task Factors Analysis. This process rates TPOs and SPOs using the following factors: criticality, currency, need for training, applicable conditions, and applicable standards. The determination of criticality and currency guides when and how the objective is trained, validated, or evaluated (see Figure 2-3). To make this determination, the analyst answers a series of questions about each TPO and SPO to describe its performance requirements, both on the line and in the training setting. Criticality is a determination of the relative impact of substandard task performance on overall safety. It indicates an increased need for awareness, care, exactness, accuracy, or correctness during task performance. Critical tasks are proficiency objectives that are trained, validated, or evaluated more frequently during an AQP evaluation period. A currency task is a proficiency objective for which individuals and/or crews maintain proficiency by repeated performance of the item in normal line, duty or work operations. Most currency items are validated during line checks and may be sampled in the Continuing Qualification Cycle. Tasks that are determined to be critical and not current are trained, validated, or evaluated each evaluation period. Tasks that are determined to be neither critical nor current are trained, validated, or evaluated each continuing qualification cycle.

NOTE: Many flightcrew job task SPOs do not fit the classic definition of a subtask; a specific separate step or activity required in the accomplishment of a task. In recognition, non-critical/non-current SPOs under a common TPO that differ only in knowledge requirements may be trained, validated, or evaluated in a Sim/flight training device (FTD), during line checks, using orals, written or electronic exams, classroom briefings or distributed material. However, it is recommended that these SPOs, as appropriate, be demonstrated in a Sim/FTD on a recurring cycle approved by the ERT.

(b) Additional factors: › 2. Criterion for success upon which performance standards are based.

The FAA recommends that the applicant examine each task, subtask, and element, as appropriate, for the following factors:

(a) Primary factors to be considered:

1. Statement of performance.

2. Environmental conditions affecting difficulty/success.

3. Performance standards (parameters with tolerances).

4. Abnormal and emergency procedure contingencies.

5. Document references (title and section) governing or specifying the operation.

6. Consequence of error to safety.

7. Relative difficulty.

(b) Additional factors:

1. Equipment and system operation dependencies (if used for establishing learning sequences for curriculum development).

2. Criterion for success upon which performance standards are based. If new performance standards are created, this criterion should be established for each task and subtasks (e.g., the tracking standards for very high frequency (VHF) omni-directional radio (VOR) approaches are based on navigation requirements). The navigation requirements are the criteria for success. Success criteria are developed in those cases where current standards are missing or thought to be inadequate.

FIGURE 2-3. AQP CONTINUING QUALIFICATION CRITICAL/CURRENCY CHART

Training

Priority

Critical

(Y/N)

Currency

(Y/N)

Terminal Proficiency Objective/Supporting Proficiency Objective

1 YES NO Train, validate, or evaluate each evaluation period. 2 NO NO Train, validate, or evaluate each Continuing Qualification Cycle. 3 YES YES Sample at first look/MV/LOE and/or line checks each evaluation period. 4 NO YES Sample at first look/MV/LOE and/or line checks each Continuing Qualification Cycle

b. Evaluation Strategy. The qualification standards document will… › (2) Conditions, Contingencies, and Media.

FIGURE 2-3. AQP CONTINUING QUALIFICATION CRITICAL/CURRENCY CHART

(CONTINUED)

Training

Priority

Critical Current Month Sample

1 Yes No 12 No 2 No No 24 No 3 Yes Yes 12 Yes 4 No Yes 24 Yes

NOTE: The sample size must be large enough to provide reasonable assurance that the population is remaining proficient.

NOTE: The qualification standards document for instructors/evaluators does not need to include conditions or a criticality/currency analysis.

b. Evaluation Strategy. The qualification standards document will identify the curriculum (indoctrination, qualification, and/or continuing qualification) in which specific proficiency objectives will be met. The applicant should consider student entry level in determining this allocation. All terminal proficiency objectives must be included in a qualification curriculum regardless of entry-level analysis. For SPOs, an entry-level analysis determines what objectives will be taught under each curriculum (see Section 3, subparagraph 2-11a of this chapter). All objectives should also be covered in continuing qualification test and evaluation strategies.

(1) Consolidation of Objectives. In the qualification standards document, qualification standards are developed at the task and subtask level only and at no lower level. Tasks become TPOs and subtasks become SPOs by combining performance statements, conditions and standards. TPOs and SPOs having common knowledge, skill, attitude, and/or CRM factors may be consolidated to avoid duplication. The consolidated tasks are translated into TPOs, and a terminal level qualification standard is developed for each one. The consolidated subtasks are translated into supporting proficiency objectives, and a supporting level qualification standard is developed for each one of those as well. An example of consolidation would be non-precision approaches. VOR and Non-Directional Beacon (NDB) approaches may be consolidated as a single proficiency objective if the performance statement and standards are the same.

(2) Conditions, Contingencies, and Media. The qualification standards will include a listing of relevant operational and environmental factors along with equipment contingencies to be addressed in training. They will also identify the media that will be employed to either test, validate, or evaluate the specific training objective. There is a difference between the qualification standard for a qualification curriculum and one for a continuing qualification curriculum. The TPOs and SPOs in a qualification standard for a qualification curriculum must identify the specific set of conditions and contingencies to be employed in training and evaluating a task (usually with an asterisk). In addition, the media description will describe the specific media in which the task will receive final evaluation. In contrast, in a continuing qualification curriculum, the TPOs and SPOs may identify a selectable menu of conditions and contingencies to be employed in training or evaluation, and a media range that will specify the minimum media level in which the task may be trained, validated, or evaluated.

c. Qualification Standards Document. The qualification standards… › (3) Testing/Validation/Evaluation and Remediation Methodology.

(3) Qualification and Continuing Qualification Curriculums. The qualification standards for both qualification and continuing qualification curricula can be combined into one document as long as the differences in conditions and media are addressed, and the tasks are annotated for applicable curriculums. However, if applicants find it more convenient to submit separate qualification standards documents for these curriculums (e.g., because of differences in conditions, contingencies, and media between the qualification and continuing qualification curricula), they may do so. In addition, some qualification standards may be “aircraft generic” in that they may apply to more than one aircraft type. For ease of use and definition of “fleet common” curricula, applicants may also find it convenient to specify these qualification standards separately from those qualification standards that apply to only one aircraft fleet type.

c. Qualification Standards Document. The qualification standards document is the third of the six documents unique to AQP that is maintained in a current status throughout the life of the program and must have a revision control methodology. It is the central AQP document because it provides the regulatory basis for all deviations from current regulations, and identifies the basic training and evaluation requirements. The qualification standards document has four parts: the prologue, a regulatory comparison, the evaluation/remediation methodology, and the specific TPO or SPO qualification standard.

(1) Prologue. The qualification standards document requires an introductory section that explains the methodology, format, and terminology of the standards to the broad range of company and FAA personnel who will need an in-depth understanding of the document to perform their job functions.

(2) Regulatory Requirements Comparison. The qualification standards document must also include regulatory comparison information. The comparison must meet the requirement of AQP regulatory guidance, which states that the AQP program “...must indicate specifically the requirements of part 61, 63, 65, 121, or 135, as applicable, that would be replaced by an AQP curriculum.” (see Appendix 3 for an example of a regulatory comparison chart).

NOTE: The purpose of these charts is not to justify differences from current traditional practices, but merely to document them once they have been justified. It is the quality control processes of AQP that form the basis for establishing an equivalent level of safety.

(3) Testing/Validation/Evaluation and Remediation Methodology. Before implementing an AQP curriculum, the applicant must decide how, when, where, and who will assess a student’s proficiency on each terminal and supporting objective. The guidelines in Figure 2-3 may be used to translate TPOs’ and SPOs’ criticality and currency ratings into testing strategies for the continuing qualification cycle. This testing strategy may include a discussion describing how similar SPOs will be addressed. For example, the TPO non-precision approach may have many SPOs, such as VOR, NDB, Localizer (LOC), etc. Depending on the criticality/currency rating, all of these approaches do not have to be evaluated each evaluation period. This section of the document is where the applicant can describe how these approaches will be alternated/sampled over multiple evaluation periods or continuing qualification cycles. The testing/validation/evaluation methodology also addresses the applicant’s approach to

(3) Testing/Validation/Evaluation and Remediation Methodology. › (c) Rating Scale/Scoring.

documenting the different qualifications of the instructor/evaluator, in terms of who will conduct the testing, validation, LOE, and line checks. In this section, the applicant can describe the level of device appropriate for conducting the evaluation. It also identifies the point in the curriculum when the testing/validation/evaluation methodology will be applied, and identifies what constitutes a failure and/or unsatisfactory performance. In addition, the applicant must specify the strategy for remedying unsatisfactory performance.

(a) General. For each terminal and supporting proficiency objective, the applicant will designate the testing/validation/evaluation strategy. These strategies could include, and are not limited to:

1. Train to proficiency.

2. Systems/knowledge validation.

3. Procedures validation.

4. Maneuvers validation.

5. LOE.

6. Operating experience.

7. Line check.

(b) Training Media. The applicant also will designate the level of training devices, simulators, or aircraft to be used to evaluate the proficiency objective.

(c) Rating Scale/Scoring. The applicant will identify the rating methodology that will be used to grade the performance of the proficiency objectives against the qualification standards. Typically, the measurement codes associated with performance events are ratings and reason codes or skill categories. Ratings are used to define different quality levels of performance. Rating codes usually are carrier specific and the FAA requires that something more sensitive to performance differences than a binary code is used (i.e., some rating method that provides more performance differentiation than pass/fail for individual items being evaluated. Results of an evaluation event, such as an LOE, are still recorded on a pass/fail basis). Each carrier should ensure that the grades established on the rating scale are clearly defined, meaningful to the instructor/evaluator (I/E), and easily used for performance assessment. Although consistency among fleets and across different types of evaluations (line check, maneuvers validation, and LOE) is important and generally desirable, rating scales may be slightly different when used for different purposes such as training vs. evaluation. Figure 2-4 provides an example of a rating scale that discriminates among performance levels:

(3) Testing/Validation/Evaluation and Remediation Methodology. › (e) Special Tracking.

FIGURE 2-4. RATING SCALE FOR PROCEDURES VALIDATION, FIRST-LOOK, MANEUVERS

VALIDATION, LOE, OR LINE CHECK

GRADE CRITERIA

1 Unsatisfactory

Major deviations from the prescribed qualification standards occur that are not recognized or corrected. Individual or crew performance could result in hull loss or loss of life. CRM/DRM skills are not effective.

2 Below Standard

Deviations from the prescribed qualification standards occur that are not recognized or corrected. Individual or crew performance is safe but would be unsatisfactory if diminished by any amount. CRM/DRM skills are not completely effective.

3 Standard with Debrief

Deviations occur from the prescribed qualification standards that are recognized and most corrected. Individual or crew performance meets expectations. CRM/DRM skills are effective.

4 Standard

Minor deviations occur from the prescribed qualification standards that are recognized and corrected in a timely manner. Individual or crew performance meets expectations. CRM/DRM skills are clearly effective.

5 Excellent

Performance remains well within the prescribed qualification standards. Individual or crew performance, management and CRM/DRM skills are exemplary.

NOTE: This example should not be taken as limiting possible intervals to a five-point scale. With appropriate scale construction and instructor/evaluator training, carriers may elect to define other scales that maximize the quality (sensitivity, reliability, validity) of the collected data.

(d) Remediation Strategy. This section of the qualification standards document should describe the methodology that will be used to remediate unsuccessful testing, validation, or evaluation sessions. This remediation strategy must detail when and what may be repeated and whether or not additional training is warranted. Remediation strategy must also specify when no more training will be offered to the individual and the resulting actions such as “Referred to Committee,” returned to previous position, etc. Chapter 3, subparagraph 3-4c has expanded discussions on validation, evaluation, and remediation in the qualification and continuing qualification curriculums. This strategy may be presented in narrative text or flowchart format.

(e) Special Tracking. Special tracking is the assignment of an individual trainee to an augmented training or evaluation schedule, or both. It should be applied to individuals who have failed to demonstrate proficiency during an evaluation event (e.g., LOE). There may be other criteria that the certificate holder may use to place an individual on special tracking. Special tracking triggers could include failure to maintain proficiency, an extended absence from duty, for new PICs, or at the request of the individual trainee. However, the AQP participant may implement an AQP strategy such as reduced training intervals that would offset the need for a special tracking strategy. This section should discuss the following:

(4) The Qualification Standard. › (h) Criticality/Currency Rating.

1. The situation(s) that requires an individual to be placed in special tracking.

2. The strategy to be used.

3. When special tracking is no longer required.

(4) The Qualification Standard. The information contained in the specific qualification standard is the basis for determining proficiency and evaluation criteria. The qualification standard is constructed by applying a performance statement, conditions, and standards to a task or subtask, thereby creating a TPO or an SPO. Although each operator will determine the format and content of its qualification standards, component fields have developed out of practice and are illustrated in Appendix 5. In this example, the phase of operations is 6.0 Approach Operations and Landing. The TPO is 6.1 Perform an Instrument Approach. The SPOs are 6.1.1 Perform a 2 Engine Precision Approach CAT I ILS (instrument landing system) Approach and 6.1.2 Perform a 1 Engine Inoperative CAT I ILS Approach.

NOTE: Variation in the format of a given air carrier’s qualification standards is permissible if all of the categories of information in the example are addressed.

(a) Header. A header identifies the airline and the document.

(b) Revision Control. Page revision control dates and revision numbers.

(c) Page Numbers. Consecutive page numbers.

(d) Phase of Operations. Number and title from task listing.

(e) Qualification Standard Title. Either TPO(s) or SPO(s).

(f) Hierarchical Task or Subtask. Identifier and title from task listing.

(g) Crew Duty Position(s). This identifies which crewmember(s) will be evaluated performing the task.

1. Pilot in Command = PIC.

2. Second in Command = SIC.

3. Flight Engineer = FE.

4. Captain, First Officer, Second Officer.

(h) Criticality/Currency Rating. This may be the first place that the task factors analysis is tied to the tasks.

NOTE: The qualification standards document for instructor/evaluators does not need to include conditions or a criticality/currency analysis.

Section 3. PHASE II: CURRICULUM DEVELOPMENT › 2-10. INSTRUCTIONAL SYSTEMS DEVELOPMENT METHODOLOGY DOCUMENT.

(i) Curriculum. This field identifies the curriculum(s) in which the task will be trained and evaluated.

(j) Evaluation Strategy. The evaluation point for a particular qualification standard (e.g., (1) train to proficiency; (2) systems validation; (3) procedures validation; (4) maneuvers validation; (5) LOE; or (6) line check).

(k) Media. The specific media in which training and/or evaluation will be conducted. For qualification, the media is the lowest media used for final evaluation. For continuing qualification, the media includes the range of media used for training, validation, and evaluation.

(l) Performance Statement. An expanded statement of expected behavior that, when executed, will complete the work required for a specific portion of a job. A performance statement specifies precisely what behavior must be exhibited, and may include the knowledge and skill issues that comprise the enabling objective supporting that performance.

(m) Operational and Environmental Conditions. Conditions describe the range of circumstances under which student performance will be measured and evaluated. Conditions include the operational environment (Navigational Aid (NAVAID) inoperable, different aircraft weight, passengers not seated, aircraft configuration, etc.), and natural environment (ceiling, visibility, wind, turbulence, etc.). The qualification standard should: (1) indicate those specific conditions to be trained and tested as part of the qualification curriculum; and (2) provide a more exhaustive listing of conditions over which crewmembers will be trained and tested during the course of successive continuing qualification cycles.

(n) Contingencies. Contingencies include abnormal situations, minimum equipment list (MEL)/Configuration Deviation List (CDL), and emergencies. The qualification standard should (1) indicate those specific contingencies to be trained and tested as part of the qualification curriculum; and (2) provide a more exhaustive listing of contingencies over which crewmembers will be trained and tested during the course of successive continuing qualification cycles.

(o) Maneuver Standards. Observable, measurable parameters of performance with tolerances (e.g., course deviation degrees (+ or -)). Standards include maneuvers, procedures, and CRM considerations.

(p) References. Identify the primary references from which performance statements and associated standards were derived. Cite documents by title and where applicable, chapter or section. Page numbers are not required.

2-10. INSTRUCTIONAL SYSTEMS DEVELOPMENT METHODOLOGY DOCUMENT. This is another of the six documents unique to AQP and must have an acceptable revision control methodology. Applicants with established curriculum development guidelines may submit these for consideration. Others should describe a systematic approach for developing a proficiency-based training system that is organized around the teaching and testing of terminal, supporting, and enabling proficiency objectives. The methodology identifies the rationale, justification, and subsequent documentation to be used in the applicant’s curriculum

2-10. INSTRUCTIONAL SYSTEMS DEVELOPMENT METHODOLOGY DOCUMENT. › a. Curriculum Development Process. Applicants should describe the…

development process. The instructional systems development methodology document describes the approach to be used by applicant airlines to develop and maintain all AQP curriculums. The Instructional Systems Development Methodology document should be finalized before constructing curriculums for each duty position. It applies to pilot, instructor, and evaluator programs, and may be expanded to include other personnel, such as F/As and dispatchers. This document is divided into two sections. The first section, Curriculum Development Procedures, describes the applicant’s approach for using the JTAs and qualification standards as baseline documents to construct their general training curriculums across all AQP courses. The second section, Line Operational Simulation (LOS) Methodology, describes the approach for developing LOS scenarios.

a. Curriculum Development Process. Applicants should describe the process they will use to build their curriculums based on the JTA, qualification standards, and proficiency objectives they develop for each duty position. This document should discuss how:

• TPOs and SPOs are allocated to curriculums. • Learning and evaluation activities are developed to support these objectives. • Instructional media and methods are assigned to objectives. • Objectives are clustered and sequenced into lessons, modules, segments, and curriculums (see Figure 2-5 for an example). • An audit trail (hierarchical numbering or a matrix) will be maintained to link job tasks, proficiency objectives, lesson activities/content, and test items.

NOTE: The resulting curriculums are translated into a course footprint and are documented in the curriculum outline. These curriculums are expanded in more detail in the student and instructor syllabi and in individual lessons and tests.

FIGURE 2-5. CURRICULUM DEVELOPMENT

2-11. CURRICULUM OUTLINE DOCUMENT. › a. Entry-Level Analysis. The applicant should develop and document a…

b. LOS Methodology. While individual LOE scenario events are approved by the POI, the ERT must first approve the methodology that generates LOE scenarios. Unless the carrier has an alternative proposal, this methodology should follow the general guidelines for LOS development produced by the Airlines for America (A4A) AQP Working Group. This approach divides the typical scenario into a series of relatively independent segments, called event sets. A typical scenario might have six or eight event sets relating to the phases of operations (ground operations, takeoff, climb, cruise, descent, approach, landing, and after landing). Each event set consists of a series of training or evaluation events, which include both technical and CRM activities. This technique enables scenarios to be constructed in a building block approach, ensuring that each event set is carefully sequenced and considered in relation to the other event sets in the scenario.

2-11. CURRICULUM OUTLINE DOCUMENT. This is the fifth of the six documents unique to AQP that must have an acceptable revision control methodology. The curriculum outline is a listing of course material divided into segments, segments into modules, modules into lessons, and lessons into elements. Curriculum outlines are developed and submitted with the understanding that application of the course material may require some flexibility regarding the actual day on which each activity is accomplished. Each part of the curriculum outline must clearly indicate the subject matter to be taught and correspond directly to the hierarchical system of the task analysis. While the curriculum outline document need only go down to the level of the element under each lesson title, the applicant will be required to show the objectives (terminal, supporting and enabling) associated with each lesson. See Figure 2-5 for an illustration of this association. This is part of the necessary audit trail that links the job requirements (JTA) to the training requirements (qualification standards) to the training activities (curriculum outline). A curriculum outline provides the basis for the curriculum footprint, which is a high level graphical overview of the curriculum content depicting the training and evaluation activities and the planned hours for each day of the curriculum (see Figure 3-4). The curriculum outline document should reference the results of the student entry analysis, if one was conducted, and will include a curriculum footprint. Figure 2-7 provides an example of a curriculum outline showing portions of ground training and flight training segments.

a. Entry-Level Analysis. The applicant should develop and document a student entry-level performance analysis for TPOs and SPOs. This analysis compares the KSAs of the student population against the TPOs and SPOs in the JTA in order to fit the instruction to the student. A four-point performance difference rating scale is suggested (see Figure 2-6). Highly skilled instructors familiar with the experience and background of the student population and knowledgeable with the terminal and supporting proficiency objectives should make the rating. This analysis provides guidance to determine efficient teaching strategies for the indoctrination and qualification curriculums. This analysis can also identify where training is not needed, where basic “enabling” skills must be taught, and what number of trials is needed for an applicant to reach TPO standards. More than one population group may be used in conducting the student entry analysis for a single duty position. The results of such an analysis may be used, for example, to justify alternative curriculum tracks or modules targeted at expected differences in entry background, and in order to achieve the most efficient use of training resources. The initial justification for alternative curriculum tracks must be validated though subsequent data analysis.

2-11. CURRICULUM OUTLINE DOCUMENT. › c. Link Qualification Standards to Curriculums. Both the qualification…

FIGURE 2-6. PERFORMANCE DIFFERENCE RATING SCALE

Performance

Difference

Code

Performance Difference Description

4 Meets or exceeds the required performance. 3 Can accomplish tasks with minor errors or omissions. May take longer than expected or allowed. 2 Cannot accomplish tasks. Does demonstrate basic background skills and knowledge. 1 Does not demonstrate basic background experience, skills, or knowledge. Unfamiliar with simplest elements of a task.

b. Trainee and Instructor/Evaluator’s Curriculums. The instructor and evaluator curriculums associated with each duty position must be developed in the same systematic manner as those developed for the duty positions themselves. This requires the development of separate JTA, qualification standards, curriculum outlines and other documents for these positions although they may share some common modules or lessons. Instructors and evaluators require an indoctrination curriculum that is separate from the duty position indoctrination curriculum, as well as a separate qualification and continuing qualification curriculum.

c. Link Qualification Standards to Curriculums. Both the qualification and continuing qualification curriculums for a given duty position are derived from the same set of qualification standards. The link between qualification standards and a curriculum is the proficiency objective. As a rule of thumb, qualification standards developed from TPOs focus more on the higher-level evaluation activities than qualification standards developed from SPOs. Qualification standards developed from SPOs focus more on the lower level learning activities that may be validated and represent the components of those higher level standards. Therefore, a qualification curriculum will focus equally on qualification standards developed from TPOs and SPOs, while the continuing qualification curriculum will focus more heavily on the qualification standards developed from TPOs. Indoctrination curriculums may or may not have qualification standards of their own. In either case, they provide training on common knowledge items (enabling objectives) that will support the mastery of qualification standards in the qualification and continuing qualification curriculums.

FIGURE 2-7. SAMPLE AQP CURRICULUM OUTLINE

B737 Qualification Curriculum Outline

Segment: Ground Training

Module: Introduction to Aircraft

Lesson #: Aircraft Overview 9.1.4

Element: Fuselage 9.1.4.1 Element: Wings 9.1.4.2 Element: Flight Controls 9.1.4.4, 9.1.4.5, 9.1.4.6 Element: Landing Gear 9.1.4.7 Element: Powerplant 9.1.4.3 Element: Fuel System 9.1.4.8 Element: Hydraulic System 9.1.4.9 Element: Electrical System 9.1.4.10, 9.1.4.10.1, 9.1.4.10.2, 9.1.4.10.3

Lesson #: Aircraft Lighting 9.1.9

Element: Exterior Lighting 9.1.9.1 Element: Cockpit Lighting 9.1.9.2 Element: Cabin Signs and Lights 9.1.9.9, 9.1.9.4, 9.1.9.6 Element: Lighting Power Sources 9.1.9.5 Element: Emergency Lighting 9.1.9.5, 9.1.9.5.1, 9.1.4.10.2

B737 Qualification Curriculum Outline

Segment: Flight Training (FT)

Module: Flight Training Device (FTD)

Lesson: FTD #1 Pre-Flight Normal Checklists

Element: Flight Compartment Safety Inspection 1.2.5.1 Element: Flows 1.2.7.1 Element: Acceptance Checklist 1.2.8.1 Element: Before Starting Engines Checklist 1.2.11.1 Element: Normal APU First Engine Start 1.9.1.1 Element: Normal APU Second Engine Start 1.9.1.4 Element: Pushback 1.9.6.1 Element: Before Taxi Checklist 1.4.1.1 Element: Normal Taxi 1.4.2.1 Element: Before Takeoff Checklist 1.4.9.1 Element: Line-Up Checklist 1.4.4.1

b. Operations. This section of the I&O Plan describes the guidance and… › (1) AQP Maintenance Strategy for Phases IV and V.

2-12. IMPLEMENTATION AND OPERATIONS PLAN (I&O PLAN) DOCUMENT. This document, like the other five unique AQP documents, must have an acceptable revision control methodology. It must be updated as necessary to accurately reflect the status of the AQP applicant’s plan for implementing and operating each of the AQP curriculums. This document is a milestone schedule detailing the transition to an AQP for crewmembers, instructors, and evaluators and a blueprint describing provisions for maintenance, administration, data management, and continuing quality control of curriculums. As a document, the I&O Plan can be sectioned into two parts. The first part spells out how the operator proposes to implement the AQP. Included in this proposal is the schedule for Phase III, training and evaluation to include I/E training, and small group tryouts. It should also include provisos for evaluating the effectiveness of performance measurement tools, and provisions for evaluating facilities, courseware, and equipment before starting the plans for the small group tryouts. The second part explains how the certificate holder intends to operate the AQP in Phases IV and V. Included in this section are strategies for maintaining the program, crew pairing policy, first-look administration, and I/E requirements. The operations plan should also include the data management plan, a statement of understanding addressing the collection and analysis of performance/proficiency data, a description of the Performance Proficiency Database (PPDB), the data management collection process, and the FAA data submission, analysis, and reporting requirements.

a. Implementation. This section provides the schedule for evaluating curricula in the small group tryout, as well as I/E training, and the strategies for evaluating facilities, courseware, and equipment. It also includes the proposal for evaluating instructors, evaluators, and performance measurement tools such as the rating scale and grade sheets. The schedule for initiating Phase III should correlate with the Master AQP Transition Schedule as submitted in the application. This part of the I&O Plan is updated each time a new curriculum is added to the AQP and a small group tryout is planned.

NOTE: If the certificate holder is requesting no-jeopardy credit for the students in the small group tryout, it must be indicated in the I&O Plan and requested in writing to the appropriate Safety Standards office manager and the POI.

b. Operations. This section of the I&O Plan describes the guidance and policies that will be used to provide for AQP maintenance, first-look administration, crew pairing policy, instructor/evaluator requirements and data management. Once established, the operations section remains reasonably stable and does not necessarily change with the addition of a new curriculum.

(1) AQP Maintenance Strategy for Phases IV and V. This section describes quality control procedures (i.e., plans for acquiring and measuring data for tracking curricula, students, instructors, and evaluator performance). It also includes the strategy employed for curriculum maintenance and update. Maintenance includes the methodology for maintaining control of the AQP. It includes approval documents, maintaining curriculum currency, upgrading equipment, monitoring and responding to demographic changes, and for using training/evaluation feedback from the PPDB and other forms of surveillance to maintain and improve the AQP.

(5) Data Plan. › (b) Data Management.

(2) First-Look Activity Administration. First-look performance items are graded procedures/maneuvers performed for the first time since the previous training cycle. The “first-look” grades are analyzed to determine trends of degraded proficiency due to numerous factors, including the length of the training interval. To maintain the validity of the performance proficiency data obtained from the performance of the “first-look” activities, this section must describe the strategy employed for those activities. This strategy must state that information or techniques that would unduly interfere with the “first-look” performance validity will not be briefed before the first execution of these items. In addition, this strategy should discuss how first-look maneuvers are selected and how they will be administered.

(3) Identify LOS Crew Scheduling and Pairing Strategy. A basic requirement of AQP is to train and evaluate crewmembers in a crew configuration identical to line operations. In AQP, line crewmembers must be scheduled and paired together, as much as practical, in a standard crew configuration (e.g., line captain with line first officer). The FAA recognizes that circumstances will occur where the initial composition of the schedule cannot be maintained. Hiring requirements, illness, high first officer to captain ratios, or failure of a crewmember to progress, are all situations that would necessitate providing a seat substitute to complete the training. This section must address the decision rules that will apply to the selection of seat substitutes for those circumstances. In all cases, the seat substitute must be task-familiar with the duty position.

(4) Instructor/Evaluator Requirements. This part will address the specific job functions, training, validation, or evaluation that instructors and evaluators are authorized to perform. It will identify the title of each position and describe the training that each receives in order to perform the job function associated with that event. The table in Chapter 5, Figure 5-1 illustrates the level of authorization needed for an individual to either train, validate, or evaluate an AQP event.

(5) Data Plan. Before an applicant can proceed with data collection and analysis, it must establish the intended purpose and method for the collection, entry, reporting, and analysis of AQP training/evaluation data for each curriculum. The plan must be thorough and accurately reflect the airline’s PPDB system. The AQP applicant must also acknowledge its responsibility to collect and analyze more data than required for submission to the FAA in order to adequately identify performance trends and requisite changes to factors that impact the performance. For example, the data that is submitted to the FAA is the result of the execution of TPOs and SPOs and is analyzed at that level by the FAA. See Chapter 8 for further explanation.

(a) Data Collection. Should address the methods used to collect performance/proficiency data for all curriculums. These methods will include the rationale for employing the method as well as providing the data input medium (e.g., grade sheets, computer-input screens, etc.) as examples that exemplify the data acquisition rationale. In addition, the data collection method should explain data input quality control, security, and usability.

(b) Data Management. Should explain the means and strategy the airline intends to employ to enter, access, and assimilate the AQP and the Single Visit Training Program (SVTP) performance/proficiency data that is collected. Included in this explanation should be:

2-14. OVERVIEW. › c. Conduct the Approved Curriculum. This training and evaluation will…

1. The type of software data management system employed (e.g., relational database, spreadsheet etc.).

2. The organization of the information in the electronic medium (e.g., database definition, database table relationships, spreadsheet description, etc.).

3. A description of the user interface to this data management system.

(c) Data Analysis. Discuss the type of analysis that will be employed to facilitate the AQP performance information needs of the airline and the FAA. This discussion of the data analysis must address how each type of AQP data will be analyzed, including training and evaluation feedback as part of determining the effectiveness of the program. This discussion should be used as a prologue to the annual AQP report.

(d) Data Reporting. Discuss the FAA AQP data reporting requirements that must be met (to include format and frequency). In addition, discuss the internal quality assurance data reporting that will be employed, to include report types, frequency, and for which airline personnel the reports are intended.

2-13. APPROVAL. Approval of above documents marks the end of Phase II and the beginning of Phase III.

SECTION 4. PHASE III: SMALL GROUP TRYOUT (IMPLEMENTATION)

2-14. OVERVIEW. In Phase III, the AQP applicant will acquire and test the resources essential to support one or more of the indoctrination, qualification, or continuing qualification curriculums. These activities include developing courseware to implement the curriculum(s), training instructors and evaluators, conducting small group tryouts, making program revisions, and submitting data.

a. Resources. During this phase, the applicant will secure the training facility and equip classrooms with training aids, courseware, and other features that contribute to creating and maintaining a positive learning environment.

b. Train Instructors and Evaluators. The applicant must train, evaluate, qualify and provide an opportunity for the FAA to observe, if necessary, their instructors and evaluators before and during the small group tryouts (see Chapter 5 for more information on the training, observation, and quality control of instructors and evaluators).

c. Conduct the Approved Curriculum. This training and evaluation will consist of small group tryouts of all lessons using actual students and instructor/evaluators. The evaluation may involve no-jeopardy credit for students, since its primary purpose is to determine lesson suitability and effectiveness. The applicant may choose, however, to give student credit for part or all training and qualification achieved in this phase. The decision to give credit must be approved by the FAA before conducting the curriculum evaluation and the air carrier’s request to do so should be submitted by letter to the appropriate Safety Standards office manager and the POI.

2-17. PHASE IV ACTIVITIES. › d. The applicant and the FAA to support analysis for special subjects,…

d. Performance/Proficiency Data Submission. The applicant will submit to the appropriate Safety Standards office manager, on a monthly basis, the required performance and proficiency data. This data will be in electronic, digitized format. Any changes to format or procedures will be discussed with the appropriate Safety Standards office manager prior to submission. The monthly submission of this performance/proficiency data will continue for the life of the curriculum’s lifetime (i.e., through Phases III, IV, and V). See Chapter 8 for more information.

e. Program Revisions. Lessons learned as a result of this phase will be incorporated as changes to the approved AQP documents from Phase I and II and in curriculum maintenance reports submitted to the FAA on an annual basis. Improvements will be made to the courseware, hardware, software, personnel, organization, and data collection and reporting system of the AQP, based on system performance and the examination of performance and proficiency data. These enhancements will be implemented using the processes and the procedures described in the applicant’s approved AQP maintenance strategy. This plan is part of the I&O Plan.

2-15. APPROVAL. Approval of the initial operations marks the end of Phase III and the beginning of Phase IV.

SECTION 5. PHASE IV: INITIAL OPERATIONS

2-16. OVERVIEW. In this phase, the applicant implements the indoctrination, qualification, or continuing qualification curriculum in accordance with the approved AQP documents. This phase for the continuing qualification curriculum calls for initial operation of the curriculum for a minimum of 2 years to complete the AQP training cycle. The 2-year Phase IV period may not necessarily be applicable to the indoctrination and qualification curriculums. Unlike the Continuing Qualification curriculums, these courses of training are not cycle-dependent. Depending on the frequency that each curriculum is applied (new hires) and the successful results of each curriculum, Phase IV may be longer or shorter than 24 months.

2-17. PHASE IV ACTIVITIES. During Phase IV, the applicant will implement and complete a full evaluation, including collecting program audit data and individual performance/proficiency data, analyses, and reports. Collected data will be used by:

a. The applicant for its internal quality control program to maintain curriculum and courseware concurrence, suitability, and adequacy.

b. The applicant to analyze and validate crewmembers, dispatchers, and other operations personnel performance.

c. The applicant and the FAA to analyze and validate instructor/evaluators performance.

d. The applicant and the FAA to support analysis for special subjects, such as CRM performance factors.

SECTION 7. DOCUMENT AND REPORTING REQUIREMENTS › 2-23. OVERVIEW.

e. The FAA to analyze and validate curriculum performance.

f. The FAA to analyze and validate program development, implementation, and maintenance procedures.

2-18. REVISIONS. The focus of this phase is the validation of the AQP curriculum by acquiring performance/proficiency data and lessons learned in the course of conducting the curriculum. Applicants will summarize the lessons learned and adjustments made to the curriculums in an annual report. In addition, adjustments made to the AQP will be reflected in revisions to the approved AQP documents. The successful completion of this phase, and approval of the commensurate revisions to the approved AQP documents and reports, will qualify an applicant for entry into the final phase of the AQP process: Phase V, Continuing Operations.

2-19. APPROVAL. Approval of the continuing operations marks the end of Phase IV and the entrance into Phase V.

SECTION 6. PHASE V: CONTINUING OPERATIONS

2-20. OVERVIEW. In this phase, the applicant continues operation of the AQP unless the FAA withdraws approval or unless the applicant withdraws or modifies the AQP. This phase requires the maintenance of the AQP-approved documentation and updated documentation of the data requirements for all curriculums.

2-21. DATA’S IMPACT ON CONTINUING OPERATIONS. Data will continue to be collected and analyzed by the applicant and the FAA as it was during Phase IV activities.

2-22. QUALITY ASSURANCE. Applicants must pay particular attention to overall program quality. The FAA expects an AQP quality assurance program to identify needed changes in curriculum, courseware, and equipment, and to make these changes before an unwanted trend in reduced proficiency manifests itself. Continued validation of performance/proficiency data as it pertains to individual and crew proficiency, as achieved and maintained by all personnel, is particularly important.

SECTION 7. DOCUMENT AND REPORTING REQUIREMENTS

2-23. OVERVIEW. The approved AQP documentation establishes the applicant’s regulatory requirements for the particular training program. This fact makes it imperative that the applicant develops a document structure that is organized and standardized. The structure must ensure that the information contained within the AQP documents will be used and translated down to the lowest level training guidance. It must allow the applicant’s entire organization to easily access and use the contents. It is important to draw a distinction between the AQP process and the required FAA documentation each applicant must provide to ensure regulatory compliance. The FAA has established a minimum listing of AQP documentation. The applicant will want to develop other more specific materials that ensure the approved AQP information distills to all training and evaluation guides.

Continue reading ↓ 120 of 394 passages shown