Order 8100.7E Aircraft Certification Systems Evaluation Program (ACSEP)
This document is cancelled. Cancelled by 8120.23. Check the official source before relying on it.
Text read by OCR from a scanned file (3 of 152 pages). Check the original, as some words may be misread.
PAGE CHANGE CONTROL CHART › 8/31/2010
CHANGE U.S. DEPARTMENT OF TRANSPORTATION
FEDERAL AVIATION ADMINISTRATION
ORDER 8100.7E CHG 1 National Policy
Effective Date: 08/24/2011
SUBJ: Aircraft Certification Systems Evaluation Program
1. Purpose. This change removes Notes 1 and 2 from paragraph 4-14f. The notes state aviation safety engineers (ASEs) who are involved in conducting an Aircraft Certification Systems Evaluation Program (ACSEP) evaluation will gain experience in conducting product audits by assisting aviation safety inspectors (ASIs) during scheduled ACSEP evaluations. These notes have been removed because ASEs do not need to gain experience in this area to perform their duties; therefore, it should not be a requirement.
2. Who This Change Affects. This change to Order 8100.7E will be used by the following offices when conducting an ACSEP activity or during training: the Washington headquarters branch levels of the Aircraft Certification Service, the branch level in the Aircraft Certification Service field offices, the Aircraft Certification Branch at the Federal Aviation Administration Academy, and the Regulatory Support Division of the Flight Standards Service.
3. Disposition of Transmittal Paragraph. Retain this transmittal sheet until the directive is cancelled by a new directive.
PAGE CHANGE CONTROL CHART
Remove Pages Dated Insert Pages Dated
4-6 8/31/2010 4-6 8/24/2011 4-7 and 4-8 8/31/2010 4-7 and 4-8 8/24/2011
Distribution: Electronic Only Initiated By: AIR-200
Distribution: A-W (IR/FS/VN) -3; A-X (CD/FS) -3; A-FFS-0 (LTD); A-FAC-0 (ALL); AMA-220 (10 copies); AMA-250 (10 copies); AFS-600 (10 copies)
Initiated By: AIR-200
This order provides guidance and assigns responsibility for the implementation of the Aircraft Certification Systems Evaluation Program. This program uses Federal Aviation Administration (FAA) engineering, flight test, and manufacturing inspection personnel to evaluate FAA-approved type design after initial approval by the FAA at production approval holders (PAH) and associated facilities.
This order has been organized into two functional components. The first consists of chapters 1 through 5, which describe the Aircraft Certification Systems Evaluation Program (ACSEP) evaluation process, including the pre- and post-evaluation activities and the certification management information system. The second consists of appendixes A through L, which describe ACSEP documentation and reporting activities.
National Policy
SUBJ: Aircraft Certification Systems Evaluation Program
ORDER
8100.7E 08/31/2010
08/31/2010 8100.7E
ii
Table of Contents
Chapter 1. General
Paragraph Page
1-1. Purpose of This Order … 1-1 1-2. Audience … 1-1 1-3. Where Can I Find This Order … 1-2 1-4. Cancellation … 1-2 1-5. Explanation of Policy Changes … 1-2 1-6. Effective Date … 1-2
Chapter 2. ACSEP Evaluator Appointment and Training
2-1. General … 2-1 2-2. Appointing Officials … 2-1 2-3. Criteria for Candidate Selection … 2-1 Figure 2-1. Criteria for Candidate Selection and Team Member
Appointment … 2-2 2-4. Criteria for Appointment … 2-2 Figure 2-2. Criteria for Team Leader Appointment … 2-4 2-5. Review of Appointment … 2-5 2-6. Reinstatement of Evaluators Failing to Meet Appointment Review Criteria 2-6
Chapter 3. Selection and Scheduling of ACSEP Evaluations
3-1. ACSEP Evaluation Intervals … 3-1 3-2. Selection of Facilities to be Evaluated … 3-1 3-3. Scheduling of ACSEP Evaluations … 3-1 3-4. Selection of ACSEP Evaluators … 3-3 Table 3-1. Selecting a PI or AE as an Evaluator … 3-5 3-5. Notification of Facilities to be Evaluated … 3-6 3-6. Modifications to Scheduled Evaluations … 3-7 3-7. Nonscheduled ACSEP Evaluations … 3-7
08/31/2010 8100.7E
iii
Chapter 4. ACSEP Evaluation Procedures
Paragraph Page
Section 1. ACSEP Evaluation Preparations
4-1. Lead Evaluation Office … 4-1 4-2. ACO, MIO, MIDO, and CMO Managers … 4-1 4-3. Evaluation Team Leader or Principal Evaluator … 4-1 4-4. Evaluation Team Member … 4-3 4-5.-4-10. Reserved … 4-3
Section 2. Conduct of the Evaluation
4-11. Team Leader or Principal Evaluator Coordination With Facility Representative … 4-3 4-12. Preevaluation Team Meeting … 4-4 4-13. Preevaluation Conference … 4-4 4-14. Evaluation of System Elements … 4-5 4-15. Recording Noncompliances … 4-7 4-16. Evaluation Meetings … 4-7 4-17. Postevaluation Conference … 4-10 4-18.-4-20. Reserved … 4-10
Section 3. Postevaluation Activities
4-21. Preparing the ACSEP Evaluation Report … 4-11 4-22. Quality Review of the ACSEP Evaluation Report … 4-11 4-23. Sending the ACSEP Evaluation Report … 4-12 4-24. Requesting Corrective Action … 4-12
Chapter 5. ACSEP and CMIS
5-1. Purpose … 5-1 5-2. Files … 5-1 5-3. Database Management … 5-1 5-4. Use of the Database … 5-1
08/31/2010 8100.7E
iv
Appendix A. Preparation of Clauses for Contracts for Support Services
Paragraph Page
1. Purpose … A-1 2. Sample Clauses and Attachment … A-1 Figure 1. Sample Certificate of Nondisclosure … A-2
Appendix B. Preparation of the Notification Letter to a PAH or Associate Facility
1. Purpose … B-1 2. Information to Include in the Notification Letter … B-1 Figure 1. Sample Paragraphs for the Notification Letter … B-2 Figure 2. Sample FAA Form 8100-7, ACSEP Evaluation Customer
Feedback Report … B-3
Appendix C. Notification Letter Requirements
1. Purpose … C-1 2. Description … C-1 Figure 1. Notification Letter Requirements Summary … C-1
Appendix D. Format for Standardized Evaluation Criteria for PAHs and Associate Facilities
Part A Standardized Evaluation Criteria for PAHs and Associate Facilities … D-1 1. Purpose … D-1 Figure 1. System Elements … D-1 2. Description of System Elements Section Format … D-1 Sec. 1 Organizational Management … D-2 Sec. 2 Design Control … D-11 Sec. 3 Software Quality Assurance … D-16 Sec. 4 Manufacturing Processes … D-23 Sec. 5 Manufacturing Controls … D-44 Sec. 6 Supplier Control … D-68 Part B Preparation Instructions for FAA Form 8100-4, ACSEP Survey Sheet for
Production Approval Holders … D-80 Figure 2. Sample FAA Form 8100-4 … D-81
Appendix E. Preparation Instructions for FAA ACSEP Executive Summary
1. Purpose … E-1 2. Specific Guidance … E-1 Figure 1. Sample Executive Summary for PAHs and Associate Facilities ... E-2 Figure 2. Sample Executive Summary for Facilities with No
Noncompliances … E-3
08/31/2010 8100.7E
v
Appendix F. Preparation Instructions for ACSEP Evaluation Special Emphasis Items
Paragraph Page
1. Purpose … F-1 2. Specific Guidance … F-1 Figure 1. Sample ACSEP Evaluation Special Emphasis Items for PAHs
and Associate Facilities …
F-2
Appendix G. Preparation Instructions for ACSEP Evaluation Lessons Learned
1. Purpose … G-1 2. Specific Guidance … G-1 Figure 1. Sample ACSEP Evaluation Lessons Learned … G-2
Appendix H. Preparation Instructions for FAA Form 8100-3, ACSEP Evaluation Report, Cover Pages
1. Purpose … H-1 2. Preparing the Front of the Form … H-1 Figure 1. Sample FAA Form 8100-3 (Front) … H-2 3. Preparing the Back of the Form … H-3 Figure 2. Sample FAA Form 8100-3 (Back) … H-4
Appendix I. Process for Sending ACSEP Evaluation Reports
1. Purpose … I-1 2. Description … I-1 Figure 1. Process for PAHs and Associate Facilities … I-1
Appendix J. Acronyms
1. Applicability … J-1
Appendix K. Definitions
08/31/2010 8100.7E
vi
Appendix L. Administrative Information
1. Distribution … L-1 2. Background … L-1 3. Authority to Change This Order … L-1 4. Forms … L-1 5. Deviations … L-1 6. Suggestions for Improvement … L-1 7. Records Management … L-2 8. Relation to Other Directives … L-2 9. Requests for Information … L-2
Appendix M. FAA Form 1320-19, Directive Feedback Information
08/31/2010 8100.7E
Chapter 1. General
1-1. Purpose of This Order.
a. This order establishes and describes the Federal Aviation Administration (FAA) Aircraft Certification Systems Evaluation Program (ACSEP). This program, an element of certificate management, is a vital element within the FAA’s mission of continued operational safety and is excluded from the Department of Transportation’s plan to reduce internal regulations by 50 percent. FAA Order 8120.2, Production Approval and Certificate Management Procedures, defines the entire certificate management program. However, there are processes in this order that are automated in the Certificate Management Information System (CMIS) program. All of these processes must be performed within CMIS. Other evaluations, audits, or inspections may be required in accordance with directorate or headquarters directives. The ACSEP is a comprehensive evaluation program that accomplishes the following:
(1) Applies standardized systems evaluation to the continued integrity of the design data after initial approval by the FAA at production approval holders (PAH) and associate facilities. The ACSEP does not reevaluate the approval of previously approved data such as quality manuals and design data.
(2) Ascertains whether PAHs and associate facilities meet the applicable requirements of Title 14, Code of Federal Regulations (14 CFR) and comply with procedures established to meet those requirements.
(3) Surveys the application of standardized evaluation criteria not required by 14 CFR or FAA-approved data to identify national trends that may require development of new or revised regulations, policy, and guidance.
(4) Provides customer focus through the establishment of a database for analyzing evaluation results and for reporting trends in continued operational safety upon which FAA customers may act.
(5) Provides continuous improvement for the FAA by continually evaluating lessons learned and customer feedback reports, and considering proposed improvements by FAA internal and external customers.
(6) Provides for employee involvement by establishing and maintaining a professional staff of trained evaluators composed of aviation safety inspectors, aerospace engineers, flight test engineers, and flight test pilots.
b. As some evaluation processes in CMIS are automated, there may be differences in CMIS processes and the stated manual processes defined in this order. Where this is the case, the automated process in CMIS takes precedence over the manual process stated in this order.
1-2. Audience. All FAA employees who participate in ACSEP evaluations conducted at a PAH and its associate facilities as part of the certificate management process.
08/31/2010 8100.7E
1-3. Where Can I Find This Order. You can find this order at http://www.faa.gov/regulations_policies/orders_notices/
1-4. Cancellation. FAA Order 8100.7D, Aircraft Certification Systems Evaluation Program, dated April 16, 2010, and all associated changes are canceled.
1-5. Explanation of Policy Changes. This revision—
a. Removes all references to Approved Production Inspection System (APIS) from this order.
b. Revises appendix D, Format for Standardized Evaluation Criteria for PAHs and Associated Facilities by—
(1) Removing APIS from all applicability tables.
(2) Updating all CFR requirements, in accordance with the new 14 CFR part 21, Certification Procedures for Products, Articles, and Parts (part 21), within the individual evaluation criteria tables.
(3) Updating standardized evaluation criteria by adding, deleting, revising, or combining criteria and/or the statement of conditions, as appropriate.
(4) Adding new evaluation criteria 118 to address a new regulatory requirement regarding corrective and preventive action.
(5) Adding new evaluation criteria 119 to address a new regulatory requirement regarding quality escapes.
(6) Adding new evaluation criteria 211 to address a new regulatory requirement regarding commercial parts.
1-6. Effective Date. This order is effective April 16, 2011.
08/31/2010 8100.7E
Chapter 2. ACSEP Evaluator Appointment and Training
2-1. General. The appointing officials designated in paragraph 2-2 will select ACSEP evaluator candidates who have attained a specified level of experience, or a combination of experience and education, as engineers, flight test pilots, or aviation safety inspectors, and who have demonstrated technical knowledge and skills. A candidate will receive formal classroom ACSEP evaluation training and serve as an evaluator-in-training during ACSEP evaluations under the direct supervision of an appointed ACSEP team leader, before appointment as an ACSEP evaluation team member. Before appointment, a candidate for evaluation team leader will have participated in ACSEP evaluations as an appointed team member and will perform as a team leader-in-training under the direct supervision of an appointed ACSEP team leader.
2-2. Appointing Officials. The following directorate and headquarters managers are authorized to select ACSEP evaluator candidates and to appoint qualified candidates as ACSEP team members or team leaders within their respective organizations:
a. ACO managers and ACO branch managers.
b. Manufacturing inspection office (MIO), MIDO, and CMO managers.
c. Directorate Standards Staff managers.
d. AIR-100 Branch managers.
e. AIR-200 Branch managers.
2-3. Criteria for Candidate Selection. The appointing official will select engineering, flight test or aviation safety inspector candidates on the basis of the following criteria (see figure 2-1):
a. Candidates have attained at least one of the following specified levels of experience or a combination of experience and education in their specific disciplines:
(1) At least 8 years of technical experience in aerospace manufacturing or design, or in the evaluation thereof.
(2) Technical or trade school certificate with 6 years of technical experience in aerospace manufacturing or design, or in the evaluation thereof.
(3) Associate’s degree in engineering or science disciplines, with 5 years of technical experience in aerospace manufacturing or design, or in the evaluation thereof.
(4) Bachelor’s degree or higher in engineering or science disciplines, with 3 years of technical experience in aerospace manufacturing or design, or in the evaluation thereof.
08/31/2010 8100.7E
Figure 2-1. Criteria for Candidate Selection
and Team Member Appointment
8 years technical experience in aerospace
mfg. or design, or
evaluation?
Tech or trade school certificate +6 yrs tech experience in aerospace mfg. or design,
or evaluation?
No
Assoc degree in Engineering or Science +5 yrs tech experience in
aerospace mfg. or design,
or evaluation?
No
Bachelor's degree in Engineering or Science
+3 yrs tech experience in aerospace mfg. or design,
or evaluation?
No
Satisfactory completion of ACSEP
training course?
CANDIDATE NOT
ELIGIBLE FOR APPOINTMENT
No
Yes Yes Yes Yes
Participation in at least 2 ACSEP evaluations as an evaluator-in-training?
Yes
Yes
CANDIDATE ELIGIBLE FOR APPOINTMENT AS ACSEP TEAM MEMBER
No
No
b. Candidates have demonstrated technical knowledge in aerospace manufacturing or design, conceptual understanding of FAA goals and objectives, effective communication and interpersonal skills, good human relations, and ability to write coherently.
2-4. Criteria for Appointment. Appointment is the formal process of certifying an ACSEP candidate as an ACSEP team member or team leader on the basis of successful completion of all requirements (see figures 2-1 and 2-2).
a. Team Member. Candidates must meet the following minimum requirements before appointment as a team member (see figure 2-1):
(1) Satisfactory completion of the ACSEP team training course and associated written examination. The course will provide training in the policy established in this order, including the techniques for applying the standardized evaluation criteria contained in appendix D, and in coordinating team member involvement.
Note: The Planning and Program Management Division, AIR-500, will ensure classes are scheduled in accordance with Aircraft Certification Service (AIR) priorities as identified in the annual call for training.
08/31/2010 8100.7E
(2) Participation of the candidate, and demonstration of the knowledge and skills acquired during ACSEP team training in at least two ACSEP evaluations as an evaluator-in-training.
Note: The candidate’s appointing official must schedule the candidate’s participation as an evaluator-in-training to be completed in as short a timeframe as possible to maximize the candidate’s use and retention of acquired knowledge and experience.
(3) The candidate’s appointing official is responsible for performing the following activities in evaluating the team member candidate:
(a) Consider the candidate’s previous experience and education.
(b) Consider the product complexity, facility size, and complexity of system elements evaluated in ACSEP evaluations in which the candidate participated.
(c) Discuss with team leader(s), evaluations in which the candidate participated to determine the candidate’s ACSEP evaluation readiness.
(d) Review ACSEP evaluation reports for evaluations in which the candidate participated.
(e) Review, when necessary, FAA Form(s) 8100-7, ACSEP Evaluation Customer Feedback Report, for evaluations in which the candidate participated.
(f) Interview the candidate.
(g) Discuss with the candidate any weaknesses or deficiencies in their evaluation readiness identified during the participation phase. Both parties will work to reduce or eliminate these weaknesses or deficiencies through additional training, additional ACSEP evaluations, Air Transportation Oversight System/Air Carrier Evaluation Program audits, or other similar activities that will increase the candidate’s evaluation readiness.
(4) On the basis of satisfactory results of the evaluation of the candidate as listed in paragraph 2-4a(3), the candidate’s appointing official will appoint the candidate as a team member and add the individual to the evaluator’s module of the CMIS program.
b. Team Leader. Candidates must meet the following minimum requirements before appointment as a team leader (see figure 2-2):
(1) Current appointment as an ACSEP evaluation team member.
(2) Participation in at least three evaluations as an appointed ACSEP evaluation team member. The candidate’s appointing official may request reduction of the requirement by providing documented justification to the appointing official’s manager. The responsibility for requesting any reduction of the requirement rests solely with the candidate’s appointing official.
08/31/2010 8100.7E
Figure 2-2. Criteria for Team Leader Appointment
Appointment as
ACSEP Team
Member?
Participated in at least 3 evaluations as a
team member?
Yes
Written justification from appointing official
for reduction of
requirement?
No
CANDIDATE NOT
ELIGIBLE FOR APPOINTMENT
No
Participated in at least 3 evaluations
as a team leader-in-
training?
Yes Manager approval?
Yes
Yes
No
Written justification from appointing official
for reduction of
requirement?
Manager approval?
Yes
No No
No
CANDIDATE ELIGIBLE FOR APPOINTMENT AS
ACSEP TEAM LEADER
Yes
Yes
No
(3) Participation as a team leader-in-training, and demonstration of knowledge and skills acquired during ACSEP team training in at least three ACSEP evaluations under the direct supervision of an appointed ACSEP evaluation team leader. The candidate’s appointing official may request reduction of the requirement by providing documented justification to the appointing official’s supervisor. The responsibility for requesting any reduction of the requirement rests solely with the candidate’s appointing official.
Note: The candidate’s appointing official must schedule the candidate’s participation as a team leader-in-training to be completed in as short a timeframe as possible to maximize the candidate’s use and retention of acquired knowledge and experience.
08/31/2010 8100.7E
(4) The candidate’s appointing official is responsible for performing the following activities in evaluating the team leader candidate:
(a) Consider the candidate’s previous experience and education.
(b) Consider the product complexity, facility size, and complexity of system elements evaluated in ACSEP evaluations in which the candidate participated.
(c) Discuss with team leader(s), evaluations in which the candidate participated to determine the candidate’s team leadership abilities.
(d) Review ACSEP evaluation reports for evaluations in which the candidate participated.
(e) Review, when necessary, Form(s) 8100-7 for evaluations in which the candidate participated.
(f) Interview the candidate.
(g) Discuss with the candidate any weaknesses or deficiencies in their team leadership abilities identified during the participation phase. Both parties will work to reduce or eliminate these weaknesses or deficiencies through additional training, additional ACSEP evaluations, Air Transportation Oversight System/Air Carrier Evaluation Program audits, or other similar activities that will increase the candidate’s leadership abilities.
(5) On the basis of satisfactory results of the evaluation of the candidate as listed in paragraph 2-4b(4), the candidate’s appointing official will appoint the candidate as a team leader and update the evaluator’s module of the CMIS program.
c. The candidate’s appointing official will document and track the completion of the requirements in paragraphs 2-4a and 2-4b for all ACSEP candidates. Upon successful completion of the requirements, the appointing official will appoint the candidate as an ACSEP evaluation team leader or team member and will formally notify the candidate of his or her appointment in writing. Ensure the appointment document includes the individual’s discipline and office identification.
Note: Provide written notification of appointment before the evaluator’s first scheduled ACSEP evaluation as a team member or team leader.
2-5. Review of Appointment. The cognizant appointing official (1) reviews the participation in ACSEP evaluations by each evaluator under their appointment authority, (2) notifies evaluators in writing, of decisions not to continue their appointment, and (3) determines the currency and continued validity of appointments as follows.
08/31/2010 8100.7E
a. Evaluation Team Members. Review evaluation team members’ participation annually. Ensure team members have accomplished the following requirements, as a minimum:
(1) Participated at an interval of once or more every 2 fiscal years as an ACSEP evaluation team member or once or more every 2 fiscal years as an ACSEP evaluation team leader, or conducted PI evaluations or MIDO audits in accordance with Order 8120.2.
Note: A supplier control audit does not count toward the continued appointment of an ACSEP team member.
(2) Demonstrated knowledge and skill in ACSEP evaluations, as determined from sources such as the ACSEP evaluation report, team leaders, cognizant managers, and satisfactory corrective action for any shortcomings in knowledge or skills noted and discussed with the team member during the interim period.
b. Evaluation Team Leaders. Review evaluation team leaders’ participation annually. Ensure team leaders have accomplished, at a minimum the following requirements:
(1) Participated at an interval of once or more every 2 fiscal years as an ACSEP evaluation team leader or as a team leader for a PI evaluation or MIDO audit with multiple team members in accordance with Order 8120.2.
Note: A supplier control audit does not count toward the continued appointment of an ACSEP team leader.
(2) Demonstrated knowledge and skill in ACSEP evaluations, as determined from sources such as the ACSEP evaluation report, cognizant managers, and satisfactory corrective action for any shortcomings in knowledge or skills noted and discussed with the team leader during the interim period.
2-6. Reinstatement of Evaluators Failing to Meet Appointment Review Criteria. Cognizant appointing officials may reinstate evaluators under their appointment authority who have not met the appointment review criteria listed in paragraph 2-5. Use the following criteria to determine eligibility for reinstatement:
a. Team members and leaders who have not met participation requirements may be reinstated after acceptable participation as an evaluator-in-training, or as a team leader-in-training as applicable, in two ACSEP evaluations.
b. Team members who have not demonstrated ACSEP evaluation knowledge or skills may be considered for reinstatement by repeating the formal ACSEP team member appointment program listed in paragraph 2-4a.
c. Team leaders who have not demonstrated ACSEP evaluation knowledge or skills may be reinstated as a team member after acceptable participation as an evaluator-in-training in two ACSEP evaluations. Consideration for reinstatement as a team leader must then follow the formal ACSEP team leader appointment program listed in paragraph 2-4b.
08/31/2010 8100.7E
Chapter 3. Selection and Scheduling of ACSEP Evaluations
3-1. ACSEP Evaluation Intervals. Evaluation intervals for PAHs and associate facilities are identified in Order 8120.2.
3-2. Selection of Facilities to be Evaluated. Procedures for selecting PAHs and associate facilities to be evaluated are identified in Order 8120.2.
3-3. Scheduling of ACSEP Evaluations. After all facilities have been selected for evaluation in accordance with paragraph 3-2, each directorate will be responsible for scheduling ACSEP evaluations at the selected facilities. Use the following procedures:
a. Estimate the onsite duration of each evaluation according to the evaluation interval listed in Order 8120.2. Consider the quality and/or engineering procedures and processes required to be in place, the number of applicable system elements, when known (see appendix D), the size and physical layout of the facility to be evaluated (single or multiple locations), and product complexity. Allow enough time to ensure that compliance to the applicable 14 CFR and FAA-approved data will be fully evaluated. Use the following list as a guide for estimating, in terms of facility size only, the onsite duration of the evaluation (excluding travel times):
(1) Small facility with fewer than 100 total full-time persons: 1 to 5 working days onsite.
(2) Medium facility with 100 to fewer than 400 total full-time persons: 3 to 5 working days onsite.
(3) Large facility with 400 to fewer than 2,000 total full-time persons: 5 to 10 working days onsite.
(4) Very large facility with 2,000 or more total full-time persons: 7 to 15 working days onsite.
Note: When estimating the onsite duration, include only those persons who are used to support the PAH activity.
b. Assign all scheduled evaluations a distinct ACSEP number, consisting of the fiscal year, directorate code (NE—Engine and Propeller Directorate, CE—Small Airplane Directorate, SW—Rotorcraft Directorate, or NM—Transport Airplane Directorate), and the evaluation order sequence. For example, 09CE123 represents the 123rd evaluation planned for completion by the Small Airplane Directorate during fiscal year 2009.
Note: Do not reassign ACSEP numbers from canceled evaluations. Each scheduled evaluation must be uniquely identified.
08/31/2010 8100.7E
c. Identify the lead evaluation office for each evaluation. This office is usually the one that performs certificate management responsibility at the facility to be evaluated. For an associate facility subject to certificate management under the handoff procedure described in Order 8120.2, the lead evaluation office is the geographic office receiving the handoff. The lead evaluation office is responsible for—
(1) Coordinating the notification letter (see paragraph 3-5), and
(2) Notifying the selected team leader and team members (see paragraph 4-1).
d. Prepare an evaluation schedule for the current fiscal year based on the facility selection criteria in paragraph 3-2 and the duration of each evaluation. Annually prepare the schedule no later than July 31.
(1) Prepare the schedule in quarterly increments using the following guidelines:
(a) ACSEP number.
(b) Scheduled start date of each evaluation.
(c) Duration of each evaluation.
(d) Facilities and types of approvals to be evaluated.
(e) Risk-Based Resource Targeting (RBRT) risk level.
(f) Product lines or authorized functions at the facilities to be evaluated.
(g) Number and disciplines of evaluators assigned to each evaluation.
(h) Additional evaluators required beyond the directorate’s resources.
(i) Number and disciplines of evaluators-in-training and team leaders-in-training.
(j) Total number of evaluations scheduled for the fiscal year.
(k) Applicable project number(s).
(2) All directorate schedules will be entered into the schedule module of the CMIS program.
(3) The ACO, MIO, MIDO, and CMO managers should schedule approval holders with multiple approvals, such as a PC and a PMA, so as to evaluate all approvals during one evaluation.
(4) When an approval holder has multiple facilities that require significant resources and time to evaluate, the ACO, MIO, MIDO, and CMO managers should consider scheduling the facilities individually.
08/31/2010 8100.7E
e. Designate an assigned engineer (AE). On the basis of the data collected for paragraphs 3-1 through 3-3d, the ACO manager determines the need to assign an FAA engineer responsibility relating to a scheduled ACSEP evaluation at a particular design approval facility. The AE must answer questions from the evaluators regarding the FAA-approved design or the design approval system in place. The AE also must coordinate any corrective action required regarding the FAA-approved design or the design approval system.
3-4. Selection of ACSEP Evaluators. The ACO, MIO, MIDO, and CMO managers select appointed ACSEP evaluators to perform each scheduled evaluation. Determine the number and types of evaluators required for each evaluation according to the following criteria:
a. Number of Evaluators Required. Determine the total number of evaluators required to ensure that compliance to the applicable 14 CFR and FAA-approved data would be fully evaluated.
(1) Estimate the number of evaluators required according to the following minimum criteria:
(a) RBRT assigned risk level.
(b) Number and complexity of applicable quality, engineering, flight test, and facility procedures and processes in place.
(c) Number of applicable system elements, when known (see appendix D).
(d) Number of suppliers to which the evaluation will be extended, when known.
(e) Size and physical layout of the facility to be evaluated (single or multiple locations).
(f) Product or design approval system complexity.
(2) Use the following as a guide for estimating the number of ACSEP evaluators required. Increase or decrease the number of estimated evaluators shown below, depending on your review of the criteria contained in paragraph 3-4a(1) and your confidence that compliance to the applicable 14 CFR and FAA-approved data will be fully evaluated:
(a) Small facility with fewer than 100 full-time persons: 1 to 3 evaluators (including team leader).
(b) Medium facility with 100 to fewer than 400 total full-time persons: 1 to 5 evaluators (including team leader).
(c) Large facility with 400 to fewer than 2,000 total full-time persons: team leader plus 5 to 10 evaluators.
(d) Very large facility with 2,000 or more total full-time persons: team leader plus up to 10 evaluators.
08/31/2010 8100.7E
Note: When estimating the number of evaluators required, include only those full-time persons who are used to support the PAH facility activity.
(3) If it is determined that one evaluator is required, select an appointed team leader to perform the evaluation; this evaluator is referred to as the principal evaluator. If two or more evaluators are selected for an evaluation, they will constitute an ACSEP evaluation team. Select an appointed team leader and the required number of appointed team members.
b. Types of Evaluators Required. Use the criteria identified in paragraphs 3-4a(1)(a) through (f) and the following criteria to determine the types of evaluators required. Select appointed ACSEP evaluators who have appropriate knowledge of the evaluation criteria identified in appendix D applicable to the facility to be evaluated and, as appropriate, to the product(s) authorized by the approval (for example, select a propulsion engineer when an engine manufacturer is to be evaluated and select a flight test pilot when a flight test program is to be evaluated). When making this determination, consider the following:
(1) It is not necessary to select both engineers and inspectors for a small facility that does not have both engineering and manufacturing capabilities.
(2) Select appointed ACSEP evaluators, as appropriate, to maintain continued appointment in accordance with paragraph 2-5.
(3) Do not include any appointed evaluators who were previously employed by the facility to be evaluated within 2 years of the scheduled evaluation.
(4) Determine whether evaluators will be made available throughout the duration of the evaluation. Each evaluator is expected to fully participate in each evaluation. Base any decision to limit participation on the established AIR priorities. Notify the team leader of any limited participation by evaluators.
c. Selection of PI and AE as Team Leaders or Evaluators. To the greatest extent practicable, the PI and the AE will not be selected as team leaders on ACSEP evaluations of facilities for which they have certificate management or surveillance responsibilities. Use the guidelines in table 3-1 to select the PI and/or AE as evaluators:
08/31/2010 8100.7E
Table 3-1. Selecting a PI or AE as an Evaluator
Number of Persons Performing the Evaluation PAH Facility Procedure
One- or two-person Do not select the responsible certificate management PI. Do not select the AE if the AE is the engineer assigned design responsibility for the facility to be evaluated.
Note: For evaluations with at least three team members, the ACO, MIO, MIDO, and CMO managers, to the greatest extent practicable, will select as evaluators the PI, or assistant PI as appropriate, and/or the AE. The ACO, MIO, MIDO, and CMO managers should assess the logistical and personal burden of selecting the PI and/or AE for all applicable evaluations, and should assign the PI and/or AE to evaluations through which the greatest benefit may be obtained. Three- or four-person Select as a team member either the responsible certificate management PI or the AE, if the AE is the engineer assigned design responsibility for the facility to be evaluated. If the AE is not assigned design responsibility, both the AE and the responsible certificate management PI may be selected as team members. Five-person or greater Select as a team member either the responsible certificate management PI or AE, or both.
d. Selection of Evaluators-in-Training and Team Leaders-in-Training.
(1) Determine the number of appointed evaluators required for the ACSEP evaluation before assigning evaluators-in-training. Assign evaluators-in-training only to evaluations for which a team is required. Do not assign evaluators-in-training to a principal evaluator. Evaluators-in-training will supplement appointed evaluators. Do not substitute evaluators-in-training for appointed ACSEP evaluators, or evaluation team leaders-in-training for appointed ACSEP evaluation team leaders.
(2) Do not assign more than two evaluators-in-training or more than one team leader-in-training to any one evaluation. Try to assign each evaluator-in-training or team leader-in-training to different team leaders during the participation phase of the training.
(3) In cases where evaluators-in-training or team leaders-in-training from other directorates or AIR-100/200 are proposed to be used in an evaluation, coordinate with the appointing managers to establish their eligibility.
08/31/2010 8100.7E
e. Additional Resource Requirements. Additional evaluators beyond the directorate’s available resources may be required depending on the size of the facility; type and complexity of product, service, or design approval system; and overall evaluation objectives. Each directorate should identify the need for these additional resources before the release of the ACSEP master schedule for the next fiscal year and coordinate the participation of the evaluators with the appropriate directorate office and CMIS coordinators. Additional support may also be available from the AIR-100 or AIR-200 divisions, if requested. If these sources of support are not available, the directorate may obtain outside support services to augment directorate resources. Support service personnel will be qualified and credible quality assurance experts and technology specialists and will meet the criteria for candidate selection specified in paragraph 2-3. Directorates will obtain any required support service personnel in accordance with budgetary directives. Appendix A contains sample contract clauses relating to obtaining support services.
Note: The cognizant directorate will complete all necessary administrative measures required for facility access by support service personnel before the scheduled ACSEP evaluation. The measures may include obtaining any security clearances from the prospective facility, ensuring that personnel have signed a certificate of nondisclosure for confidentiality of information (see appendix A), and ensuring that personnel are aware of their limitations (as agreed to between the directorate and the facility to be evaluated) of access and entry to the facility’s proprietary or sensitive processes or systems.
f. Scheduled Changes. Each directorate must update schedule changes electronically in the CMIS program at least quarterly. Evaluations added to the master schedule will be assigned a new ACSEP number in accordance with paragraph 3-3b.
3-5. Notification of Facilities to be Evaluated. The lead evaluation office identified in accordance with paragraph 3-3c will notify facilities using the sample formats in appendixes B and C. Coordinate with the responsible PI to ensure that the letter does not arrive during scheduled shutdown periods or during any other extended periods when the letter may not be acted upon. For notifications of first-time ACSEP evaluations, inform the facility that ACSEP reference material is available on the FAA’s Web site. If the facility cannot access the Web site, provide the reference material to the facility. Appendix C provides a summary of notification letter requirements. Notify facilities as follows:
a. PAH/Associate Facility. The lead evaluation office will perform these tasks:
(1) Prepare the notification letter and send it to the facility to be evaluated no later than 50 calendar days before the evaluation.
(2) Provide a copy of the notification letter to the designated evaluation team leader or principal evaluator, the PI, and the AE.
08/31/2010 8100.7E
b. Changes After Notification Letter Is Sent. As appropriate, notify the facility, responsible PAH or associate facility, requesting MIDO or CMO, AIR-200, and the team leader or principal evaluator of any changes to the evaluation schedule or team composition after the notification letter has been sent.
3-6. Modifications to Scheduled Evaluations. Every effort will be made to maintain established evaluation schedules. However, modifications to the evaluation schedule should be considered under special circumstances. The ACO, MIO, MIDO, and CMO managers will jointly reschedule any affected evaluation in coordination with the PI, AE, and the team leader or principal evaluator, and update the schedule in the CMIS program. Special circumstances that may warrant modifications to the evaluation schedule include—
a. Risk to evaluators’ safety,
b. Change in a facility’s production or delegation status from active to inactive,
c. Involvement of the FAA in a facility’s labor-management dispute,
d. Reduction in the effectiveness of the evaluation, and
e. A nonscheduled ACSEP evaluation that requires scheduled resources (see paragraph 3-7).
3-7. Nonscheduled ACSEP Evaluations. The ACO, MIO, MIDO, and CMO managers may also conduct nonscheduled ACSEP evaluations when situations warrant, as determined by directorate offices or AIR-200 — Production and Airworthiness Division. Nonscheduled ACSEP evaluations will be planned, conducted, and reported in accordance with this order to the greatest extent practicable. Appropriate emphasis on planning the evaluation should be provided despite the reduced time that may be available between the decision to conduct the nonscheduled ACSEP evaluation and the actual conduct of the evaluation. Situations that may warrant a nonscheduled ACSEP evaluation include the following:
a. Accidents and incidents,
b. Deliberate violations,
c. Repetitive service difficulty reports,
d. Excessive owner/operator complaints,
e. PAH’s or associate facility’s refusal/failure to take appropriate corrective action,
f. PAH’s or associate facility’s inability to control suppliers,
g. Renewal of a PAH’s or associate facility’s production activity after a prolonged period of inactivity, and
h. Any other situation as deemed necessary in the interest of safety.
08/31/2010 8100.7E
Intentionally Left Blank
08/31/2010 8100.7E
Chapter 4. ACSEP Evaluation Procedures
Section 1. ACSEP Evaluation Preparations
4-1. Lead Evaluation Office. Perform, at a minimum, the following evaluation preparations:
a. Notify, through CMIS, the selected evaluation team leader and team members, or the principal evaluator, at least 60 calendar days before each directorate evaluation. A record of the notification does not need to be retained.
b. Ensure logistical support for an evaluation within the geographical area.
4-2. ACO, MIO, MIDO, and CMO Managers. Notify, through CMIS, all evaluators within the directorate selected for AIR-200-led evaluations and evaluations in support of other directorates. Send notification at least 60 calendar days before each evaluation. Send a copy of the notification to the lead evaluation office and AIR-200. A record of the notification does not need to be retained.
4-3. Evaluation Team Leader or Principal Evaluator. Coordinate evaluation preparation. The team leader provides orientation to team members, and assigns system elements to team members. These actions, as appropriate, require coordination with the PI, AE, and the facility to be evaluated. The team leader or principal evaluator will perform the following, as appropriate:
a. Upon receipt of a copy of the notification letter, contact the lead evaluation office to identify the responsible PI and AE and obtain from the PI and AE such items as the following:
(1) Applicable FAA-approved procedures, including engineering and quality manuals, procedures manuals, and handbooks, when practical. Obtain documentation in electronic format, if available, to simplify copying and distribution to team members. If applicable data are available only electronically, work with the PI or AE to identify relevant documents and to obtain printed copies of only those pages necessary to support the ACSEP evaluation.
(2) Current facility data available in the CMIS.
(3) Known or suspected problem areas, including any areas the PI and AE would like special emphasis on during the evaluation, such as requests to conduct a product audit in accordance with Order 8120.2.
(4) Current self-disclosure items reported under FAA Order 2150.3, Compliance and Enforcement Program.
(5) Agreements made between the cognizant ACO, MIO, MIDO, or CMO and the facility to be evaluated.
(6) Facility access information, including badges and security clearances.
(7) Lodging information.
08/31/2010 8100.7E
(8) Any other items necessary to prepare for the evaluation.
b. Prepare a written evaluation plan, using the form found in the CMIS program, for conducting the evaluation. The evaluation plan includes the following items:
(1) Name and address of the facility to be evaluated.
(2) Dates of the evaluation.
(3) Names of the team leader and members (when more than one evaluator is selected).
(4) Evaluation objectives. List the reason for the ACSEP evaluation and what information is expected to be obtained during the evaluation (for example, establish facility compliance with the procedures established to meet the applicable requirements of 14 CFR or establish cause of repetitive service difficulty reports).
(5) Type(s) of approval.
(6) Type certificate (TC) or supplemental type certificate (STC) number, as applicable.
(7) Current product line.
(8) Number of employees associated directly with the production approval activity.
(9) List of top-level FAA-approved procedures (for example, quality manual index of procedures, procedures manual, PMA approval letter, and TC data sheets).
(10) FAA/facility agreements in effect; for example, agreement on frequency of submittal of minor design changes.
(11) Plant layout.
(12) Organizational chart.
(13) Major processes.
(14) Unusual features of the product, manufacturing and inspection methods, or design approval system.
(15) Self-disclosure items under Order 2150.3, Compliance and Enforcement Program.
(16) Special emphasis items recommended by the PI and AE.
(17) System element, to include product audit, assignments (when more than one evaluator is selected).
(18) Access information, including facility point of contact.
(19) Lodging information.
08/31/2010 8100.7E
(20) Equipment required (for example, notebook computer, safety shoes, and coveralls).
c. Coordinate assignments, requirements, and arrangements with team members as far in advance of the evaluation as possible, but no later than 30 calendar days before the evaluation. Notify team members immediately of changes in schedule, assignments, requirements, and arrangements. Provide copies of all relevant facility documents to team members, when feasible.
d. Forward an FAA certificate of nondisclosure (see appendix A) to any outside support service personnel assigned no later than 35 calendar days before the evaluation. Obtain signed statements no later than 25 calendar days before the evaluation and forward them to the facility via the PI.
e. Notify the lead evaluation office immediately of changes in team numbers or composition.
f. Coordinate with the certificate management PI or AE, or geographic PI, as appropriate, to resolve specific planning problems relating to the facility to be evaluated.
g. Arrange, as appropriate, for the availability of a notebook computer and portable printer for the duration of the evaluation, and for the accomplishment of postevaluation activities. Use of a notebook computer during the evaluation will allow for quick access and search of ACSEP documentation and for preparation of high-quality documents for presentation during the postevaluation conference.
4-4. Evaluation Team Member. Perform these tasks:
a. Upon notification by the team leader, confirm availability for the evaluation, system elements assigned, and travel arrangements.
Note: Notify the team leader immediately if you become unavailable for the evaluation.
b. Before the evaluation, review all material provided by the team leader, the PI, or the AE appropriate to the assigned system elements. When possible, make a preliminary selection of the procedures you plan to evaluate.
4-5. through 4-10. Reserved.
Section 2. Conduct of the Evaluation
4-11. Team Leader or Principal Evaluator Coordination with Facility Representative. The team leader or principal evaluator will coordinate with the designated representative of the facility to be evaluated to ensure that administrative arrangements for items such as team access, escorts, meeting rooms, and safety and security requirements are complete.
08/31/2010 8100.7E
4-12. Preevaluation Team Meeting. The team leader and all team members meet in advance of starting the evaluation, usually at the facility to be evaluated. They review the following evaluation elements, as appropriate, for proper coordination and understanding:
a. Current quality system or design approval system, and corrective action history of the facility to be evaluated in the selected areas.
b. Team functional assignments.
c. Evaluation plan.
d. Evaluation objectives.
e. Working relationship of the facility to be evaluated with the FAA.
f. Organizational structure of the facility to be evaluated.
g. Approved quality system documents, including quality manuals and/or quality data submitted by PAHs to describe their quality systems.
h. Approved design approval system documents, including any procedures manual or handbook.
i. Agreements made between the cognizant ACO, MIO, MIDO, or CMO and the facility to be evaluated.
4-13. Preevaluation Conference. Soon after arrival at the facility to be evaluated, the evaluation team leader or principal evaluator conducts a preevaluation conference with appropriate senior management, cognizant supervisory personnel, and other appropriate personnel of the facility who will be associated with the evaluation, including escorts. The team leader or principal evaluator must perform the following tasks, as appropriate:
a. Introduce team members and support service personnel.
b. Give a brief overview of ACSEP, highlighting the cooperative intention of the evaluation.
c. Provide the evaluation’s scope and objectives.
d. Review details of the evaluation agenda, including the standardized evaluation criteria and procedures to be used.
e. Review administrative arrangements for the postevaluation conference.
f. Discuss Form 8100-7 sent with the notification letter to the facility being evaluated. Explain that this form is designed to obtain senior management assessment of the conduct of the ACSEP evaluation and is used by the FAA for continuous quality improvement of the certificate management program. Encourage senior management to complete the form and send it to the address on the form within 30 calendar days of the postevaluation conference.
08/31/2010 8100.7E
g. Allow time for a question-and-answer session.
4-14 Evaluation of System Elements. The ACSEP evaluation team evaluates up to six system elements and conducts at least one product audit at PAHs and associate facilities. Each system element addresses a specific activity or function that may affect the maintenance of FAA-approved design or quality data. Each system element is defined in appendix D. The ACSEP evaluation team will perform the following tasks, as appropriate:
a. Review FAA-approved quality systems manuals or procedures manuals/handbooks to determine if current data ensure that regulatory requirements are met, if conforming products and parts are manufactured, and if design approval systems are maintained and controlled.
b. Review design system, design approval system, and quality system data to determine if current data are FAA-approved.
c. Review other facility procedures (related to the production approval facility) that are not part of the facility’s FAA-approved data to determine if the current procedures impact any of the system elements.
d. Review PAH supplier records by selecting a random sample of PAH supplier audit reports. (Refer to appendix D, section 6, paragraph 1a.)
(1) The reports may consist of onsite evaluations, mail-in surveys, third-party evaluations, or a combination of all three. The reports must be reviewed for compliance with the PAHs’ quality system requirements. This may include, but is not limited to, the following conditions:
(a) Adherence to scheduled frequency of supplier control audits.
(b) Appropriate documentation of audits. This includes a signature by an appropriate authority, and attachment of required certifications and test documents.
(c) Determination of whether noncompliances provide evidence of root cause, corrective action, followup, and closure.
(d) If a history of similar noncompliances is evident, determination of whether the PAH is appropriately conducting root cause analysis and applying corrective action.
(2) FAA Form 8100-1, Conformity Inspection Record will be used to record the following information. The completed record will be entered in CMIS as part of the ACSEP report.
(a) Total number of audit reports reviewed.
(b) Identification of suppliers reviewed.
(c) Total number of noncompliances documented for all supplier reports reviewed.
08/24/2011 8100.7E,CHG1
(3) The component page of the ACSEP report entitled Special Emphasis Items may be used to record any additional or supplemental information pertaining to the supplier audit record review that the evaluator considers important. Include this information as a note under the heading, "Note to MIO Manager and Cognizant Principal Inspector."
Note: The results will be used for two purposes: (1) to identify areas that may require more focused attention during evaluation of the supplier control system element and (2) as input into the following year's RBRT assessment of the PAH.
(4)Any noncompliance noted during the review of PAH supplier audit reports will be recorded under supplier control system element criteria number 602. Noncompliance will also be documented in accordance with paragraph 4-15 of the order.
Note: Paragraph 4-14d, and appendix D, section 6, paragraph 1a, apply only to PAH facilities that use suppliers in the process of manufacturing FAA-approved products. Review of supplier records should be started early in the evaluation process to allow for additional time in case issues are noted.
e. Evaluate compliance to facility procedures and quality requirements. Prioritize evaluation according to any special concerns raised by the PI or AE. Use the standardized evaluation criteria in appendix D to determine the depth of the evaluation in the subject area. Evaluate, as necessary, a combination of document and product review to determine if the system element meets applicable requirements.
Note: The standardized evaluation criteria are a list of questions and related statements of condition in appendix D used primarily to plan and document the results of the evaluation of each system element in a standardized manner. The criteria are designed to cross all the functional areas within a facility's organization that have the greatest potential to impact the integrity of the FAA-approved design and product quality. All responses to the questions are direct inputs to the database from which trend analysis is accomplished. Each evaluator should be knowledgeable of all the criteria applicable to the system element assigned to be evaluated and should strive to evaluate as many of the procedures, requirements, and products related to the criteria as time allows.
f. Select at least one team member to conduct at least one product audit at a PAH or associate facility of a manufactured product (for example, characteristic dimensioning, processing attributes, and physical examination) to determine compliance with current system procedures and quality requirements. Refer to Order 8120.2 for product audit areas, criteria, andprocedures for recording audit results.
08/24/2011 8100.7E, CHG 1
g. On the basis of facility procedures or quality requirements, identify, and document additional standardized evaluation criteria questions and statement-of-condition practices and principles not contained in appendix D that were required to document what was evaluated. Write or type additional criteria and statement-of-condition practices and principles, and include the appropriate reference to the facility procedures or quality requirements and the evaluator's recommendation of the system element to which the criteria and statement of condition apply. Team members must present new criteria and statement-of-condition practices and principles to the team leader as soon as they are completed.
h.Detect and report noncompliances and areas that may require additional evaluation by the PI or AE.
4-15. Recording Noncompliances. Evaluators will record all noncompliances on FAA Form 8100-6, Noncompliance Record, or electronic equivalent, according to the guidelines in Order 8120.2.
Note: Record as a certification-related noncompliance any condition that questions the certification basis. Address the noncompliance on the Executive Summary (refer to paragraphs 4-16b(2)(c) and 4-21b, and appendix E) and as a special emphasis item in the evaluation report (refer to paragraphs 4-16b(2)(d) and 4-21c, and appendix F).
4-16. Evaluation Meetings.
a. Daily Meeting. The team leader or principal evaluator holds the following daily meetings, as appropriate:
(1)Meeting with Evaluation Team Members.The team leader will review and discuss the following with team members:
(a)Status of the evaluation.
(b)Problems encountered.
(c)Plan of the next day's evaluation.
(d) All Form(s) 8100-6, or electronic equivalent, prepared during the day to ensure correctness, adequacy, and completeness.
(2)Meeting/Communication with PI and AE. The team leader or principal evaluator ensures that the certificate management PI and AE, and the geographic PI, as applicable, are informed of all discussions concerning the status of the evaluation. This meeting should occur daily when the PI and AE are part of the evaluation team. Otherwise, coordinate with the PI and AE to establish the method and frequency at which these discussions should occur.
08/24/2011 8100.7E, CHG 1
(3) Meeting with the Evaluated Facility's Designated Representative. The team leader or principal evaluator holds a brief meeting daily with the evaluated facility's designated representative to discuss the progress of the evaluation, including problems encountered, the status of actions requested by the team, schedule changes, and the coordination of further evaluation activities.
b. Final Critique Meeting/Evaluation Wrap-Up. At the conclusion of the evaluation, the team leader holds a final critique meeting. The principal evaluator allows time to finalize the details of the evaluation. The team leader and members or the principal evaluator do the following, as appropriate:
(1) Team Members or Principal Evaluator.
(a)Complete all required Form(s) 8100-6, or electronic equivalent. When using an electronic equivalent, print to paper when all information has been entered. Team members discuss Form(s) 8100-6 with the team leader to determine if there are any possible violations of the applicable requirements of 14 CFR. The team leader must resolve any disagreement on noncompliance(s). The lead evaluation office, or requesting MIDO or CMO, as applicable, must determine the level of corrective action required (see paragraph 4-24).
(b) Ensure that all true copies of objective evidence are attached to the appropriate Form(s) 8100-6, or electronic equivalent, appropriately referenced, and clearly identified in accordance with Order 2150.3.
(c)Complete FAA Form 8100-4, ACSEP Survey Sheet for Production Approval Holders, or electronic equivalent, in accordance with appendix D (part B). When using an electronic equivalent, print to paper when all information has been entered. Prepare original forms as follows:
1 PAH or Associate Facility. Prepare one original Form 8100-4.
2Facility with Multiple Production Approvals. Prepare one original Form 8100-4. Base the survey responses on the criteria for the highest-level quality requirement; for the purposes of ACSEP, the quality levels, from highest to lowest, are PC, TSO authorization and PMA. For example, if a facility has a PMA and a TSO authorization, prepare one Form 8100-4 based on the TSO authorization criteria.
(2) Team Leader or Principal Evaluator.
(a) Resolve team disagreements on specific noncompliances.
(b)Discuss all noncompliances with the certificate management PI or AE, delegated facility AE, and geographic PI, as applicable.
08/31/2010 8100.7E
(c) Prepare the ACSEP Evaluation Executive Summary (see appendix E). Prepare original forms as follows:
1 PAH or Associate Facility. Prepare one original summary.
2 Facility with Multiple Production Approvals. Prepare one original summary. For example, if a facility has a PMA and a TSO authorization, prepare one original summary.
(d) Identify and record specific problems or concerns that the ACSEP evaluation team believes require further action and that should be brought to the attention of the ACO, MIO, MIDO, or CMO managers, the geographic PI, the AE, and the Flight Standards principal maintenance inspector (as appropriate). Use the instructions in appendix F to record these special emphasis items. Prepare original documents as follows:
1 PAH or Associate Facility. Prepare one original document.
2 Facility with Multiple Production Approvals. Prepare only one original document. For example, if a facility has a PMA and a TSO authorization, prepare one original document.
(e) Discuss with team members, as appropriate, and record any lessons learned during the ACSEP evaluation that may improve ACSEP policy or evaluation techniques. Use the instructions in appendix G. Prepare only one original document and include copies with each report.
(f) Verify that signed original Form(s) 8100-6 have been prepared for inclusion, as applicable, in each ACSEP evaluation report to be sent to the responsible certificate management MIDO, CMO, or ACO having delegation oversight. See paragraph 4-21f. Each report to be sent must include all applicable Form(s) 8100-6. When a signed original Form 8100-6 is applicable to two or more reports, do the following:
1 Reproduce the signed original Form(s) 8100-6 as required for inclusion in the applicable ACSEP evaluation report(s) to be sent to the responsible certificate management MIDO or CMO with oversight.
2 Identify all true copies of the signed form in accordance with Order 2150.3.
(g) Provide a copy of the completed final draft Form(s) 8100-6 to the certificate management PI or AE, and the geographic PI, as applicable, when they are present.
(h) Verify that the required number of true copies of objective evidence have been prepared for inclusion, as applicable, in each ACSEP evaluation report to be sent to the responsible certificate management MIDO or CMO having oversight.
08/31/2010 8100.7E
(i) Provide all true copies of objective evidence to the certificate management PI or AE, when present. When the PI or AE is not present, forward the copies in accordance with the applicable instructions in paragraph 4-23a. If the objective evidence will be necessary as a reference during preparation of the evaluation report, make a separate copy and identify each page as “For Reference Only.”
(3) Certificate Management PI or AE, or Geographic PI (When Present). As appropriate, consider providing a copy of the completed final draft Form(s) 8100-6 to the facility’s management. Clearly mark each copy as “DRAFT” before release.
4-17. Postevaluation Conference. The team leader or principal evaluator must conduct a postevaluation conference with appropriate senior management and cognizant supervisory personnel of the evaluated facility. The team leader or principal evaluator must, as appropriate, do the following:
a. Introduce FAA personnel not previously introduced at the preevaluation conference.
b. Give a brief presentation of the overall results of the evaluation, using each completed ACSEP Evaluation Executive Summary as a reference:
(1) Provide a copy of each completed ACSEP Evaluation Executive Summary to the evaluated facility’s designated representative.
(2) Summarize all noncompliances. Mention only noncompliances previously discussed with the certificate management PI and AE, the geographic PI, as applicable, and facility personnel.
c. Explain the purpose and use of the ACSEP database.
d. Explain corrective action and followup procedures.
Note: Emphasize that the PI or AE may conduct additional investigations into noncompliances reported in the ACSEP evaluation report. The results of these investigations may be included with the letter requesting corrective action for the ACSEP evaluation noncompliances.
e. Remind senior management about Form 8100-7 and encourage them to complete the form and send it to the address on the form within 30 calendar days of the postevaluation conference.
f. Request final comments. Clarify any misunderstandings or disagreements before departure.
g. Adjourn the ACSEP evaluation.
4-18. through 4-20. Reserved.
08/31/2010 8100.7E
Section 3. Postevaluation Activities
4-21. Preparing the ACSEP Evaluation Report. The team leader or principal evaluator must prepare the ACSEP evaluation report. When a facility has one or more production approvals, prepare one original evaluation report. Format and compile each evaluation report in the CMIS program. The report will consist of the following:
Note: Ensure that the evaluation report identifies only noncompliances presented at the postevaluation conference.
a. FAA Form 8100-3, ACSEP Evaluation Report, or printed copy of electronic equivalent (appendix H). Each form or printed copy must be an original and signed. Prepare an original form or printed copy for each PAH affected.
b. ACSEP Executive Summary or printed copy of electronic equivalent (appendix E). Each summary must be an original and signed. Prepare an original summary or printed copy for each PAH affected.
c. ACSEP Evaluation Special Emphasis Items or printed copy of electronic equivalent (appendix F). Prepare an original list of special emphasis items or printed copy for each PAH affected.
d. ACSEP Evaluation Lessons Learned or printed copy of electronic equivalent (appendix G). Prepare an original list of lessons learned or printed copy for each evaluation.
e. Form 8100-4 or printed copy of electronic equivalent (appendix D, part B). Prepare an original form or printed copy for each PAH facility affected.
f. Form 8100-6 or printed copy of electronic equivalent. Each report must include all applicable Form(s) 8100-6.
Note: Do not include reproductions of true copies of objective evidence in an original evaluation report. Objective evidence must be a true copy signed and dated in accordance with Order 2150.3.
4-22. Quality Review of the ACSEP Evaluation Report. The ACSEP Evaluation Report contains the data that forms the basis of corrective action requests (see paragraph 4-24) and the ACSEP national database described in chapter 5 of this order. To this end, the evaluation report must be accurate and complete. Directorate managers (or delegated individuals) must establish a review process within their directorates that ensures accuracy and completion of the evaluation report before distribution.
08/31/2010 8100.7E
4-23. Sending the ACSEP Evaluation Report. Using CMIS, the team leader or principal evaluator and the responsible ACO and MIO managers (or delegated individuals) will process the evaluation report as follows (see appendix I):
a. Team Leader or Principal Evaluator.
(1) Make the evaluation report available to the responsible MIDO/CMO manager or delegate within 15 working days of the postevaluation conference. The manager or delegate must return the report to the team leader or principal evaluator for correction and/or continued processing within 5 working days of receipt.
(2) Make the evaluation report available to the responsible certificate management MIO manager within 5 working days of receipt of the MIDO/CMO manager or delegate comments. Do not send copies of objective evidence to the MIO manager. Send or deliver all true copies of any objective evidence to the attention of the certificate management PI.
(3) Make the evaluation report available to the cognizant ACO manager and to AIR-200. The copy for the ACO manager may be tailored according to the needs of that manager. Include copies of any objective evidence that the ACO manager may require to investigate identified special emphasis items. These copies must be sent or delivered to the attention of the ACO manager. Do not send copies of objective evidence to AIR-200.
(4) Make the evaluation report available to the immediate supervisor of any evaluators-in-training assigned to the team.
b. Certificate Management MIO Manager.
(1) Make the evaluation report available to the certificate management PI within 3 working days of receipt of the report from the ACSEP team leader.
(2) Include any additional evaluation documents that the team leader provides.
c. Certificate Management ACO Manager.
(1) Make the evaluation report available to the AE within 3 working days of receipt of the report from the ACSEP team leader.
(2) Send or deliver all copies of any objective evidence to the attention of the AE, as applicable; send the true copies of the objective evidence under separate cover.
Note: ACO investigations of special emphasis items identified during the conduct of an ACSEP evaluation should be coordinated with the responsible MIDO or CMO.
4-24 Requesting Corrective Action. The PI must request corrective action in accordance with Order 8120.2.
08/31/2010 8100.7E
Chapter 5. ACSEP and CMIS
5-1. Purpose. Evaluation and audit data resulting from PAH certificate management activities is stored in CMIS. Upon extraction from CMIS, this data can be manipulated using Excel or other software with statistics capabilities. The software will be used to detect shifts in performance and statistically significant trends within the manufacturing industry, by Directorate, by production approval type, or by other categories as supported by the data available within CMIS. CMIS data may also be used to study various aspects of the performance of ACSEP evaluations on an as-required basis.
5-2. Files. CMIS contains all ACSEP evaluation-related forms, including FAA Form 8100-3, the ACSEP Evaluation Report; FAA Form 8100-4, the ACSEP Survey Sheet for Production Approval Holders; FAA Form 8100-6, the Noncompliance Record; FAA Form 8100-7, the ACSEP Evaluation Customer Feedback Report; and ACSEP Evaluation Lessons Learned.
5-3. Database Management. AIR-200 is responsible for monitoring CMIS and will, as appropriate, do the following:
a. Review the database as follows:
(1) Enter into CMIS any completed Form 8100-7 as returned by the facility.
(2) Highlight noncompliance trends with respect to the system elements.
(3) Analyze noncompliance trends with respect to the system elements.
(4) Highlight trends emerging in the performance of ACSEP evaluations.
b. Provide selected data and reports.
Note: All recipients of CMIS evaluation and audit data will use the information internally only and will not release results outside of AIR. Refer to appendix L, paragraph 9 of this order.
5-4. Use of the Database. Directorates may use CMIS to obtain reports on noncompliances, frequently used 14 CFR references, and PAH compliance. They may use the database to detect shifts in performance and statistically significant trends for different segments of the industry. Directorates also may use the database to assist in scheduling.
08/31/2010 8100.7E
Intentionally Left Blank
08/31/2010 8100.7E Appendix A
Appendix A. Preparation of Clauses
for Contracts for Support Services
1. Purpose. This appendix provides sample contract clauses and a sample certificate of nondisclosure for use in contracts for obtaining services to support ACSEP evaluations, information systems, and system analyses.
2. Sample Clauses and Attachment. The following sample clauses provide the minimum requirements to be included in a contract for support services. Figure 1 shows a sample attachment to the Confidentiality of Information clause requiring support service personnel to agree to its terms and conditions.
a. The following clause is applicable to all contractors:
H.1 Confidentiality of Information.
a. To the extent that the work under this contract requires that the contractor be given access to confidential or proprietary business or technical information belonging to the Government or other companies, designees, contractors, or competitors, or to the extent that in performing the work under this contract, the contractor gains access to Government data through any means, then the contractor must, after receipt thereof, treat such information as confidential and agree not to appropriate such information to its own use or to disclose such information to third parties unless specifically authorized by the contracting officer in writing; however, the foregoing obligations must not apply to the following:
(1) Information that, at the time of receipt by the contractor, is in public domain.
(2) Information that is published after receipt thereof by the contractor or otherwise becomes part of the public domain through no fault of the contractor.
(3) Information that the contractor has in its possession at the time of receipt thereof and was not acquired directly or indirectly from the Government or other companies.
(4) Information that the contractor can demonstrate was received by it from a third party who did not require the contractor to hold it in confidence.
b. The contractor must execute the certificate set forth as attachment 1 for each employee who will participate as an evaluator under this contract. The certificate must be presented by the contractor’s employees or forwarded by the FAA to various companies who may be evaluated under the contract.
08/31/2010 8100.7E Appendix A
Figure 1. Sample Certificate of Nondisclosure
Attachment 1. Certificate of Nondisclosure
The undersigned hereby agrees to the terms and conditions set forth in the clause below:
H.1 Confidentiality of Information.
a. To the extent that the work under this contract requires that the contractor be given access to confidential or proprietary business or technical information belonging to the Government or other companies, designees, contractors, or competitors, or to the extent that in performing the work under this contract, the contractor gains access to Government data through any means, then the contractor must, after receipt thereof, treat such information as confidential and agree not to appropriate such information to its own use or to disclose such information to third parties unless specifically authorized by the contracting officer in writing; however, the foregoing obligations must not apply to the following:
(1) Information that, at the time of receipt by the contractor, is in public domain.
(2) Information that is published after receipt thereof by the contractor or otherwise becomes part of the public domain through no fault of the contractor.
(3) Information that the contractor has in its possession at the time of receipt thereof and was not acquired directly or indirectly from the Government or other companies.
(4) Information that the contractor can demonstrate was received by it from a third party who did not require the contractor to hold it in confidence.
b. The contractor must execute the certificate set forth as attachment 1 for each employee who will participate as an evaluator under this contract. The certificate must be presented by the contractor’s employees or forwarded by the FAA to various companies who may be evaluated under the contract.
… … Authorized Contractor Agent Date Contractor Employee Date
08/31/2010 8100.7E Appendix A
b. The following- clause is applicable to support service personnel who will support ACSEP evaluations and should be used in conjunction with clause H.1:
H.2 Relationships. The contractor must provide support to the Government by completing work assigned under this contract. Support must be provided in the following areas: auditing of quality and engineering functions; collection, evaluation, and processing of data; and written documentation of incidents not in compliance with ACSEP evaluation criteria. The contractor must not provide technical direction under the contract. The contractor must abide by any limitations of access and entry to proprietary or sensitive processes or systems that the Government may stipulate. Although the effort under this contract may include the collection and processing of data, as well as the formulation of noncompliances and recommendations, the final disposition of all information must remain the sole province of the Government.
c. The following clause is applicable to support service personnel who will support database management or system analysis and should be used in conjunction with clause H.1:
H.2 Relationships. The contractor must provide support to the Government by completing work assigned under this contract. Support must be provided in the following areas: input, analysis, and trending of data; and compilation of analytical reports. The final disposition of all information must remain the sole province of the Government.
08/31/2010 8100.7E Appendix A
Intentionally Left Blank
08/31/2010 8100.7E Appendix B
Appendix B. Preparation of the Notification
Letter to a PAH or Associate Facility
1. Purpose. This appendix provides instructions and sample paragraphs for preparing a notification letter to a PAH or associate facility for a scheduled evaluation.
2. Information to Include in the Notification Letter. Figure 1 provides sample paragraphs with the minimum information to include in a notification letter to a PAH or associate facility. Additional paragraphs may be added as necessary to provide specific directorate or AIR-100/200 information.
a. First Paragraph. The first paragraph is introductory and serves to establish the regulatory basis for the evaluation and to identify the facility and type of approval being evaluated. This paragraph applies to all approval types.
b. Second Paragraph. The second paragraph identifies the dates of the evaluation and provides a general outline of the functions to be evaluated.
c. Third Paragraph. The third paragraph identifies the approximate number of evaluators who will be participating in the evaluation and the team leader or principal evaluator, as applicable. In addition, when support service personnel are used to support an evaluation, this paragraph must state the general purpose of the support service personnel, advise use of the FAA certificate of nondisclosure, request special requirements, and identify the support service personnel.
d. Fourth Paragraph. The fourth paragraph requests appropriate senior management attendance at preevaluation and postevaluation conferences, as well as cognizant technical and supervisory personnel. It also requests assignment of knowledgeable escorts.
e. Fifth Paragraph. The fifth paragraph requests senior management feedback on the conduct of the ACSEP evaluation through Form 8100-7 to be sent to the cognizant ACO or MIO manager. This form should be prepared electronically and may be provided to the facility to be evaluated in either electronic or printed format. Prepare Form 8100-7 (figure 2) by typing in the following:
(1) Block 1. The ACSEP number.
(2) Block 2. The name of the evaluated facility.
(3) Block 3. The start and end dates of the evaluation.
(4) Block 4. The address of the cognizant ACO or MIO manager. Enclose a prepaid self-addressed envelope in which the facility may return the form.
f. Final Paragraph. The final paragraph is a closing paragraph indicating to whom specific questions concerning the evaluation should be addressed. It directs that questions relative to scheduling be addressed to the lead evaluation office or requesting MIDO or CMO and that questions relative to the conduct of the evaluation be addressed to the team leader or principal evaluator.
08/31/2010 8100.7E Appendix B
Figure 1. Sample Paragraphs for the Notification Letter
The Federal Aviation Administration (FAA), in accordance with its responsibilities under the recodified Federal Aviation Act of 1958 (, as amended,) and applicable requirements of Title 14, Code of Federal Regulations, has selected (name of PAH/associate facility), located in (city, state), for the conduct of an evaluation. Your certification as a (type of approval holder) has been approved by the FAA contingent upon the Administrator’s right to evaluate and inspect your organization, facilities, product, and records. This includes your entire network of suppliers and approval extensions, as appropriate.
The evaluation of your facility is scheduled to be conducted from (start date) to (end date) under the FAA’s Aircraft Certification Systems Evaluation Program (ACSEP). This evaluation will be broad-based in nature and will encompass elements such as design control, manufacturing processes and controls, and supplier control. Procedures and records will be examined in addition to a “hands-on” witnessing of relevant system processes.
(The FAA evaluation team will consist of approximately (total number) members.) The (FAA team leader designated/principal evaluator) for this evaluation is (Mr./Ms.) (name) who may be reached at (telephone number). (His/Her) address is (office address). The evaluation team will be supported by a support service person who will be performing specific duties on behalf of the FAA. This person is identified below. This person will sign an FAA certificate of nondisclosure that will be forwarded to the facility via the FAA (principal inspector/assigned engineer) before the start of the evaluation. Please inform the FAA of any special requirements necessary for this person to access your facilities and restricted areas.
Support Service Person’s Name Company Affiliation
(Name) (Company)
Attendance by a representative of senior management responsible for the facility to be evaluated, as well as cognizant technical and supervisory personnel, is requested during the preevaluation and postevaluation conferences. We further suggest that escorts who are knowledgeable of the various areas to be visited be provided to ensure the evaluation is conducted smoothly and with minimal disruption to your staff.
One of the primary features of the ACSEP is continuous quality improvement. As part of this process, it is important for us to know what your senior management thought about the conduct of the ACSEP evaluation. We therefore encourage senior management to complete the attached FAA Form 8100-7, ACSEP Evaluation Feedback Report, and return it in the enclosed prepaid self-addressed envelope within 30 calendar days of the postevaluation conference.
If you have any questions concerning the scheduling of this evaluation, please feel free to contact me. If you have any questions concerning the conduct of the evaluation, please contact the (team leader/principal evaluator) (Mr./Ms.) (name of team leader/principal evaluator), at the above address and telephone number.
08/31/2010 8100.7E Appendix B
Figure 2. Sample FAA Form 8100-7, ACSEP Evaluation Customer Feedback Report
Form Approved OMB No. 2120-0605
ACSEP EVALUATION FEEDBACK REPORT
ACSEP No. (1)
Name of Evaluated Facility: (2)
Dates Evaluated: (3)
As part of the Federal Aviation Administration (FAA) and industry continuous improvement efforts for the Aircraft Certification Systems Evaluation Program (ACSEP), this form is provided for your use in furnishing the FAA with comments regarding the conduct of the evaluation recently conducted at your facility. We sincerely encourage you to tell us how we did, and thank you for the time you will take to support our quality improvement and customer service objectives.
Please check the appropriate rating in each of the tables below, and provide any comments that you deem appropriate.
1. Pre-evaluation arrangements Unsatisfactory Poor Satisfactory Good Excellent • Timeliness • Coordination/Planning Comments/recommendations for improvement:
2. Pre-evaluation conference Unsatisfactory Poor Satisfactory Good Excellent • Communication • Presentation • Purpose of evaluation explained Comments/recommendations for improvement:
FAA Form 8100-7 (10/02)
08/31/2010 8100.7E Appendix B
Figure 2. Sample FAA Form 8100-7, ACSEP Evaluation Customer Feedback Report (Continued)
ACSEP EVALUATION FEEDBACK REPORT, con’t
3. Daily meetings Unsatisfactory Poor Satisfactory Good Excellent • Explanation of noncompliances • Resolution of issues Comments/recommendations for improvement:
4. Postevaluation conference Unsatisfactory Poor Satisfactory Good Excellent • Communication • Explanation of executive summary • Explanation of followup actions Comments/recommendations for improvement:
5. Conduct of the evaluation Unsatisfactory Poor Satisfactory Good Excellent • Team professionalism • Overall conduct of the ACSEP team Comments/recommendations for improvement:
Signature (optional) Date Please return completed form to:
(4)
FAA Form 8100-7 (10/02)
08/31/2010 8100.7E Appendix C
Appendix C. Notification Letter Requirements
1. Purpose. This appendix provides a tabular summary of the primary notification letter requirements identified in chapter 3 of this order.
2. Description. Figure 1 provides a summary by facility type of notification letter requirements for which the lead evaluation office is responsible. It identifies the type of notification activity required and when the notification activity should be accomplished.
Figure 1. Notification Letter Requirements Summary
FACILITY TO BE
EVALUATED
NOTIFICATION
ACTIVITY
TIMETABLE (calendar days before
evaluation) ♦ PAH ♦ Associate Facility
n Letter to facility 50
(Within area of responsibility) Ref. para. 3-5a
o Copy to designated team leader or principal evaluator
50
p Copy to PI/AE 50
08/31/2010 8100.7E Appendix C
Intentionally Left Blank
Appendix D. Format for Standardized Evaluation Criteria for
PAHs and Associate Facilities
Part A. Standardized Evaluation Criteria for
PAHs and Associate Facilities
1. Purpose. This appendix provides standardized evaluation criteria used to document the evaluation of the system elements listed in figure 1 for PAHs and associate facilities.
Figure 1. System Elements
Section
No. System Element Appendix D
Page No. 1 Organizational Management D-2 2 Design Control D-11 3 Software Quality Assurance D-16 4 Manufacturing Processes D-24 5 Manufacturing Controls D-44 6 Supplier Control D-67
2. Description of System Elements Section Format. Each section of this appendix addresses one of the six system elements listed in figure 1. Each section is formatted as follows:
a. System Element Description. This is a brief description of what the system element is intended to accomplish or control.
b. System Element Standardized Evaluation Criteria. The evaluation criteria are located in this order and can also be found as part of the order located on the FAA’s Web site, and are formatted as follows:
(1) Standardized Evaluation Criteria. Each criterion is identified by a numbered question within a box. The format of each question number is based on the specific system element section number identified in figure 1.
(2) Applicability. This identifies whether the criterion applies to a specific type of production approval (PC, PMA, and TSO authorization). A table format is used that identifies the type of facility across the top and a code for the type of applicability in the first column. The codes for the types of applicability are defined as follows:
(a) A. This row within the applicability block is used to identify the 14 CFR source requirements applicable to a specific facility. The applicability to a specific facility is indicated by the specific 14 CFR part or section reference (for example, 14 CFR part 21, Certification Procedures for Products, Articles, and Parts, § 21.137, Quality System.
(b) E. This row within the applicability block is used to identify the enforceable 14 CFR requirement applicable to a specific facility. The applicability to a specific facility is indicated by the enforceable 14 CFR part or section reference (for example, § 21.146, Responsibility of Holder).
(c) P. This applicability code is used within the “A” row to identify criteria that reflect industry best practices and accepted total quality management principles. These practices and principles are often contained in FAA-approved data or other facility procedures. The evaluator must determine the actual level of application at each facility.
(d) N. This applicability code is used within the “A” row or “E” row to indicate that the criterion is generally not applicable at a specific facility.
Note 1: Applicability indicated for a specific type of production approval includes any associate facilities established under that approval.
Note 2: When a “P” or “N” is used in the applicability table, a criterion is applicable and enforceable if it is addressed in the approval holder’s FAA-approved data/quality manual. (Reference § 21.146, § 21.316, or § 21.616)
(3) Statement of Condition. The statement of condition provides guidelines, not requirements that may assist the evaluator in determining adherence to the criteria. These guidelines are not the only acceptable means of implementation. Evaluators may identify additional practices in FAA-approved data or other facility procedures that indicate adherence to the requirements of the criteria.
Section 1. Organizational Management
1. System Element Description. This system element addresses the evaluated facility’s organizational management structure and responsibilities for design control and production functions. This includes procedures and methods used to notify the FAA of specific conditions as required by the applicable CFR (such as recording, reporting, investigation, determining cause, and effecting corrective actions of significant or reported failures, malfunctions, or defects). This function also addresses internal audits whereby the facility ascertains its own abilities and procedural compliance to established policy and guidance.
2. System Element Standardized Evaluation Criteria. The following criteria are used to document the evaluation of this system element.
101. Is the production approval/authorization retained at the evaluated facility in which the product is manufactured and was it available to the FAA upon request?
Applicability
PC PMA TSO A § 21.146 § 21.316 § 21.616 E § 21.146 § 21.316 § 21.616
Statement of Condition
a. There is objective evidence that the production certificate is available as required. This should include all attachments, such as the Production Limitation Record.
102. Is the evaluated facility operating within the production limitations of the production approval?
Applicability
PC PMA TSO A § 21.142 § 21.309 § 21.316
Statement of Condition
a. There is objective evidence that the evaluated facility is manufacturing, for sale/installation, those products it is authorized to manufacture under a production approval.
103. Has the PAH provided to the FAA a document describing how its organization will ensure compliance with provisions of the regulatory subpart?
Applicability
PC PMA TSO A § 21.135 § 21.305 § 21.605 E § 21.146 § 21.316 § 21.616
Statement of Condition
a. The policy document includes:
(1) A description of assigned responsibilities and delegated authority.
(2) A description of the functional relationship of those responsible for quality to management and other organizational components.
104. NO LONGER APPLICABLE.
105. Has the PAH provided to the FAA a quality manual describing its quality system? Are the documents prepared in the English language and retrievable in a form acceptable to the FAA?
Applicability
PC PMA TSO A § 21.138 § 21.308 § 21.608 E § 21.138 § 21.308 § 21.608
Statement of Condition
a. There is objective evidence that the quality manual, including electronically stored versions, is available in the major quality and inspection areas, and is subject to periodic review and revision.
(1) Everyone associated with the quality system is performing within their described assigned responsibilities and delegated authority.
(2) A table or organizational chart describes the functional relationship of the quality organization to management and to the other organizational components.
(3) A description of assigned responsibilities and delegated authority to make changes to the quality system is provided.
(4) The individual identified for managing the quality program has the necessary authority and organizational freedom.
106. Is quality system data, and changes thereto, submitted to the FAA?
Applicability
PC PMA TSO A § 21.150 § 21.320 § 21.620 E § 21.150 § 21.320 § 21.620
Statement of Condition
a. Each change to the quality system is subject to review by the FAA.
b. The holder of a production certificate must immediately notify the FAA, in writing, of any change that may affect the inspection, conformity, or airworthiness of its product or article.
107. Are tags, forms, and other documents described and controlled?
Applicability
PC PMA TSO A P P P E N N N
Statement of Condition
a. Procedures include, but are not limited to—
(1) A sample of each tag, form, and other document with instructions for use as applicable.
(2) A formal change control procedure.
b. There is objective evidence of observance to established procedures.
108. Has the evaluated facility established procedures for control of quality records?
Applicability
PC PMA TSO A § 21.137 § 21.307 § 21.607 E § 21.146 § 21.316 § 21.616
Statement of Condition
a. There is objective evidence that—
(1) Procedures are maintained for identifying, storing, protecting, retrieving, and retaining quality records.
(2) Quality records are retained for at least 5 years.
(3) Quality records having critical components are identified and retained for at least 10 years.
(4) TSO authorization technical design data necessary to determine conformity and airworthiness of each article produced, is retained by the manufacturer.
(5) Records are legible, complete, and accurate.
109. Has the PAH made appropriate notification to the FAA with regard to undue burden, relocation, or change to manufacturing facilities?
Applicability
PC PMA TSO A § 21.139 § 21.309 § 21.609 E § 21.146 § 21.320 § 21.620
Statement of Condition
a. Has “no undue burden” been determined by the FAA for PAH manufacturing facilities located outside the United States?
b. Has FAA approval been obtained before location changes to any PAH manufacturing facilities were made?
c. Has the PAH notified the FAA in writing of any change to its manufacturing facilities that may affect the inspection, conformity, or airworthiness of its product or article?
110. Are failures, malfunctions, and defects reported to the FAA?
Applicability
PC PMA TSO A § 21.3 § 21.3 § 21.3 E § 21.3 § 21.3 § 21.3
Statement of Condition
a. The organization has established procedures for reporting failures, malfunctions, and defects.
b. The organization’s procedures for reporting failures, malfunctions, or defects—
(1) Establish definitions of conditions that must be reported to the FAA.
(2) Establish a method of documenting reportable conditions and a method for reporting them to the FAA.
(3) Require prompt evaluation of each condition to determine if it is reportable to the FAA.
(4) Require that the condition be reported to the FAA within 24 hours (with provisions for weekends and holidays) after it has determined that the failure, malfunction, or defect required to be reported has occurred.
(5) Require retention of each reported condition document, the FAA response, and the organization’s disposition of the condition.
c. There is objective evidence of adherence to these procedures.
111. Are service bulletins and maintenance manuals approved by authorized personnel and coordinated with FAA engineering?
Applicability
PC PMA TSO A P P P E N N N
Statement of Condition
a. Procedures define specific organizational and individual responsibilities for issuing service bulletins, maintenance manuals, service difficulty reports, and other related communication.
b. Changes are approved by authorized personnel and coordinated with FAA engineering.
c. There is objective evidence of observance to established procedures.
112. Are there procedures for receiving and processing feedback on in-service failures, malfunctions, and defects?
Applicability
PC PMA TSO A § 21.137 § 21.307 § 21.607 E § 21.146 § 21.316 § 21.616
Statement of Condition
a. Procedures provide for—
(1) A requirement to include a process for assisting the design approval holder to address any in-service problem involving design changes.
(2) A requirement to include a process for assisting the design approval holder to determine if any changes to the Instructions for Continued Airworthiness are necessary.
(3) Determination of appropriate manufacturing, design, or instructions for continued Airworthiness responsibilities for the reported problem.
(4) A system of tracking for accountability.
(a) Records are generated and maintained.
(b) Contents of each record used include when the report was received, what was reported, and actions taken.
(c) Requirements established that tape files, microfilm, etc., used for record retention exhibit legible data, acceptance stamps and/or signatures, as required.
b. There is objective evidence of observance to established procedures.
113. Are service problems (both design and manufacturing), unairworthy conditions, unsafe features, or unsafe characteristics reported by the FAA or users investigated? Are prompt corrective actions taken by the evaluated facility?
Applicability
PC PMA TSO A § 21.3 § 21.3 § 21.3 E § 21.3 § 21.3 § 21.3
Statement of Condition
a. Procedures provide for—
(1) A method of investigating, identifying, locating, and reporting suspected unsafe products.
(2) Prompt corrective action, which includes, at a minimum—
(a) Root cause determination and correction of deficient design or manufacturing.
(b) A means of purging, tracking, and accountability of known unsafe products.
(3) Investigating reports of unairworthy conditions or unsafe features or characteristics reported by the FAA.
(4) Reporting investigation results and actions taken or proposed to the FAA.
b. There is objective evidence of observance to established procedures.
114. Do procedures provide a method to notify users and recall products, when necessary, when nonconformances are suspected or known to exist in products in service?
Applicability
PC PMA TSO A P P P E N N N
Statement of Condition
a. There is objective evidence of observance to established procedures.
115. Is there a means for keeping users of the product/article informed of service information, including field purges?
Applicability
PC PMA TSO A P P P E N N N
Statement of Condition
a. Procedures provide for informing product users of service difficulties, and of required field purges for suspected or known unsafe conditions.
b. There is objective evidence of observance to established procedures.
116. Are there procedures for planning, conducting, and documenting internal audits?
Applicability
PC PMA TSO A § 21.137 § 21.307 § 21.607 E § 21.146 § 21.316 § 21.616
Statement of Condition
a. Procedures provide for—
(1) Planned and documented internal audits of personnel, procedures, operations, equipment, material, processes performed, and records in all major functional areas.
(2) Criteria for conducting compliance, systems, and product audits.
(3) A formal audit schedule that is available, approved by management, and followed.
(4) Requirements for the qualification and training of personnel who are performing the audits.
(5) Auditors who are independent of the activity being audited.
(6) Special audits when significant customer problems are detected, or when there are significant changes to processes or systems.
(7) Methods for identifying and reporting nonconformance and obtaining required corrective action.
117. Are results of internal audits reported to the appropriate facility personnel?
Applicability
PC PMA TSO A § 21.137 § 21.307 § 21.607 E § 21.146 § 21.316 § 21.616
Statement of Condition
a. Procedures provide for—
(1) A requirement to report the results of internal audits to the manager responsible for implementing corrective and preventive actions.
(2) Review of internal audit results by personnel having responsibility for the areas audited.
(3) Root cause determination and development of appropriate and prompt corrective action.
(4) Followup audits (as necessary) to ensure effective implementation of corrective action.
(5) Actions taken to determine if changes are required to the quality system or other similar processes (which may not have been evaluated) in addition to correcting reported noncompliance.
118. Are there procedures for implementing corrective and preventive actions to eliminate the causes of an actual or potential nonconformity to the approved design, or noncompliance with the approved quality system?
Applicability
PC PMA TSO A § 21.137 § 21.307 § 21.607 E § 21.146 § 21.316 § 21.616
Statement of Condition
a. There is objective evidence of observance to established procedures.
119. Are there procedures for identifying, analyzing, and initiating corrective action for products or articles that have been released from the quality system, but do not conform to applicable design data or quality system requirements?
Applicability
PC PMA TSO A § 21.137 § 21.307 § 21.607 E § 21.146 § 21.316 § 21.616
Statement of Condition
a. There is objective evidence of observance to established procedures.
Section 2. Design Control
1. System Element Description. The methods for approving, controlling, and documenting FAA-approved designs and design changes. Specific functions necessary include the planning and integration of the evaluated facility’s procedures for continuously maintaining the integrity of design data, drawings, part lists, and specifications necessary to define the configuration and the design features of the product. This includes software used in type-certificated aircraft or related products (airborne software).
2. System Element Standardized Evaluation Criteria. The following criteria are used to document evaluation of this system element.
201. Are there procedures for the control of technical data/documents and do they include storage, maintenance, and protection?
Applicability
PC PMA TSO A P P P E N N N
Statement of Condition
a. Procedures provide for—
(1) Storing, maintaining, and protecting design data/documents to preserve their integrity, including magnetic storage media used as part of design documentation, if applicable.
(2) Identification of technical data/documents.
(3) Indication of technical data/documents approval, including FAA approval.
(4) A list of technical data/documents necessary to define configuration of the FAA-approved design.
b. There is objective evidence of observance to established procedures.
202. Are there procedures for controlling design data and subsequent changes?
Applicability
PC PMA TSO A § 21.137 § 21.307 § 21.607 E § 21.146 § 21.316 § 21.616
Statement of Condition
a. There is objective evidence of—
(1) Use of current, correct, and approved data.
(2) Control of design and technical data document issuance, including persons authorized to obtain documents and for retrieval of obsolete documents.
(3) The method for making available to or notifying employees concerning changes in technical data.
(4) Verification that correct documents are in use for the product being produced.
(5) Current design and technical data document distribution lists.
(6) A complete and current file of technical data, including design drawings and specifications.
(7) Electronically stored and transmitted technical design and quality data are adequately controlled.
203. Do the manufacturing, quality, and service/support organizations participate in the review of design and technical data changes?
Applicability
PC PMA TSO A P P P E N N N
Statement of Condition
a. Procedures provide for the manufacturing organization, quality organization, and service/support organization to review design and technical data changes before release to ensure the product can be produced in conformity to an FAA-approved design.
(1) The product can be properly evaluated and verified to be in conformity to an FAAapproved design. Inspection equipment is available or can be procured that will adequately verify conformity to FAA-approved design, and that can be controlled for accuracy, when required.
(2) Service/product organization review design data changes before release to ensure appropriate airworthiness and service documents that are affected by the design change are revised as required.
b. There is objective evidence of observance to established procedures.
204. NO LONGER APPLICABLE.
205. Are changes to technical data referenced on FAA-approved design data (specifications, installation instructions [when applicable], and airborne software documentation) appropriately documented and approved?
Applicability
PC PMA TSO A P P P E N N N
Statement of Condition
a. Procedures provide that changes to technical data referenced on FAA-approved design data are documented and approved in the same way as changes to product design.
b. There is objective evidence of observance to established procedures.
206. Are minor design changes approved under a method acceptable to the FAA?
Applicability
PC PMA TSO A § 21.95 § 21.319 § 21.619 E § 21.95 § 21.319 § 21.619
Statement of Condition
a. There is objective evidence that—
(1) Minor changes in a type design are approved by the FAA or by a method acceptable to the FAA. For example, an FAA-approved procedure whereby the PAH approves minor design changes.
(2) For TSO articles, all necessary revised data are submitted to the FAA when minor changes are made and agree with any part number plan specified in the original application.
(3) For PMA, a minor change to the design of an article has no appreciable effect on the approval basis.
b. There is objective evidence of observance to established procedures.
207. Are major design changes, including process specification changes, submitted to the FAA for approval?
Applicability
Statement of Condition
a. There is objective evidence that—
(1) Major design changes are submitted to the FAA for approval, including changes to manufacturing and special process specifications.
(2) Design changes resulting from applicable airworthiness directives (AD), and design changes, which contribute to the safety of the product, are submitted to the FAA for approval.
(3) For TSO articles, a new type or model designation has been assigned to a changed article and that there has been prompt application for a new TSO authorization.
208. Have design changes necessary to correct unsafe conditions been incorporated into the FAA-approved design, when applicable?
Applicability
PC PMA TSO A § 21.99 § 21.307 § 21.607 E § 21.99 § 21.307 § 21.607
Statement of Condition
a. There is objective evidence that design changes necessary to correct unsafe conditions have been incorporated into the FAA-approved design. This evidence may include one or more of the following:
(1) Identification of applicable ADs.
(2) Tracking the status of AD incorporation.
(3) Furnishing the customer with the AD incorporation status at the time the product is delivered.
209. Are the instructions for continued airworthiness kept current with design changes, when appropriate, and made available to appropriate persons?
Applicability
PC PMA TSO A § 21.50 § 21.50 P E § 21.50 § 21.50 N
Statement of Condition
a. There is objective evidence of observance to established procedures.
210. Is descriptive data and information on FAA-approved design changes resulting from incorporation of ADs or that contribute to the safety of the product made available to users of the product?
Applicability
PC PMA TSO A § 21.99 P P E § 21.99 N N
Statement of Condition
a. There is objective evidence that all applicable descriptive data and information covering FAA-approved design changes or improvements that contribute to the safety of the product are made available to product users.
211. If commercial parts are used, has the PAH provided to the FAA a listing of parts defined as commercial, along with additional information, as required?
Applicability
PC PMA TSO A § 21.50 § 21.50 N E § 21.50 § 21.50 N
Statement of Condition
a. Procedures provide for the submittal of commercial parts list and supporting data for all parts designated as commercial.
Section 3. Software Quality Assurance
1. System Element Description. This system element addresses the planning and integration of the evaluated facility’s procedures for continuously maintaining the integrity of software used in type-certificated aircraft or related products (airborne software), and the integrity of software and related hardware used for product acceptance. Document DO-178, Software Considerations in Airborne Systems and Equipment Certification (current edition), of the Radio Technical Commission for Aeronautics (RTCA), or comparable means, should be used as guidance for control of airborne software.
2. System Element Standardized Evaluation Criteria. The criteria used to document the evaluation of this system element are divided into two parts: Part A, Airborne Software, and Part B, Product Acceptance Software.
Part A. Airborne Software
301. Is there a Software Configuration Management Plan or procedure to control airborne software configuration?
Applicability
PC PMA TSO A P P P E N N N
Statement of Condition
a. Procedures provide for—
(1) Installation of the correct version of the software in the delivered product in accordance with the FAA-approved design.
(2) A method by which controlled software containing the FAA-approved design data is transitioned into production. The media containing the software installed in the product is directly traceable to the Software Configuration Management library.
(3) Documentation of integration of software with hardware to specify a unique version for incorporation into the product.
(4) Cross-reference of software documents to their associated software.
(5) The technical data/documents control system that includes software identification methods at the media level and at the product level. The media level identification is incorporated into the software, and the product level identifications are marked on the outside of the product indicating software configuration.
b. There is objective evidence of observance to established procedures.
302. Is there a Configuration Index Document (CID) listing all software documents under configuration control and defining the hardware and software part numbers?
Applicability
PC PMA TSO A P P P E N N N
Statement of Condition
a. Procedures provide for trace ability of hardware and software part numbers to the drawing control system.
b. There is objective evidence of observance to established procedures.
303. Are there practices and procedures for reporting, tracking, and resolving software problems?
Applicability
PC PMA TSO A P P P E N N N
Statement of Condition
a. Corrective action procedures, for problems found subsequent to the FAA-approved design, include provisions for airborne software and hardware/software combinations. Procedures may parallel or be part of hardware corrective action procedures.
b. Problem reports addressing changes to software code are under change control.
c. The production test procedures have been modified to reflect the software change and successfully executed against the changed version.
d. There is objective evidence of observance to established procedures.
304. Is obsolete and noncurrent software media recalled and purged, when applicable?
Applicability
PC PMA TSO A P P P E N N N
Statement of Condition
a. Configuration control procedures for airborne software include methods of purging software for removal of obsolete and noncurrent media, when applicable. Procedures may parallel or be part of hardware purging procedures.
b. Procedures include methods to identify, store, or dispose of obsolete and noncurrent media, when applicable.
c. There is objective evidence of observance to established procedures.
305. Are there methods and facilities to protect computer programs from unauthorized access, inadvertent damage, or degradation?
Applicability
PC PMA TSO A P P P E N N N
Statement of Condition
a. Procedures provide—
(1) Configuration control of the airborne software within the product design files.
(2) Limited access to software files and protection from unauthorized changes.
(3) Separate archives for masters and duplicates.
(4) That masters and duplicates are not revived by the same machine simultaneously.
(5) Minimized risk of deterioration and regeneration of errors on selected storage medium.
(6) Assurance that the reproduction of code occurs error free.
b. There is objective evidence of observance to established procedures.
306. Are there procedures to ensure documentation and archival for each version of the delivered airborne software version?
Applicability
PC PMA TSO A P P P E N N N
Statement of Condition
a. Procedures (that is, version description document) provide for methods to identify, document, and archive the software environment for each version of delivered airborne software.
b. There is objective evidence of observance to established procedures.
307. Is software identified/marked externally/internally in accordance with the engineering drawing requirements?
Applicability
PC PMA TSO A P P P E N N N
Statement of Condition
a. Work instructions detail the identification/marking requirements.
b. There is objective evidence of observance to established instructions.
308. Is airborne software programmed media handled and stored properly (for example, environmental controls and magnetic interference precautions)?
Applicability
PC PMA TSO A P P P E N N N
Statement of Condition
a. Procedures provide for special handling of programmed media.
b. There is objective evidence of observance to established procedures.
309. Are build and load instructions established, maintained, and used?
Applicability
PC PMA TSO A P P P E N N N
Statement of Condition
a. Procedures provide—
(1) Software build and load into hardware components.
(2) Successful testing of the hardware after the software load.
b. There is objective evidence of observance to established procedures.
Part B. Product Acceptance Software
310. Is there a Software Configuration Management Plan or procedure to control product acceptance software configuration?
Applicability
PC PMA TSO A P P P E N N N
Statement of Condition
a. Procedures provide for—
(1) Identification of software for an application.
(2) Control of approved versions for product acceptance.
(3) Control of obsolete and noncurrent software.
(4) Identification of software with a software configuration identification.
(5) Documentation of integration of software with hardware to specify a unique version for incorporation into the product.
(6) Cross-reference of software documents to their associated software.
(7) The technical data/documents control system that includes software identification methods at the media level and at the product level. The media level identification is incorporated into the software, and the product level identifications are marked on the outside of the product indicating software configuration.
b. There is objective evidence of observance to established procedures.
311. Are all changes to product acceptance software documented and approved?
Applicability
PC PMA TSO A P P P E N N N
Statement of Condition
Continue reading ↓ 120 of 235 passages shown