Safety Recommendation A-14-121

We believe AC 25.1309-1B contains information for applicants, certification engineers, and engineering designees on using traceability principles in aircraft development and certification programs that is consistent with the draft. Consequently, Safety Recommendation A-14-121 is classified CLOSED-- ACCEPTABLE ACTION.

We also believe that revising the FAA’s web-based training course for certification engineers and engineering designees to reflect the guidance contained in AC-25.1309-1B will satisfy Safety Recommendation A-14-122. Pending our review of the revised training, Safety Recommendation A-14-122 remains classified Open—Acceptable Response.

-From Michael G. Whitaker, Administrator: The Federal Aviation Administration (FAA), in our efforts to continue developing a comprehensive approach to system safety assessments, requirements management, and assumptions tracking from development through certification of new technology-type certification projects, has increased the scope of the new advisory circular (AC) to include instructional and procedural guidance. This AC will inform applicants, certification engineers, and engineering designees about the use of traceability principles in aircraft development and certification programs, in addition to other topics.

We now anticipate releasing the draft AC for public comment by June 30, 2024.

Additionally, FAA Notice N8110.118, Submittal and Disclosure of Safety Critical Information by Applicants for Transport Category Airplane Type Certificates, was published for public comment on July 25, 2023. This draft Notice:

• Provides guidance on how to use a certification plan as a conduit for the applicant’s submittal of safety critical information.

• Provides guidance for this information to be regularly updated, at milestones defined within the agreed-upon project-specific certification plan.

• Recommends processes for the FAA to receive and review safety critical information.

• Provides connectivity for FAA certification specialists and applicants between type certification procedures and safety assessments.

The FAA anticipates issuing the final Notice in Calendar Year 2024. The draft Notice is available at the following website:

https://www.faa.gov/media/68591

The FAA notes that during the public comment period for the System Safety Assessment Notice of Proposed Rulemaking, we received the Board’s comment regarding AC 25.1309-1B, System Design and Analysis. The Board noted that “finalizing the revisions proposed in draft AC-25.1309-1B, which contains guidance that informs applicants, certification engineers, and engineering designees about the use of traceability principles in aircraft development and certification programs, will satisfy Safety Recommendation A-14-121.” We anticipate publishing AC 25.1309-1B by August 8, 2024.

In our March 2, 2023, comments responding to your “System Safety Assessments” NPRM, we informed you that finalizing the revisions proposed in draft AC 25.1309-1B, which contains the recommended guidance for certification engineers, will satisfy Safety Recommendation A-14-119. Pending timely publication of a final AC that is consistent with the draft, Safety Recommendation A-14-119 is classified Open—Acceptable Response.

We also said in our NPRM response that finalizing the revisions proposed in draft AC-25.1309-1B, which contains guidance that informs applicants, certification engineers, and engineering designees about the use of traceability principles in aircraft development and certification programs, will satisfy Safety Recommendation A-14-121. Pending timely publication of a final AC that is consistent with the draft, Safety Recommendation A-14-121 remains classified OPEN-- ACCEPTABLE RESPONSE.

Our March 2023 NPRM response further said that revising the FAA’s web-based training course for certification engineers and engineering designees to reflect the guidance contained in draft AC-25.1309-1B will satisfy Safety Recommendation A-14-122. Pending our review of the revised training, Safety Recommendation A-14-122 remains classified Open—Acceptable Response.

Prior to the FAA publishing draft AC-25.1309-1B for public comment, your staff provided an advance copy of the draft guidance for my staff to review. In our August 30, 2022, letter regarding this recommendation, we said that we supported the revisions proposed in draft AC-25.1309-1B, and publishing a final AC that is consistent with the draft will satisfy Safety Recommendation A-14-123, assuming no substantive changes are made. In our March 2023 NPRM comments, we informed you that finalizing the revisions proposed in draft AC 25.1309 1B will not complete the actions needed to satisfy Safety Recommendation A-14-123.

We believe the guidance contained in draft AC 25.1309-1B will help ensure that applicants discuss key assumptions related to safety-significant failure conditions, their validation, and their traceability to requirements and proposed methods of compliance during certification planning meetings for type designs involving special conditions. Therefore, we believe the guidance contained in the draft AC, if finalized, will represent an acceptable alternate solution to this recommendation. Pending timely publication of a final AC that is consistent with the draft, Safety Recommendation A-14-123 is classified Open—Acceptable Alternate Response.

The National Transportation Safety Board (NTSB) has reviewed the Federal Aviation Administration’s (FAA) notice of proposed rulemaking (NPRM) titled, “System Safety Assessments,” which was published at 87 Federal Register 75424 on December 8, 2022. We welcome the opportunity to provide comments on this NPRM, in which the FAA proposes to amend certain airworthiness regulations to standardize the criteria for conducting safety assessments for systems, including flight controls and powerplants, installed on transport-category airplanes.

According to the NPRM, the proposed action addresses a congressional mandate, safety recommendations from the NTSB, industry working groups safety analyses, and updates to regulations and provides insight to new regulations related to system safety. The NPRM also moves the United States closer to harmonizing Title 14 Code of Federal Regulations (CFR) 25.1309 with European Aviation Safety Agency Certification Specification-25. The intended effect of this proposed action is to improve aviation safety by making system safety certification requirements more comprehensive and consistent. The NTSB has long recognized the need for system safety assessments to improve safety in aircraft design.

The following comments discuss 10 safety recommendations issued to the FAA regarding issues addressed in the NPRM and accompanying draft advisory circulars (ACs).

Training and Guidance for FAA Certification Engineers and Engineering Designees

Safety Recommendation A-14-119 asked the FAA to provide its certification engineers with written guidance and training to ensure that (1) assumptions, data sources, and analytical techniques are fully identified and justified in applicants’ safety assessments for designs incorporating new technology and (2) an appropriate level of conservatism is included in the analysis or design, consistent with the intent of AC 25.1309 (Arsenal draft). Footnote: Following the January 7, 2013, battery fire incident involving a Japan Airlines Boeing 787-8 in Boston, Massachusetts, we issued Safety Recommendations A-14-119 and A 14 121 through -123 regarding system safety and certification.

Safety Recommendation A-14-119 is currently classified Open—Unacceptable Response. For more information about this investigation’s findings, see NTSB. 2014. Auxiliary Power Unit Battery Fire, Japan Airlines Boeing 787-8, JA829J, Boston, Massachusetts, January 7, 2013, Aircraft Incident Report AIR-14/01. Washington, DC: NTSB. Our review of draft AC-25.1309-1B, System Design and Analysis, found that it is responsive to Safety Recommendation A-14-119 with respect to providing written guidance to FAA certification engineers.

-From Billy Nolen, Acting Administrator: The Federal Aviation Administration (FAA) is developing a comprehensive approach to system safety assessments, requirements management, and assumptions tracking from development through certification of new technology-type certification projects. This action will better assure that safety issues are identified, managed, and mitigated appropriately.

As part of this approach, we continue to draft a new standalone advisory circular (AC) to inform applicants, certification engineers, and engineering designees about the use of traceability principles in aircraft development and certification programs, in addition to other topics, as discussed in our previous response. Since this draft AC overlaps with aspects from the Aircraft Certification, Safety, and Accountability Act and the FAA’s broader comprehensive approach, we have adjusted our schedule for its publication. We now anticipate releasing the draft AC for public comment by June 30, 2023, and publishing it by June 30, 2024.

We note that you intend to publish a new standalone advisory circular (AC) with guidance about the use of traceability principles in aircraft development and certification programs. We support the draft AC, which we reviewed last year, and believe publishing a final AC that is consistent with the draft will satisfy these recommendations, assuming no substantive changes are made. Pending our review of the published AC, Safety Recommendations A-14-121 and -123 remain classified OPEN-- ACCEPTABLE RESPONSE.

-From Steve Dickson, Administrator: The Federal Aviation Administration (FAA) is developing a new standalone advisory circular (AC) to inform applicants, certification engineers, and engineering designees about the use of traceability principles in aircraft development and certification programs. This new AC currently describes the program management benefits of using traceability procedures as described in SAE International’s Aerospace Recommended Practice (ARP) 4754A, “Guidelines for Development of Civil Aircraft and Systems.” The new AC addresses traceability during the development and certification of aircraft and articles.

The goals are to improve the visibility of an applicant’s assumptions and track an applicant’s deliverables during certification programs.

The FAA submitted a draft of the new AC to the National Transportation Safety Board staff for review in August 2021 and received a positive response. We plan to publish the new AC by September 30, 2022.

-From Steve Dickson, Administrator: Staff from the Federal Aviation Administration (FAA) and National Transportation and Safety Board (NTSB) met on November 6, 2019, to discuss these recommendations. We notified NTSB staff that we determined that Advisory Circular (AC) 20-174, Development of Civil Aircraft and Systems, dated September 30, 2011, cites SAE Aerospace Recommended Practice (ARP) 4754A, Guidelines for Development of Civil Aircraft and Systems, as an acceptable method for developing a design assurance level for new technologies.

These two documents address Safety Recommendation A-14-121 for traceability principles, methods of compliance, and certification when special conditions are involved for new technology. It should be noted that the Boeing 787-8 was type certificated on August 26, 2011, which is prior to the formal recognition of ARP 4754A. All FAA documents are available at http://rgl.faa.gov.

The FAA continues to develop a new standalone AC to address the traceability in lieu of incorporating the information into the existing FAA Order 81 I0.4C, Type Certification with Change 6. We determined that a new AC is sufficient to address traceability, and we will no longer update Order 8110.4. This AC will provide guidance to certification engineers as well as designees, on how to evaluate and use traceability to increase visibility of assumptions and applicant deliverables during type design of novel technologies. The AC publication has been extended to December 2020 due to extended comments and revision updates.

We previously told you that your plan to improve traceability documentation guidance for applicants in FAA Order 8110.4, “Type Certification,” and issue a new standalone advisory circular (AC) would satisfy these recommendations if the documents provided sufficient guidance for FAA engineers and designees reviewing systems safety assessments. On November 6, 2019, your staff updated us on this AC and said that it would address the use of traceability principles. We note that you still intend to publish this AC, but that it would be in lieu of incorporating similar guidance into FAA Order 8110.4.

We also note that you believe that AC 20-174, “Development of Civil Aircraft and Systems,” which cites SAE Aerospace Recommended Practice (ARP) 4754A, addresses these recommendations.

We continue to believe the new standalone AC will likely address both of these recommendations; however, if you no longer intend to update Order 8110.4, we would like to know how you will ensure that adequate traceability has been documented when reviewing and approving systems safety assessments. Although AC 20-174 references ARP 4754A as an acceptable method for establishing a development assurance process, the AC does not mention traceability. As part of our investigation of this incident, we reviewed ARP 4754A and how it was used in the battery certification.

We issued these recommendations because we determined that ARP 4754A does not include detailed implementation instructions on how traceability methods should be used to verify the methods of compliance, particularly with special conditions.

Pending our review of the published AC and information on how the FAA will ensure that applicants’ systems safety assessments are thoroughly evaluated, Safety Recommendations A 14 121 and -123 remain classified OPEN--ACCEPTABLE RESPONSE.

We previously said that your plan to improve traceability documentation guidance for applicants in FAA Order 8110.4, “Type Certification,” and a new advisory circular (AC) that you are developing would satisfy these recommendations if the documents provided sufficient guidance for FAA engineers and designees reviewing systems safety assessments. We note that, because of other safety priorities and changes within your organization, the revisions to FAA Order 8110.4 and the complimentary AC have been delayed from June 2016 to a new planned completion date of December 2018.

Pending completion of these revisions and development of the complementary AC, Safety Recommendation A-14-121 remains classified OPEN--ACCEPTABLE RESPONSE. Pending incorporation of this guidance into your training programs for FAA certification engineers and engineering designees, Safety Recommendation A-14-122 remains classified “Open—Acceptable Response.” Pending the revisions to FAA Order 8110.4 and the creation of policy documents related to applicant responsibilities to show compliance with the order, Safety Recommendation A-14-123 remains classified “Open—Acceptable Response.”

-From Michael P. Huerta, Administrator: FAA Comment. In our previous letter to the Board on May 20, 2016, the Federal Aviation Administration (FAA) noted that we were developing a revision to FAA Order 8110.4C, Type Certification, as well as a new complimentary advisory circular (AC) that will provide procedural guidance to the applicant. However, due to other safety priorities and changes within our organizations, the draft revision to FAA Order 8110.4C and the draft complimentary AC are still in the early stages of development. The development process, which requires field and public comment periods, can be lengthy.

Based on current priorities, we expect to complete the revision to Order 8110.4C and the AC by December 2018.

Once the procedural guidance for the applicant is incorporated into an AC, as described in our response to A-14-121, the FAA will incorporate a discussion, as appropriate, of the expectation of the applicant's showing of compliance into the training programs for our certification engineers and engineering designees.

I will keep the Board informed of the FAA's progress on these recommendations and provide an update by June 2018.

We continue to believe that your plan to improve traceability documentation guidance for applicants may provide the needed information for FAA engineers and designees reviewing systems safety assessments. However, we believe that you also need to (1) acknowledge the FAA’s role in reviewing for traceability and (2) ensure that the planned revisions to FAA Order 8110.4C and draft Advisory Circular (AC) 25.1309 (“Arsenal”) provide sufficient guidance for thoroughly evaluating an applicant’s systems safety assessment.

Pending our review of your proposed revisions to FAA Order 8110.4C and AC 25.1309, which should address our concerns, as discussed above, Safety Recommendation A 14 121 remains classified “Open—Acceptable Response.” Until you incorporate guidance into training programs for FAA certification engineers and engineering designees about how to review an applicant’s systems safety analysis to ensure compliance with traceability requirements, Safety Recommendation A 14 122 remains classified “Open—Acceptable Response.”

Pending our review of your proposed revisions to FAA Order 8110.4C and the creation of policy documents related to applicant responsibilities to show compliance with the order, Safety Recommendation A-14-123 remains classified OPEN—ACCEPTABLE RESPONSE.

-From Michael P. Huerta, Administrator: The FAA is developing a revision to FAA Order 81 I 0.4C, Type Certification, as well as a new complimentary advisory circular (AC) that will provide procedural guidance to the applicant. The current draft AC expands on the applicant expectations for traceability between the applicant's showing of compliance and the certification requirements. The FAA is considering additional revisions to discuss the importance of traceability principles to address this recommendation. However, the draft revision to FAA Order 811 0.4C and the draft complimentary AC are still in the early stages of development.

The development process, which requires field and public comment periods, can be lengthy. Based on current priorities, we expect to complete the revision to Order 81 I 0.4C and the AC by June 2016.

CC# 201600220, dated May 12, 2016: The National Transportation Safety Board (NTSB) has reviewed the Federal Aviation Administration’s (FAA) notice of proposed rulemaking (NPRM) titled “Revision of Airworthiness Standards for Normal, Utility, Acrobatic, and Commuter Category Airplanes,” which was published at 81 Federal Register (FR) 13451 on March 14, 2016. The NPRM proposes to amend airworthiness standards for normal, utility, acrobatic, and commuter-category airplanes by removing prescriptive design requirements and replacing them with performance based airworthiness standards.

The proposed standards would also replace the current weight and propulsion divisions in small airplane regulations with performance- and risk based divisions for airplanes with a maximum seating capacity of 19 passengers or less and a maximum takeoff weight of 19,000 lbs. or less. In addition, the NPRM proposes to adopt additional airworthiness standards to address certification for flight in icing conditions, enhanced stall characteristics, and minimum control speed to prevent departure from controlled flight for multiengine airplanes.

On April 8, 2016, the FAA also published on its website draft Advisory Circular (AC) 23.10 titled “FAA Accepted Means of Compliance Process for 14 [Code of Federal Regulations] CFR Part 23.” The draft AC provides guidance on how to submit a proposed means of compliance (MOC) with Part 23 for acceptance by the Administrator in accordance with proposed section 23.10, “Accepted Means of Compliance,” in the NPRM. This letter provides comments on the NPRM. We will submit separate comments regarding draft AC 23.10.

Related Safety Recommendations and Investigations We issued safety recommendations to the FAA and investigated accidents/incidents related to the following topics discussed in the NPRM: Certification One of the FAA’s stated goals in this rulemaking is to encourage the introduction of new and innovative technology. Our investigation of the January 7, 2013, fire in the aft cabin of a Japan Airlines Boeing 787-8, JA829J, which was parked at a gate at General Edward Lawrence Logan International Airport, Boston, Massachusetts, focused on issues related to the safety of new technology introduced to aviation (specifically, new lithium-ion battery applications).

The National Transportation Safety Board (NTSB) has reviewed the Federal Aviation Administration’s (FAA) notice of proposed special conditions titled “Special Conditions: Honda Aircraft Company, Model HA-420 HondaJet, Lithium-Ion Batteries,” which was published in 80 Federal Register 19889 on April 14, 2015.

As the FAA states in its notice, the applicable airworthiness regulations contained in 14 Code of Federal Regulations Part 23 do not include adequate or appropriate safety standards for the HondaJet’s installation of lithium-ion (Li-ion) batteries, which possess certain failure, operational characteristics, and maintenance requirements that differ significantly from that of nickel cadmium and lead acid rechargeable batteries. The proposed special conditions contain the additional safety standards that the FAA administrator considers necessary to establish a level of safety equivalent to that established by the existing airworthiness standards.

In 2014, in connection with our investigation of the January 7, 2013, auxiliary power unit Li-ion battery fire on board a Japan Airlines Boeing 787, the NTSB issued two sets of safety recommendations (A-14-32 through -36 and A-113 through -127) to the FAA addressing Li-ion batteries to be used on transport-category airplanes, which are subject to Part 25 regulations, and the certification of new technology. The safety intent of these recommendations also apply to Li-ion battery installations in general aviation airplane types such as the HondaJet.

We believe that the proposed special conditions are generally adequate but that proof of their effectiveness is in the applicants’ methods of compliance. The NTSB notes that some of the special conditions need more detail to promote an equivalent level of safety, as intended. For example, proposed paragraph e states “no corrosive fluids or gasses that may escape from any Li-ion battery may damage surrounding airplane structure or adjacent essential equipment.” The NTSB believes that this special condition should explicitly indicate electrical wiring in the list of items that could be damaged.

We note your position that the responsibilities discussed in this recommendation should be placed on the applicant rather than on FAA certification engineers and FAA designees. We issued these recommendations because we found in our investigation of the January 7, 2013, incident on the Japan Airlines Boeing 787 that critical assumptions and conclusions made in the safety analyses of GS Yuasa’s and Thales (the suppliers of the components involved in the fire), which were used in Boeing’s electrical power system (EPS) safety assessment, were neither fully delineated nor justified with appropriate data and engineering rationale.

However, multiple independent reviews of the EPS safety assessment that Boeing-authorized representatives and FAA certification engineers had conducted did not reveal these deficiencies. We believe that the review process for safety assessments should be designed to closely examine the data used to support conclusions and challenge assumptions, particularly those that could result in significant safety consequences if incorrect.

We agree that the applicant has an obligation to show traceability, but when the FAA reviews and approves a systems safety assessment, that review should ensure that adequate traceability is present in the documents. Improving the guidance to applicants on the documentation of traceability needed may provide the needed guidance for FAA engineers and designees reviewing systems safety assessments.

However, we believe that you also need to acknowledge the FAA’s role in reviewing for traceability and consider whether the planned revisions to FAA Order 8110.4C and AC 25.1309 (Arsenal draft) described in your letter will provide sufficient guidance for thoroughly reviewing an applicant’s systems safety assessment.

Pending your making the revisions to FAA Order 8110.4C and AC 25.1309 (Arsenal draft) that you described and replying to us about the sufficiency of these documents as discussed above, Safety Recommendation A-14-121 is classified OPEN—ACCEPTABLE RESPONSE. Pending your incorporating guidance into the training programs for FAA certification engineers and engineering designees about how to review an applicant’s systems safety analysis to ensure compliance with the traceability requirements, Safety Recommendation A-14-122 is classified “Open—Acceptable Response.”

Pending revisions to FAA Order 8110.4C and the creation of policy documents related to applicant responsibilities for showing compliance with the order, Safety Recommendation A-14-123 is classified

-From Michael P. Huerta, Administrator: The FAA agrees that traceability between the applicant's showing of compliance and the certification requirements is important. However, the FAA believes that the burden is more appropriately placed on the applicant rather than on the FAA certification engineers and the FAA designees. Placing the burden on the applicant provides them with the assurance that they have addressed all requirements of the regulations, whether or not the FAA reviews the data.

Per FAA Order 8 11 0.4C, Type Certification, the applicant is expected to provide traceability of their substantiation to the certification requirements. Per Order 81 00.4C, the applicant's substantiation "presents and explains the inter-relationship of the evidence in a logical order leading from the requirement to the claim," where "evidence is certification data collected from FAA publications, certification testing, analysis, engineering examinations. similarity, and software design assurance, and any other data deemed acceptable by the FAA standards staffs."

The FAA is currently revising Order 8110.4C and the guidance related to applicant responsibilities. Also, draft AC 25.1309-Arsenal expands on the applicant expectations for traceability between the applicant's showing of compliance and the certification requirements. However, the FAA will consider additional revisions to the draft to discuss the importance of traceability principles.

Although the FAA agrees with the merit of applying traceability of the applicant's showing to specific requirements, the FAA also plans to consider reviewing the guidance for development of special conditions. Additional guidance to the certification offices may be warranted to ensure that the expectations for the applicant are clearly defined.