AMS Lifecycle Phase: Solution Implementation

Description

Product Teams should conduct routine tests of their systems and verify that their systems are properly configured with the appropriate security mechanisms and policies. These routine tests help prevent many types of incidents from happening in the first place. Security testing is the best way to determine if the system configured to the correct security controls and policies.

The Operational Test and Evaluation (OT&E) demonstrates that the system is operationally effective and operationally suitable for use in the NAS. This includes its ability to protect itself and the NAS from security incidents. Integrating security testing with OT&E is a step toward ascertaining whether the system is operated according to its security requirements (both operational and technical controls). The test focuses on demonstrating that operational requirements, including security, have been met and that all critical operational issues have been resolved. This test is typically conducted at the Technical Center, before the system is placed in the field. The objectives of this test are to uncover design, implementation, and operational flaws that could allow the violation of the Basic Security Policy, determine the adequacy of security mechanisms, assurances and other properties to enforce the security policy, assess the degree of consistency between the system documentation and its implementation.

Testing the security controls during Site Acceptance Testing (SAT) allows the product team to evaluate how those security controls work at the site and make any adjustments for "real-world" situations that may not have been present during testing in a lab. This testing includes both the actions of people who operate or use the system and the functioning of the technical controls. This testing should also be conducted after the system has undergone major upgrades to be sure it is still configured to the appropriate security mechanisms and security policies.

The results of the security testing should be documented and submitted as part of the SCAP. OT&E security test report should document that testing that took place and any vulnerability that were discovered. This can be used to develop a mitigation schedule and plan. Security test results should be made available for staff as a reference point for defining mitigation activities, and to assess the implementation status of system security requirements. The results can also enhance risk assessments and performance improvement efforts, as well a benchmark for tracing an organization's progress in meeting the security requirements.

Tasks

• Review the Information Systems Security Program Implementation Guide for latest SCAP template
• Develop Security Test Plan
• Develop Security Test Procedures
• Run test along with SAT and OT&E
• Develop Security Test Report
• Develop Mitigation Plan and Schedule for completion of mitigation tasks

Resources

• NIST Special Publication 800-42, Guidelines on Network Security Testing
• NIST Special Publication 800-64, Security Considerations in the Information System Development Life Cycle
• FAA's ISS Handbook