CHAPTER 6
SYSTEM TEST, EVALUATION, AND VALIDATION
1) Specific test and evaluation requirements are first identified during the conceptual design
phase when system operational requirements, the maintenance and support concept, and the
2) Refer to Figure 6.2 (page 153). The “process,” reflected in the figure, is established to
acquire the necessary confidence, as one proceeds through the system design and
5) For a new system operating within the context of a SOS configuration, the same overall
testing approach, as shown in Figure 6.2 (page 153), applies. Initial Analytical Evaluation
(simulation) and Type 1 testing are accomplished as with any new system requirement
(Sections 6.2.1 and 6.2.2, page 154. In Type 2 testing, the objective is to simulate (to the
6) Planning for system test and evaluation begins in the conceptual design phase when the
requirements for the system are first defined and are being specified (i.e., operational
8) The proper level of logistic support is essential for the successful evaluation of a system
and its elements. Further, the nature and configuration of the maintenance and support
infrastructure utilized must represent that which will be available later on when the system
is in operational use by the customer, particularly with a system operating within a SOS
configuration.
availability (Ao) is a function of system “uptime” and “downtime,” or MTBM and MDT,
or the percentage of time that the system is “operational” when required. Thus, the data
collected must enable the assessment of the system in terms of its mission, its time of
operation, and any downtime that is expended in the accomplishment of maintenance. Life–
cycle cost (LCC) includes all future costs associated with research and development,
design, test and evaluation, production and/or construction, system utilization, maintenance
and support, and system retirement, material recycling and disposal. Thus, the data
collected must include ALL costs, cost projections and profiles, the appropriate reliability
and maintainability data, logistics data, and so on. Reliability (R, MTBF) is a function of
system operating time (or operating cycles), system failures and failure rates, the actual
causes of failures, and so on. The data collected must cover system operations, system
failures, and the requirements associated with the repair actions taken in response to
failures. Reference: Section 6.5 (page 162) and Figure 6.4 (page 164).
12) Evaluating the adequacy of the system’s supply chain can effectively commence as part of
Type 2 testing (Section 6.2.3, Item 10, page 155). Specific candidates for evaluation may
13) Sustainability pertains to the system’s overall impact on the environment, with the
objective of not causing any degradation to the environment while the system is being
operated and maintained in the field by the user. As the system (and elements thereof)
14) If a “non–compliance” situation should evolve as a result of evaluation, the causes for such
need to be identified and a recommendation for corrective action must be initiated; i.e., an
Rights and Permissions Department, Pearson Education, Inc., Upper Saddle River, NJ 07458.
42
18) Refer to Figure 6.2 (page 153). System evaluation during conceptual design and early in the
43
19) Good test reporting is essential if one is to properly “assess” the system in terms of whether
20) Inherent within the systems engineering process are (a) the initial establishment of system–
level requirements; (b) the iterative process of synthesis, analysis, and evaluation
