Dod major range and test facility base

i

5a. CONTRACT NUMBER
5b. GRANT NUMBER
5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S) 5d. PROJECT NUMBER 5e. TASK NUMBER 5f. WORK UNIT NUMBER

11. SPONSOR/MONITOR’S REPORT NUMBER(S)

Prescribed by ANSI Std Z39-18

ii

FOREWORD

This book is one of many technical management educational guides written from a Department of Defense perspective; i.e., non-Service specific. They are intended primarily for use in courses at Defense Acquisition University and secondarily as a desk reference for program and project man-agement personnel. These guidebooks are written for current and potential acquisition management personnel who are familiar with basic terms and definitions employed in program offices. The guide-books are designed to assist government and industry personnel in executing their management re-sponsibilities relative to the acquisition and support of defense systems.

CONTENTS

Page

1.1 Introduction ........................................................................................................... 1-1

1.2 Testing as a Risk Management Tool.................................................................... 1-1

2.2 Defense System Acquisition Process ................................................................... 2-1

2.3 T&E and the Systems Engineering Process (SEP) ............................................. 2-5

3.1 Introduction ........................................................................................................... 3-1

3.2 The Congress ........................................................................................................ 3-1

Chapter 4 — Program Office Responsibilities for Test and Evaluation

4.1 Introduction ........................................................................................................... 4-1

4.5 Integrated Product Teams for Test and Evaluation.............................................. 4-4

4.6 Test Program Funding/Budgeting......................................................................... 4-4

4.11 PMO Responsibilities for Operational Test and Evaluation ............................... 4-6

4.12 Summary.............................................................................................................. 4-10

5.4 Program Management Documentation ................................................................. 5-4

5.5 Test Program Documentation ............................................................................... 5-6

6.2 Development Test and Evaluation ........................................................................ 6-1

6.3 Operational Test and Evaluation........................................................................... 6-3

6.8 Nuclear Hardness and Survivability (NHS) Testing............................................ 6-9

6.9 Summary................................................................................................................ 6-9

7.3 DT&E Responsibilities ......................................................................................... 7-4

7.4 Test Program Integration ...................................................................................... 7-5

7.8 DT&E of Limited Procurement Quantity Programs ........................................... 7-9

7.9 Summary................................................................................................................ 7-9

8.4 Summary................................................................................................................ 8-6

Chapter 9 — Combined and Concurrent Testing

9.5 Summary................................................................................................................ 9-4

Chapter 10 — Production-Related Testing Activities

10.5 Transition to Production ..................................................................................... 10-2

10.6 Low Rate Initial Production (LRIP) .................................................................. 10-4

11.1 Introduction ......................................................................................................... 11-1

11.2 Purpose of OT&E ............................................................................................... 11-1

11.7 Test Reporting ..................................................................................................... 11-5

11.8 Summary.............................................................................................................. 11-7

12.3 System Development and Demonstration (SDD) .............................................. 12-2

12.4 Production and Deployment (P&D) ................................................................... 12-4

13.1 Introduction ......................................................................................................... 13-1

13.2 Difference Between “Test” and “Evaluation” .................................................... 13-1

13.7 Evaluating Development and Operational Tests ................................................ 13-6

13.8 Summary.............................................................................................................. 13-8

14.4 Support to Test Design and Planning ................................................................ 14-4

14.5 Support to Test Execution .................................................................................. 14-7

Chapter 15 — Test Resources

15.1 Introduction ......................................................................................................... 15-1

x

Chapter 16 — Test and Evaluation Master Plan

MODULE V — Specialized Testing

Chapter 17 — Software Systems Testing

17.5 T&E in the Software Life Cycle ....................................................................... 17-4

17.6 Summary.............................................................................................................. 17-8

18.4 Summary.............................................................................................................. 18-7

Chapter 19 — Logistics Infrastructure T&E

19.5 Summary.............................................................................................................. 19-8

Chapter 20 — EC/C4ISR Test and Evaluation

20.5 T&E of Surveillance and Reconnaissance Systems.......................................... 20-6

20.6 Summary.............................................................................................................. 20-7

21.3 Test Program Responsibilities ............................................................................ 21-2

21.4 Test Team Structure ............................................................................................ 21-3

Chapter 22 — International Test and Evaluation Programs

22.1 Introduction ......................................................................................................... 22-1

22.6 Summary.............................................................................................................. 22-4

Chapter 23 — Commercial and Non-Development Items

23.5 Summary.............................................................................................................. 23-5

Chapter 24 — Testing the Special Cases

24.5 Advanced Concept Technology Demonstrations................................................ 24-5

24.6 Summary.............................................................................................................. 24-5

APPENDICES
Appendix A – Acronyms and Their Meanings ................................................................. A-1 Appendix B – DoD Glossary of Test Terminology .......................................................... B-1 Appendix C – Test-Related Data Item Descriptions ......................................................... C-1 Appendix D – Bibliography ............................................................................................... D-1 Appendix E – Index ............................................................................................................E-1 Appendix F – Points of Contact for Service Test and Evaluation Courses ..................... F-1

LIST OF FIGURES
Figure 1-1. The 5000 Model ............................................................................................... 1-2 Figure 1-2. Life Cycle Cost Decision Impact and Expenditures ...................................... 1-3 Figure 2-1. Testing and the Acquisition Process................................................................ 2-2 Requirements to Design ................................................................................... 2-4 Figure 2-2.

Figure 7-2. Design for Testing Procedures ......................................................................... 7-8 Figure 8-1. Relationship of DT&E to the Acquisition Process......................................... 8-1 Figure 8-2. Technical Review Timeline .............................................................................. 8-3 Specifications Summary................................................................................... 8-4 Figure 8-3.

Figure 11-1. Organizational Breakdown of the I-81mm
Mortar Operational Test Directorate ...................................................... 11-6 Figure 12-1. OT&E Related to the Milestone Process ...................................................... 12-2 Figure 12-2. Sources of Data .............................................................................................. 12-3

Figure 14-2. Values of Selected Criteria Conducive to Modeling and Simulation .......... 14-4

Figure 14-3. Modeling and Simulation Application in Test and Evaluation .................... 14-5

Figure 17-3. Spiral Model of AIS Development Process .................................................. 17-5

Figure 18-1. Live Fire Test and Evaluation Planning Guide ............................................. 18-3

Figure 20-4. The Evolutionary Acquisition Process .......................................................... 20-6

Figure 21-1. Simple Multi-Service OT&E Test Team Composition ................................. 21-2

Table 8-1. Technical Reviews and Audits ......................................................................... 8-2

Table 9-1. Combined vs. Concurrent Testing: Advantages and Limitations ................... 9-2

Table 18-1. Relationships Between Key Concepts ........................................................... 18-1

Table 18-2. Types of Live Fire Testing ............................................................................. 18-2

The Test and Evaluation (T&E) process is an

integral part of the Systems Engineering Process 1.2 TESTING AS A RISK MANAGEMENT

Program Manager (PM) must balance the risks been made.

of cost, schedule, and performance to keep the

1-2

• The focus of T&E should be on how to best

support the acquisition process;
• T&E planning with Operational Test (OT) per- sonnel should start early in the acquisition

• Contractor testing, developmental testing, and operational testing have some overlapping functions;

guidance in reducing the risks frequently associ-ated with test programs. The DoD manual Tran-sition from Development to Production contains sample risk management templates.4