Major Capability Acquisition (MCA)
Activities Across Phases
Technical Reviews
Develop Strategies
Program Management
Cost Estimation/Affordability
MDD
MSA PHASE
Develop Requirements
Analysis of Alternatives
Study Contracts
Milestone A
TMRR PHASE
Mature Requirements
Prototype Contracts
Prototyping
Develop Preliminary Design
CDD Validation
Dev RFP Release Decision
Milestone B
EMD PHASE
Development Contracts
Complete System Design
Develop System
Developmental Testing
Milestone C
P&D PHASE
Production Contracts
Low Rate Initial Production
Operational Testing
FRP Decision
Full Rate Production/Deployment
O&S PHASE
Sustainment Contracts
Sustain System
IOC/FOC
Acquisition Categories (ACATs)
*AAF Pathway Resources*
Glossary
Technology Maturity and Risk Reduction (TMRR) Phase
How To Use This Site
Each page in this pathway presents a wealth of curated knowledge from acquisition policies, guides, templates, training, reports, websites, case studies, and other resources. It also provides a framework for functional experts and practitioners across DoD to contribute to the collective knowledge base. This site aggregates official DoD policies, guides, references, and more.
DoD and Service policy is indicated by a BLUE vertical line.
Directly quoted material is preceeded with a link to the Reference Source.
Reference Source: DODI 5000.85 Section 3.8
The TMRR phase is guided by the draft CDD and the acquisition strategy. The purpose of this phase is to reduce technology, engineering, integration and life-cycle cost risk to the point that a decision to contract for EMD can be made with confidence in successful program execution for development, production and sustainment.
This phase includes a mix of activities intended to reduce program risks. These include the design and requirements trades necessary to ensure an affordable product and an executable development and production program. Close collaboration with the requirements community is essential and will inform development and validation of the CDD.
The acquisition strategy will describe the overall approach to acquiring the capability and include the program schedule, risks, funding, business strategy, and an IP strategy. PS and sustainment planning continues during this phase and will include consideration of data rights. Program security and program protection requirements will be evaluated. Unless waived by the MDA, a preliminary design review (PDR) will be conducted prior to Milestone B. This phase normally includes multiple competitive sources conducting technology risk reduction activity to demonstrate new technologies in a relevant environment. An ICE and an ITRA will be conducted for MDAPs, before granting Milestone B approval. Development testing will be guided by the test and evaluation master plan.
During the TMRR phase, the requirements validation authority will validate the CDD (or equivalent requirements document) for the program. This action will precede the Development RFP release decision point.
Systems Engineering in the TMRR Phase
Source: DAG CH 3-3.2.3 Technology Maturation and Risk Reduction Phase
The primary objective of the Technology Maturation and Risk Reduction (TMRR) phase is to reduce technical risk and develop a sufficient understanding of the materiel solution to support sound investment decisions at the pre-Engineering and Manufacturing Development (EMD) Review and at Milestone B regarding whether to initiate a formal acquisition program. The Systems Engineer supports the production of a preliminary system design that achieves a suitable level of system maturity for low-risk entry into EMD. Usually the Systems Engineer implements a strategy of prototyping on a system element or subsystem level, balancing capability needs and design considerations to synthesize system requirements for a preliminary end-item design for the system. The prototyping objectives should focus on risk reduction and/or competition.
The major efforts associated with the TMRR phase are:
- Determining the appropriate set of technologies to integrate into a full system.
- Maturing the technologies, including demonstrating and assessing them in a relevant environment.
- Conducting prototyping of the system and/or system elements.
- Performing trade studies, refine requirements and revise designs.
- Developing the preliminary design, including functional and allocated baselines, specifications, interface control drawings/documents, architectures and system models.
- Performing developmental test activities as appropriate.
SE activities should be integrated with TMRR phase-specific test and evaluation and logistics and sustainment activities identified in CH 8–4.2. and CH 4–3.2., respectively.
During the TMRR phase, the program develops and demonstrates prototype designs to reduce technical risk, validate design approaches, validate cost estimates and refine requirements. In addition, the TMRR phase efforts ensure the level of expertise required to operate and maintain the product is consistent with the force structure. Technology development is an iterative process of maturing technologies and refining user performance parameters to accommodate those technologies that do not sufficiently mature (requirements trades). The Initial Capabilities Document, the Acquisition Strategy (AS), Systems Engineering Plan (SEP) and Capability Development Document (CDD) guide the efforts of this phase. The CDD enters the TMRR phase as a draft and is validated during this phase to support preliminary design activities and the PDR.
There are two key technical objectives in the TMRR phase: technical risk reduction and initial system development activity, culminating in preliminary design. The Systems Engineer in the TMRR phase manages activities to evaluate prototyped solutions (competitive and risk reduction prototypes) against performance, cost and schedule constraints to balance the total system solution space. This information can then be used to inform the finalization of the system performance specification as a basis for functional analysis and preliminary design.
Effective systems engineering (SE), applied in accordance with the SEP and gated by technical reviews, reduces program risk, identifies potential management issues in a timely manner and supports key program decisions. The TMRR phase provides the Program Manager (PM) with a preliminary design and allocated baseline that are realistic and credible.
SE Roles and Responsibilities
Source: DAG CH 3-3.2.3 Technology Maturation and Risk Reduction Phase
The program office team provides technical management and may employ industry, Government laboratories, the Service science and technology (S&T) community or Federally Funded Research and Development Centers (FFRDCs)/universities to accomplish specific risk-reduction or prototype tasks as described in the SEP.
In addition to the general responsibilities identified in CH 3–2.5. Engineering Resources, the PM focuses on the following TMRR activities, which rely on and support SE efforts:
- Awarding TMRR phase contract(s).
- Providing resources for technical reviews.
- Planning and executing the Technology Readiness Assessment (TRA) (MDAPS only).
- Influencing development of the CDD.
- Developing the Acquisition Strategy (AS).
- Developing the strategy and objectives for use of prototypes; considering both contracted efforts and government sources.
- Supporting the Development RFP Release Decision Point.
- Ensuring the Government preserves the rights needed to be consistent with the life-cycle acquisition and support strategy. During TMRR, proprietary development and design can often lead to issues with intellectual property and associated data rights (see CH 3–4.1.7. Technical Data Management Process).
- Supporting the Configuration Steering Board once the CDD has been validated. This board assumes responsibility to review all requirements changes and any significant technical configuration changes for ACAT I and IA programs in development, production and sustainment that have the potential to result in cost and schedule impacts to the program.
In addition to the general roles and responsibilities described in CH 3–2.5. Engineering Resources, during this phase it is the Systems Engineer’s responsibility to:
- Lead and manage the execution of the technical activities as documented in the SEP.
- Plan and execute technical reviews, including the System Requirements Review (SRR), System Functional Review (SFR), and Preliminary Design Review (PDR)
- Measure and track program maturity using technical performance measures, requirements stability and integrated schedules.
- Support award of TMRR phase contract(s), as necessary.
- Balance and integrate key design considerations.
- Maintain the Systems Engineering Plan (SEP), including generating the update in support of Milestone B.
- Lead the initial development of the system to include functional analysis, definition of the functional and allocated baselines and preliminary design (see CH 3–4.2.2. Requirements Analysis Process and CH 3–4.2.3. Architecture Design Process).
- Support configuration management of the baselines, since they are required in later technical reviews, audits and test activities (e.g., functional baseline at the Functional Configuration Audits (FCAs)).
- Conduct technical activities in support of the Development RFP Release Decision Point.
- Conduct a rigorous and persistent assessment of technical risk, determine risk mitigation plans and work with the PM to resource the mitigation plans.
- Support the Technology Readiness Assessment (TRA) including creation of the plan, the pre-EMD preliminary TRA and the TRA final report (MDAPs only).
- Support requirements management, and monitor for unnecessary requirements growth (e.g., derived versus implied requirements).
- Manage interfaces and dependencies.
- Maintain oversight of the system (software and hardware) development processes, system testing, documentation updates and tracking of the system development efforts.
- Support the PM in his interactions with the Configuration Steering Board.
SE Inputs for TMRR Phase
Source: DAG CH 3-3.2.3 Technology Maturation and Risk Reduction Phase
Inputs for TMRR Phase |
DoD Component combat developer (e.g., Requirements Manager) provides:
|
Analysis of Alternatives (AoA) Report and AoA Sufficiency Report (See CH 2–2.3.) |
Preferred materiel solution
|
Acquisition Decision Memorandum (ADM) (may contain additional direction) |
SEP (See CH 3–2.2. Systems Engineering Plan) |
Reliability, Availability, Maintainability and Cost Rationale (RAM-C) Report (See CH 3–4.3.19.)
|
Reliability Growth Curves (RGC) (See CH 3–4.3.19.)
|
Program Protection Plan (PPP) (See CH 9–3.4.2.2.) |
Trade-off analysis results
|
Assumptions and constraints
|
Environment, safety and occupational health (ESOH) planning (See CH 3–4.3.9.) |
Risk assessment (See CH 3–4.1.5.)
|
Consideration of technology issues |
Initial identification of critical technologies
|
Interdependencies/interfaces/memoranda of agreements (MOAs) |
Life-Cycle Mission Data Plan for Intelligence Mission Data (IMD)-dependent programs (See CH 3–4.3.12. Intelligence (Life-Cycle Mission Data Plan) and CH 7–4.1.3.) |
Draft system performance specification |
Other technical information generated during the MSA phase
|
Prototyping strategy
|
Validated On-Line Life-cycle Threat (VOLT) Report (See CH 7–4.1.2.) |
Affordability Assessment (See CH 1–4.2.15. and CH 3–4.3.2.)
|
AS (See CH 1–4.1.) |
Life-Cycle Sustainment Plan (LCSP) (See CH 4–3.1.) |
Test and Evaluation Master Plan (TEMP) (See CH 8–4.1.) |
Informed advice to the developmental test and evaluation (DT&E) assessments (See CH 8–4.1.)
|
Draft and final Request for Proposal (RFP) |
Security Classification Guide (SCG) |
Other analyses
|
Spectrum Supportability Risk Assessment (See DoDI 4650.01 and CH 3–4.3.20.) |
SE Activities in the TMRR Phase
Source: DAG CH 3-3.2.3 Technology Maturation and Risk Reduction Phase
The TMRR phase activities begin when a favorable Milestone A decision has been made (see CH 3–3.2.2. Materiel Solution Analysis Phase) and end with a successful Milestone B decision.
The TMRR phase addresses a set of critical activities leading to the decision to establish a program of record. The SE activities are aimed at reducing technical risk and providing the technical foundation for this decision. Depending on the nature of the technology development strategy, the order and characteristics of these activities may change. During the TMRR phase, systems engineers follow comprehensive, iterative processes to accomplish the following:
- Perform Technology Maturation. The AS identifies technologies requiring further maturation before they can be implemented within a solution. Technology maturation involves design, development, integration and testing. There could be one or more risk areas related to hardware, software or information technology, and there may be multiple industry contracts/Government efforts for maturing the technology. The TEMP should stipulate the test and evaluation approach for assessing the results of the technology maturation activities (see CH 8–4.2.). The Systems Engineer participates in the technology readiness assessment (TRA). The TRA focuses only on technology maturity as opposed to engineering and integration risk. OSD TRA Guidance provides policy and guidance for TRAs.
- Perform Prototyping. Prototyping is an engineering technique employed for several reasons: to reduce risk, inform requirements and encourage competition. For example, the primary objective for competitive prototyping (CP) is acquiring more innovative solutions at better value by ensuring competition. CP are addressed in statute for MDAPs (see P.L. 114-92 (SEC. 822 para (c))). Other prototypes should be considered if they materially reduce engineering and manufacturing development risk at an acceptable cost. At this point in the life cycle, the CP strategy should focus on mitigating key technical risk areas. The program office should have a clear understanding of technical, engineering and integration risks at Milestone A. Current policy does not require full-up system prototypes; therefore, competitive prototyping may include prototyping critical technologies, system elements, integration of system elements or full-up prototypes. Because a primary objective of this type of prototyping is to support a follow-on award choice between developers, contract incentives should be aligned to CP strategy goals. These goals most often emphasize cost, schedule and performance realism and quantification. Contract goals should require the solutions demonstrated during CP be used in the subsequent PDR/CDR designs. The CP strategy should be identified in the SEP and AS, tasks specified in RFPs/Task Orders, technically managed by the program office and included in the TEMP with specific test objectives. Risk reduction prototypes can be at the system level or can focus on technologies, sub-components or components, and may or may not include objectives associated with competitive contracts. And in nearly all cases, prototypes can be extremely useful in assessing technical performance, supporting trade studies and updating requirements.
- Perform System Trade Analysis. The Systems Engineer assesses alternatives with respect to performance, cost, schedule and risk, and makes a recommendation to the PM. The SE assessment should consider the full range of relevant factors, for example, affordability goals and caps, technology maturity, development and deployment constraints, modular open system approaches and user-identified needs and shortfalls. System trades should be used to inform and shape the CDD and cost and schedule objectives to be documented in the Acquisition Program Baseline (APB).
- Develop System Architecture. See CH 3–4.2.3. Architecture Design Process for additional information.
- Develop Functional Baseline. See CH 3–4.1.6. Configuration Management Process for additional information.
- Develop Allocated Baseline. See CH 3–4.1.6. Configuration Management Process for additional information.
- Develop Preliminary Design(s). May involve competitive, preliminary design activities up to and including PDRs. See CH 3–3.3.4. Preliminary Design Review for additional information.
- Develop Allocated Technical Performance Measures (TPMs). The allocated baseline establishes the first physical representation of the system as system elements with system-level capabilities allocated to system element-level technical performance measures.
- Support CDD Validation. The purpose of this support is to inform the MDA and requirements validation authority about the technical feasibility, affordability and testability of the proposed requirements. CDD (or an equivalent requirements document) forms a basis for the set of requirements used for design activities, development and production. Specific SE attention is given to trade-off analysis, showing how cost varies as a function of system requirements (including Key Performance Parameters), major design parameters and schedule. The results should identify major affordability drivers.
- Support Development RFP Release Decision Point. The purpose of the MDA-level review is to assess the AS, RFP and key related planning documents and determine whether program plans are affordable and executable and reflect sound business arrangements. Specific SE attention is given to engineering trades and their relationship to program requirements and risk management. Typically, this event occurs after PDR to allow for feedback from the PDR into the technical aspects of the RFP. The Development RFP Release event can come before the PDR if there is confidence the RFP will not need to be substantially changed.
- Finalize Documents. The Systems Engineer updates the SEP and PPP and provides inputs for updating the LCSP, TEMP and other program documents.
The Systems Engineer uses technical reviews and audits to assess whether preplanned technical maturity points are reached during the acquisition life cycle as the system and system elements mature. A key method for doing this is to identify technical risks associated with achieving entrance criteria at each of these points (See the DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs.) Technical reviews typically conducted in the TMRR phase are:
- System Requirements Review (SRR) (see CH 3–3.3.2. System Requirements Review)
- System Functional Review (SFR) (see CH 3–3.3.3. System Functional Review)
- Software Specification Review (SSR) for programs with significant software development; a SSR is typically performed before, and in support of, a PDR. The SSR technical assessment establishes the software requirements baseline for the system elements under review (e.g., computer software configuration items (CSCI)) to ensure their preliminary design and, ultimately, the software solution has a reasonable expectation of being operationally effective and suitable.
- Preliminary Design Review (PDR) mandated (unless formally waived) to confirm the development of the allocated baseline (see CH 3–3.3.4. Preliminary Design Review)
Test activities during the TMRR phase that depend on SE support and involvement include developmental test and evaluation of system and/or system element prototypes and Early Operational Assessments (EOAs). Developmental Test and Evaluation (DT&E) activities, for example, should be closely coordinated between the engineering and test communities, since DT&E activities support:
- Technical risk identification, risk assessment and risk mitigation
- Providing empirical data to validate models and simulations and
- Assessing technical performance and system maturity (see CH 8–4.2.)
Test and Evaluation in the TMRR Phase
Reference Source: DAG CH 8-4.2 Technology Maturation & Risk Reduction Phase
The Acquisition Strategy and the Milestone A TEMP guide this acquisition phase. The Chief Developmental Tester, in collaboration with the T&E WIPT, monitors the execution of the T&E events and reviews T&E reports, . Multiple technology development demonstrations/ evaluations, defined in the Acquisition Strategy (AS) and the TEMP, may prove necessary before the user and developer can substantiate a preferred solution is feasible, affordable, and supportable; satisfies validated capability requirements; and has acceptable technical risk. Programs identify critical program information during this phase as well as implement program protection measures to prevent disclosure of critical information.
Contractor DT&E. During early DT&E, the contractor approach includes tests and evaluations for optimizing designs and functionalities. The contractor may also use modeling and simulation, laboratory, test bench, and system mock-ups or prototypes to gain knowledge of integrated system performance. Government T&E organizations may observe the critical contractor testing, conduct additional T&E, and, when practical, facilitate early user involvement. The government may be able to utilize contractor data to enhance or replace planned government testing, and to enable T&E efficiencies to be realized if testing is observed by a government T&E organization. The TMRR contract with industry should support open communication between government and contractor test organizations.
Government DT&E. During early DT&E, the Chief Developmental Tester uses government testing to evaluate competitive prototypes, competing technologies, technology maturity of critical technology elements, etc. The Lead DT&E Organization conducts developmental testing and evaluation activities for the program, as directed by the Chief Developmental Tester. The Lead DT&E Organization also ensures the AoA assumptions and findings are validated.
Early Operational Assessments. Early Operational Assessments (EOAs) provide a means to evaluate a program’s progress early in the process towards developing an operationally effective, suitable, and survivable system. EOAs are typically an analysis, based on a review of current program plans and documentation, as well as data from early developmental testing, technology assessments, modeling and simulation, and program reviews. EOAs enable the OTA to provide early input on key issues that, if not corrected, could have a detrimental effect to the determination of operational effectiveness, operational suitability, and survivability (including cybersecurity) or lethality. EOAs provide a means to examine the links and consistency between the concept of operations, requirements, and technology limitations to provide recommendations to the program and the requirements authority.
EOA reports are provided to support one or more of the design phase life-cycle events (namely, the CDD Validation Decision, the Development RFP Release Decision, or Milestone B). For programs entering development at Milestone B, the lead OTA (as appropriate) prepares and reports EOA results after program initiation and prior to the Critical Design Review (CDR).
T&E Support during Technology Maturation & Risk Reduction
The TMRR phase includes activities intended to reduce the specific risks associated with the functions, technologies, environments, and developed products. This includes additional design trades and requirements necessary to ensure an affordable product and an executable development, production, and sustainment program.
Logistics Risk Assessment. During the TMRR phase, programs conduct a logistics risk assessment as part of life-cycle considerations. The PM finalizes sustainment requirements for approval at the CDD Validation Decision, and decomposes sustainment requirements into more detailed requirements to support the Preliminary Design Review (PDR) and for use during the logistics risk assessment. The T&E WIPT leverages the logistics risk assessment during development of the Milestone B TEMP. Refer to the DAG, CH 4, Life Cycle Logistics for more information.
Technology Readiness Assessment (TRA). The TRA is a systemic, metric-based process that establishes the maturity of critical technologies. The TRA may be conducted concurrently with other technical reviews. The Chief Developmental Tester assists the chief engineer/lead systems engineer when assessing the technological maturity and integration risk of critical technologies. Refer to the DAG, CH 3.4.1.3., Technical Assessment Process, for more information.
Preliminary Design Review (PDR). Conducted after preliminary design efforts, but before the start of detail design, the PDR provides the first opportunity for the DoD to closely observe the contractor’s hardware and software design. The contractor describes all design changes made with respect to trade studies, design considerations, and design decisions that provide a rationale for the system’s preliminary design. The contractor also provides a hardware or hands-on demonstration of some of the preliminary designs to better illustrate important aspects. The Chief Developmental Tester provides developmental test data collected to date to support the PDR. Test organizations attend technical reviews to provide current assessments, keep abreast of program progress, and provide insight into design direction. Unless waived by the MDA, the Preliminary Design Review occurs prior to Milestone B. Refer to the Preliminary Design Review, for more information.
Capability Development Document (CDD) Validation. The Technology Maturation and Risk Reduction (TMRR) Phase requires continuous and close collaboration between the program office and the requirements validation authority During this phase, the Requirements Authority for the program validates the final CDD in order to provide a basis for preliminary design activities and the Preliminary Design Review, which normally occurs prior to Milestone B unless waived by the MDA. Prior to validation, the program coordinates the Capability Development Document (or other draft requirements document) with the MDA to ensure requirements remain technically achievable and affordable. The T&E WIPT reviews the draft CDD and coordinates the input with the PM. This effort provides the T&E WIPT a key opportunity to review requirements, to determine if they are clear, testable, measurable, and technically achievable.
Risk Management for TMRR Phase
Reference Source: DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs, DASD/SE, Jan 2017
If a TMRR phase is necessary, it should focus on reducing risks in technology, engineering, integration, and life cycle cost to the point that the Milestone Decision Authority (MDA) can make an EMD decision with confidence that the cost and schedule objectives carry understood and manageable risk. If the requirements community has clear and stable requirements and the supporting technology is mature, it may be possible to skip this phase and go directly to EMD or beyond.
Key risk areas include system performance and affordability. The PM decides what risk reduction activities to conduct in the TMRR phase but should prioritize starting with elements that represent the highest risk that can be reduced during this period of lower financial commitment. The PM should consider including special contract incentives for the high-risk areas. Typically, these activities include risk-reduction prototyping (which may be competitive) of the system, critical subsystems, technology, sub-component, or component level. Prototyping of immature technologies can help inform decisions on how and whether to proceed. Another TMRR phase risk reduction activity is to identify and assess the materials and manufacturing processes the program will require.
TMRR phase activities include the following Systems Engineering Technical Reviews (SETR) to assess and manage risk: System Requirements Review (SRR), System Functional Review (SFR), and Preliminary Design Review (PDR). Throughout the TMRR phase, the program team should conduct a rigorous assessment of technical risk, develop risk mitigation options, and execute and monitor risk mitigation plans.
Suggested Activities and Practices in the TMRR Phase to Reduce Risk
- Assess TMRR risks mitigation effectiveness, and evidence that the program has demonstrated critical technologies in a relevant environment and end-item design context.
- Develop an EMD schedule that includes time for integration, interdependency linkages, and mitigation of manufacturing risks.
- Conduct system or subsystem risk reduction prototyping validation and competition.
- Assess the preliminary design and allocated baseline to identify problematic requirements and risks to meeting operational requirements, technical achievability, and cost/affordability targets. Ensure derived requirements do not contribute to requirements creep.
- Conduct systems engineering trade-off and preliminary design activities to support the assessment of final requirements in the CDD.
- Incorporate decisions from Configuration Steering Board (CSB) meetings and/or knowledge point review (includes requirements and intelligence communities).
- Track program interdependencies, interfaces, and associated memorandums of agreement (MOA) in periodic meetings with external programs and associated contractors/stakeholders.
- Verify validity of program framing assumptions.
- Plan for contingencies and technical risk mitigation activities by establishing reasoned cost, schedule, and performance margins.
- Ensure risk mitigation plans are reflected in the Integrated Master Plan (IMP), IMS, Technical Performance Measures (TPM), and the EVM baseline. This may or may not require a change to the contractor work packages or resources. (Continue in subsequent phases.)
- Identify resourced off-ramps for any critical technologies in the IMS.
- Avoid allowing the urgency of the schedule to outweigh good engineering and management.
- Conduct a government Technology Readiness Assessment (TRA) and risk assessment early in the TMRR phase. Ensure prototyping activities are relevant to the planned end item design and include plans to demonstrate technologies that present uncertainty.
- Identify applicable commercial technologies and develop an integration plan.
- Consider directed options (directed subcontractors) as opposed to industry teaming.
- Enable early evaluation of risks by planning an effective developmental test and evaluation program with adequate test articles and schedule duration for regression testing.
- Conduct a government TRA before MS B to identify and assess critical technology elements in the contractor’s EMD proposal.
- Conduct schedule risk analyses (SRA) on a regular basis to evaluate the likelihood to achieve the planned schedule.
- For trade studies affecting KPP/KSAs, develop a decision hierarchy to promptly identify and mitigate technical risks and their impact on cost, schedule, and performance.
- Consider S&T investments to support EMD and beyond.
- Develop MOAs with all external interdependent programs.
Sustainment Planning in TMRR Phase
Source: DAG CH 4-3.2 Technology Maturation and Risk Reduction Phase
Planning for sustainment ramps up during the Technology Maturation and Risk Reduction (TMRR) phase. The PM evaluates the materiel solution for operational suitability, ability to meet KPPs and KSAs, and life cycle affordability. The PM defines risks and opportunities to sustainment and refines the sustainment strategy and requirements.
- Product Support Strategy
- Supportability design requirements to influence specifications RFP
- Intellectual Property Strategy
- Metrics – Goals and Thresholds:
- System and Subsystem Level
- Supply Chain
- Sustainment Metrics
- Test Strategy for System Suitability and Supportability
- Supportability Analysis (Business Case Analysis, Failure Modes and Effects Criticality Analysis, Reliability Centered Maintenance, Level of Repair Analysis, etc.)
- Affordability Cap
- O&S Cost Estimating – Assumptions, Resources, Estimates
- Core Workload and Estimates
The PM revises the sustainment strategy during TMRR by conducting analyses that help identify risks and opportunities in O&S Cost, maintainability, availability, reliability, and readiness.
Reference Source: DAG CH 4-3.1.4 Design Interface
Design Interface
The design phase can best influence sustainment outcomes, as decisions made during this time can make it easier to maintain and reduce O&S Costs. The systems engineering process relies on balancing often conflicting requirements such as weight versus reliability. The design decisions rely on thorough trade studies that can accurately and completely provide the life cycle cost impact for each alternative being assessed. It is difficult and expensive to redesign to restore reliability or maintainability; close
interaction between the sustainment, engineering, and design communities throughout the design phase delivers required maintainability and reliability.
The PM should formulate design requirements to minimize support equipment, including testing, measurement, and diagnostic equipment. When the use of support equipment cannot be eliminated, the PM should standardize support equipment design for the broadest possible range of applications, consistent with maintenance concepts.
The PM should also consider the use of Condition Based Maintenance Plus when selecting maintenance concepts, technologies, and processes for all new weapon systems, equipment, and materiel programs. Readiness requirements, life cycle cost goals, and Reliability-Centered Maintenance (RCM) based functional analysis should be formulated in a comprehensive reliability and maintainability (R&M) engineering program.
Reliability, Availability, Maintainability, and Cost Rationale Report
The Reliability, Availability, Maintainability, and Cost (RAM-C) Rationale Report provides a quantitative basis for reliability requirements and improves cost estimates and program planning. The PM participates in the RAM-C analysis, along with the R&M engineer and cost analyst, to ensure that the Sustainment KPPs and KSAs are valid and feasible and that appropriate trade studies have been conducted to illustrate the trade space between the sustainment parameters. This report is used throughout the system’s life cycle as a baseline for how requirements are measured and tested. For more, see the RAM- C Rationale Report Manual and CH 3 Section 4.3.19 Table 48 R&M Activities by Acquisition Phase.
Reference Source: DAG CH 4-3.2.1.1.4 Core Workload and Estimate
If the core determination results in required organic depot support, the PM works with DoD Component- level organizations to determine what workload will be maintained in the depot (e.g., engine, airframe, chassis). Once the type of work to be completed in the depot is determined, the PM develops an estimate for that core depot workload. The core depot workload assessment is measured projected man-hours.
Because specific design details may not be known prior to a PDR, the PM may need to estimate the core workload man-hours based on data from legacy or analogous systems.
Reference Source: DAG CH 4-3.2.1.1.5 Life Cycle Sustainment Plan Reviews
Life Cycle Sustainment Plan Reviews
During the TMRR phase, the PM updates the LCSP to refine the activities that occur during subsequent phases of the program. The PM ensures that sufficient sustainment planning is in place to influence the RFP. Depending on the Acquisition Category level, the PM holds LCSP review meetings with key stakeholders, including sustainment commands; Warfighter organizations; Office of the Under Secretary of Defense for Acquisition, Technology and Logistics; and DoD Component oversight organizations to discuss the Product Support Strategy. Potential discussion points:
- Identification of LCSP annexes to include and rationale for excluded annexes.
- Sustainment metrics (requirements) and their incorporation into the draft RFP.
- O&S Cost & Affordability Goals/Caps.
- Portfolio level affordability.
- Planned core applicability or requirements.
- Planning for IP and data rights strategy.
- Legacy system versus future program cost drivers.
- Should Cost initiatives.
- Developmental testing/operational testing integration with the Sustainment Plan.
- Transportability testing timeline.
- Impact of final source selection (before or after Milestone B).
Maintenance Plan
As part of the revision of the sustainment strategy during TMRR, the PM updates the maintenance plan based on design knowledge obtained during the TMRR phase and input from industry. Considerations may include whether the proposed material solutions are commercially available off-the-shelf (COTS) or government off-the-shelf (GOTS), require development of new repair capabilities, are deemed not economical to repair, or require intermediate/off-equipment field maintenance.
Software Sustainment
The PM also considers test and integration needs, transition of operational software and support tools from the developer to the post-deployment support organizations, help-desk requirements, and safety critical requirements.
Reference Source: DAG CH 4-3.2.1.3.2 Affordability
Affordability
The O&S Cost affordability goals documented in the Acquisition Decision Memorandum at Milestone A guide the cost and engineering trade-offs during TMRR. As the design matures, the PM ensures that O&S Cost affordability remains a factor in engineering and sustainment trades. Cost estimating during the TMRR phase supports the Development RFP Release and Milestone B decision through the update of the LCSP and the LCCE.
Reference Source: DAG CH 4-3.2.3.4 Sustainment Planning
Sustainment Planning
As early as the TMRR phase, the PM can use the LCSP to develop RFPs that provide potential vendors with sustainment requirements and insight into the operational and maintenance environments in which the materiel solution must perform. While preparing for the Milestone B source selection process, the PM considers the following evaluation criteria:
- Reliability risk based on each vendor’s capability to meet potential reliability metrics and the threshold hours/miles.
- Successful transportability testing prior to source selection.
- O&S Cost estimates. The government could chose to provide an O&S Cost model/formula and assumptions of CONOPS and usage or could request specific data from each bidder independently to evaluate O&S Costs. The PM ensures that the O&S Cost evaluation include total government costs to operate and support the system, not just contractor costs.
The PM uses the LCSP to develop RFP requirements that consider sustainment and reliability outcomes during the Pre-Milestone B RFP process:
- The System Performance Specifications support the RFP and clearly identify sustainment and reliability requirements.
- The Program Office evaluates achieving reliability requirements as a condition for source selection.
Evaluate the system reliability models and predictions that support the specification requirements. Including reliability metrics enables evaluation of offeror’s approach and expected R&M performance. For example, define a specified mileage/hour mean time between operational mission failure through a combination of reliability growth testing and operational assessment per approved program reliability growth curve.
As early as the TMRR phase, the program can address O&S Cost management through a series of CDRL requirements. The Program Office could use reports required in the RFP and SOW to track part consumption trends, cost drivers, and failure causes to improve training, redesign when necessary, increase reliability, and decrease O&S Cost. The PM could use these and other reports called for in the RFP and SOW to track part consumption trends, cost drivers, and failure causes to improve training, redesign when necessary, increase reliability, and decrease O&S Cost.
The Development RFP provides an opportunity for implementing Should Cost initiatives by setting requirements for addressing system deficiencies and risks. For example, if the engineers and cost estimators identified repair time as an O&S Cost driver, the PM may develop a Should Cost initiative to reduce repair time. The Development RFP may include language that provides an incentive for the bidder to design the system in a way that reduces repair time.
Reference Source: DAG CH 4-3.2.4.1.1 Intellectual Property Strategy
Intellectual Property Strategy
By Milestone A, the PM will have developed an Intellectual Property (IP) Strategy that includes planning for the acquisition and delivery of data that will be required to execute the sustainment strategy. Planning for the IP Strategy should begin by the TMRR phase, although it is not required to be included in the LCSP until the FRP decision review (DR). While preparing for the source selection at Milestone B, the PM addresses:
- The cost of the Technical Data Package (TDP), both for the entire package and broken out by component/sub-component. TDP is an important selection criterion, as acquiring data rights may be needed to support future competition.
- Regardless of the government’s decision to purchase the TDP, the Program Office normally includes an Operation, Maintenance, Installation, and Training (OMIT) clause that obligates the winning vendor to provide necessary repair instructions for government purposes (i.e., establishment of organic repair capabilities).
- Technical manual, national maintenance work requirements, depot maintenance work requirements, and troubleshooting and repair procedures could also be included in Integrated Product Support (IPS) Contract Line Item Numbers.