Major Capability Acquisition (MCA)

AAF  >  MCA  >  Prototyping

Prototyping

How to use this site

Each page in this pathway presents a wealth of curated knowledge from acquisition policies, guides, templates, training, reports, websites, case studies, and other resources. It also provides a framework for functional experts and practitioners across DoD to contribute to the collective knowledge base. This site aggregates official DoD policies, guides, references, and more.

DoD and Service policy is indicated by a BLUE vertical line.

Directly quoted material is preceeded with a link to the Reference Source.

Prototype: A model (e.g. physical, digital, conceptual, and analytical)
built to evaluate and inform its feasibility or usefulness.

DoD Prototyping Handbook

Reference Source: 10 U.S.C. 2431b 

Preference for Prototyping.—To the maximum extent practicable and consistent with the economical use of available financial resources, the milestone decision authority for each major defense acquisition program shall ensure that the acquisition strategy for the program provides for—

 

(1) the production of competitive prototypes at the system or subsystem level before Milestone B approval; or
(2) if the production of competitive prototypes is not practicable, the production of single prototypes at the system or subsystem level.

 

Reference Source: DODI 5000.85, Section 3C.a(4)

(a) The acquisition strategy will address how program management will create and sustain a competitive environment, from program inception through sustainment. Program management should use competition at various levels to create competitive environments that encourage improved performance and cost control. Decisions made in the early phases of the acquisition process can either improve or reduce program management’s ability to maintain a competitive environment throughout the program life cycle.

(b) Strategies to be considered include: competitive prototyping, dual sourcing, and a modular open systems approach that enables competition for upgrades, acquisition of complete technical data packages, and competition at the subsystem level. This also includes providing opportunities for small business and organizations employing those with disabilities.

 

Reference Source: DODI 5000.85, Section 3C.3.d.(1)

PMs are responsible for prioritizing programmatic risks and mitigating them within program constraints. Most of program management is about the process of eliminating programmatic risk over the life of the program. Formal risk management is one tool to accomplish that objective. Top program risks and associated risk mitigation plans will be detailed in the program acquisition strategy and presented at all relevant decision points and milestones. The PM will consider the following risk management techniques:

  • Prototyping at the system, subsystem, or component level; and competitive prototyping, where appropriate.
  • Modeling and simulation, to include the need for development of any new modeling and simulation tools to support a comprehensive risk management and mitigation approach.
  • Technology demonstrations and decision points to discipline the insertion of planned technologies into programs or the selection of alternative technologies provide additional discussions of technical management activities.
  • Intelligence analyses, data dependencies, and threat projections.
  • Multiple design approaches.
  • Alternative designs, including designs that meet requirements but with reduced performance.
  • Phasing program activities or related technology development efforts to address high-risk areas early.
  • Manufacturability.
  • Industrial base availability and capabilities (further discussed in Paragraph 3C.5.).
  • Analysis or detailed identification of sub-tiers in the prime contractor supply chain.
  • Independent risk assessments by outside subject matter experts.
  • Providing schedule and funding margins for identified risks.

 

The Office of the Under Secretary of Defense for Research and Engineering OUSD(R&E), published a revised DoD Prototyping Handbook in Nov 2019. Below are a few key excerpts:

Benefits of Prototyping

Reference Source: DoD Prototyping Handbook Section 4.2

  • Reduce technical risk
  • Validate designs and feasibility of design concepts
  • Obtain early feedback from warfighter
  • Refine performance requirements
  • Identify cost drivers and obtain information on potential costs
  • Investigate integration challenges
  • Identify potential reliability and sustainability issues
  • Validate new ways of using fielded equipment and technologies
  • Validate changes to how systems are employed
  • Verify analytical models

 

Rapid Learning. Prototyping can enhance rapid learning through the use of the “test-analyze-fix-test” (TAFT) approach to capability development. Using this approach, prototypes undergo repetitive iterations of the TAFT process as long as funding and schedule permit or until the desired performance is achieved or the purpose is realized. This approach helps reveal problems early and enables developers to evaluate the modifications they make to mitigate the problems.

Accelerated Demonstration. Prototyping can be used to demonstrate the value of new concepts, technologies, components, systems, and applications earlier in the technology development process than would have been possible if the final development article was used for testing.

Rapid Delivery of Capability to the Field. Prototyping can also be used to develop and demonstrate solutions to existing and emerging operational capability gaps. When these prototypes are deemed viable solutions, they are often left in the field to be used by operators as solutions to their pressing needs.

Fail Fast, Fail Cheap to Learn Fast and Save Money. “Fail Fast, Fail Cheap” is a term of art that the prototyping community uses to describe the great value of prototyping. Dr. Griffin echoed that philosophy in his April 18, 2018 statement to Congress. Rather than avoiding failure, Dr. Griffin encourages the Department to adopt a “willingness to learn from failure” as it uses prototyping and experimentation to quickly deliver innovative solutions. This philosophy seeks to use the simplest and least expensive representative model possible (rather than an expensive final development article) to quickly determine the value of an approach, concept, or technology through incremental development and evaluation. When testing reveals something isn’t working as expected or desired (i.e., a “failure”), the prototype design can either be modified and reevaluated, or decision makers can pivot to a different approach. The faster prototyping “fails,” the faster learning can occur, and the faster decisions can be made regarding the next appropriate step in the development or innovation process. “Failing fast” with prototyping enables the DoD to drive down technical risk, inform requirements, and ensure an integrated and interoperable capability before either weighing down the research and engineering phase of an acquisition with costly procurement decisions or weighing down a procurement program with costly technical risk.

Identifying Military Capability Gaps for Prototyping Projects.

Reference Source: DoD Prototyping Handbook Section 5.1

Prototyping should start with a clear understanding of why the prototype is being developed. For projects in which the objective is to push the boundaries of technology, the purpose could come in the form of a statement of intent by the researcher or an emerging need not yet formally recognized by the Department. However, for projects seeking to directly support an operational mission, the purposes will come in the form of an identified existing or emerging military capability gap.

Military capability gaps can be obtained from numerous sources. Certainly, the most obvious are validated requirements that are documented through formal processes. Examples include requirements listed in approved Joint Capabilities Integration and Development System (JCIDS) documents and strategic needs recorded in the following documents:

  • The 2018 National Defense Strategy;
  • The USD(R&E)’s Road to Dominance modernization priorities;
  • The Chairman’s Risk Assessment; and
  • The Joint Requirements Oversight Council-led Capability Gap Assessment.

Formal requirements also include capability gaps that have been validated by Components or the Joint Staff (JS) and documented by Joint or Military Services’ requirements processes, such as Integrated Priority Lists (IPL) and Initial Capability Documents. In addition, for urgent requirements, warfighters can use their Components’ urgent needs processes or work through the JS’ urgent needs process in which Combatant Commands (CCMD), the Chairman of the Joint Chiefs of Staff, or the Vice Chairman of the Joint Chiefs of Staff can submit Joint Urgent Operational Needs Statements and Joint Emergent Operational Needs Statements.

Unlike formal acquisition programs, however, prototyping is often not bound by traditional Joint or Military Service requirements processes. In fact, the NDS encourages the use of prototyping prior to defining requirements. In these cases, prototyping projects can be initiated using military capability gaps identified and provided by the warfighter, outside of Joint and Military Services’ requirements processes. Sources for these gaps include critical intelligence parameter breaches; emerging needs that are identified through threat, intelligence, and risk assessments; and offsetting or disruptive needs that are identified through experiments, demonstrations, and exercises.

Planning Prototyping Projects

Reference Source: DoD Prototyping Handbook Section 5.2

Successful prototyping begins with effective planning. For traditional acquisition programs, that usually begins with an acquisition strategy and an acquisition plan. Prototyping projects, however, are not obligated to comply with the same documentation requirements as traditional acquisition programs, and not all of the traditional acquisition content is essential for prototyping projects. Instead, plans for prototyping projects should include only the planning content necessary to effectively execute and manage the project.

The most critical element of a prototyping project plan is a clear, unambiguous purpose statement for the project. This statement should include a clear articulation of the problem/need to be addressed, a description of the future decision to be made, the data set that will be generated by the project, and an explanation of how that data will be used to inform the decision. Often, PMs and approval authorities make the mistake of identifying too many variables for a prototyping project to focus on. Instead, prototyping projects should be designed to focus on answering one question at a time. When that question is answered, if appropriate, the process can be repeated to answer additional questions as many times as time and budget permits. Focusing on multiple questions at one time introduces unnecessary concurrent risks to the project and may result in data that are inaccurately applied to answer the questions.

After the purpose statement is defined, the following foundational planning topics should be considered for the plan:

  • Prototype Description
  • Funding and Cost: funding source and execution plan
  • Schedule: project schedule, including technical stage gates and “go/no-go” criteria
  • Risks: project risks and plans to mitigate the risks

Best Practices for Planning Prototyping Project

Reference Source: DoD Prototyping Handbook Section 5.2.1

Prototyping SMEs suggested the following additional topics and considerations for prototyping project plans:

 

  • Cross-Functional Team (CFT). At a minimum, decision-makers, operators who identified the capability need, technology experts, and the transition partner should be identified. Points of contact from the requirements, contracting, finance, and developmental and operational testing communities should also be considered.
  • Evaluation Discussion. A discussion of how the prototype will be evaluated in a relevant environment. The evaluation criteria that will be used to determine if the project has accomplished its purpose should also be included.
  • Integration with Existing Systems. If the project is conducted in support of an existing major defense acquisition program (MDAP) or fielded PoR, the plan should address how the prototype will integrate with the MDAP acquisition strategy or the configuration of a fielded PoR.
  • Intellectual Property and Data Rights. A clear description of DoD rights to any intellectual property or data generated from the project should be included.
  • Sustainment Considerations. Sustainment considerations should be included for projects that expect to leave a residual capability in the field.
  • Transition Plan. A discussion of what will happen with the prototype at the conclusion of the project if the project accomplishes its stated purpose should be included. The transition plan should include as much detail as is available, especially the transition
    partner if known.
  • Waivers and Delegations. A listing of recommended waivers and delegations required to effectively execute the prototyping project in the shortest possible timeframe while ensuring sufficient project management rigor and oversight should be included.

 

The following are examples of the criteria that three DoD organizations within the Office of the Under Secretary of Defense for Research and Engineering (OUSD(R&E)) use in planning their prototyping projects:

The Defense Advanced Research Projects Agency (DARPA) uses a set of questions crafted by a former DARPA director, George H. Heilmeier, to help Agency officials think through and evaluate proposed research programs. A slightly modified set of the questions, known as the “Heilmeier Catechism,” follow:

  • What are you trying to do? What problem are you trying to solve? (Articulate your objectives using absolutely no jargon.)
  • How is it done today? Who does it? What are the limits of current practice?
  • What is new about your approach? Why do you think you can be successful at this time? (Have you done a first-order analysis of your approach?)
  • Who cares? If you are successful, what difference will it make?
  • What are the risks?
  • How long will it take? How much will it cost? What are the mid-term and final “exams” to check for success?
    • What does success look like and how will you demonstrate it?
    • What is your execution plan?
    • How will you measure progress?
    • What are your milestones/metrics?
    • How will your results transition?

 

The Rapid Reaction Technology Office (RRTO) uses a high-level prototyping project planning template for its projects. This 3-page template includes the following elements:

  • Project Description
  • Objective and Value
  • Key Participants
  • Metrics
  • Project Schedule, Task Descriptions, and Deliverables
  • Risk Assessment
  • Spend Plan

 

The Joint Capability Technology Demonstration (JCTD) office uses a robust planning process for their projects. Their plans consist of two primary sections (see Table 3). The first, an Implementation Directive, acts as a “contract” among stakeholders. The second section is the Management Plan, a living document that is updated throughout the life of the project.

Table3 JCTDs Prototype Plan

Soliciting Prototyping Project Proposals

Reference Source: DoD Prototyping Handbook Section 5.3

The need to solicit for proposals depends on the dynamics of the project. When a technology or concept solution is already known, this step is not needed. However, as often is the case, the problem statement is drafted without a specific solution in mind. In these cases, reaching out to other DoD sources, industry, academia, and defense labs for solutions to the problem affords the opportunity to uncover potential solutions heretofore untapped. Once the capability need is clearly defined and the prototyping project plan is drafted, the next major activity is soliciting technology solutions that meet the stated need. Prototyping solutions can be obtained from a number of different sources. PMs or Program Executive Officers of other programs may be able to offer solutions. Many national laboratories, defense laboratories, centers of excellence, and other DoD organizations (e.g., U.S. Army’s Prototyping Integration Facilities) have organic prototyping capability that should be considered. Another approach is reaching out to DoD Federally Funded Research and Development Centers (FFRDC) and University Affiliated Research Centers that develop prototypes. Finally, industry and other academic institutions are also great sources of innovation and prototypes.

Occasionally, solicitation is unnecessary, as industry and other offerors use internal independent research and development (IRAD) funding or other non-DoD funding sources to develop and submit prototypes to DoD as unsolicited proposals. These proposals should be taken seriously and be evaluated for their applicability in satisfying warfighter gaps.

Non-Traditional Solicitation Sources

Reference Source: DoD Prototyping Handbook Section 5.3.2

Defense Innovation Unit (DIU)

DIU is a Joint DoD organization in OUSD(R&E) that connects commercial innovators with defense organizations to rapidly meet Joint warfighter needs. DIU uses a model they created, called Commercial Solutions Opening (CSO), to solicit solutions from nontraditional vendors. DIU works with its DoD partners to develop short (from a few sentences to a paragraph), broad Areas of Interest (AoI) in laymen’s terms that it posts to its website. Interested companies respond directly to DIU with solution briefs in the form of papers (five pages or less) or slide decks (fifteen slides or less).The CSO model is available for use by other DoD components as well. Ultimately, however, the key to DIU’s success is the networks they have cultivated that help them identify and encourage nontraditional vendors to respond to DoD AoIs.

Defense Consortia

Defense Consortia are collaborative partnerships between the U.S. Government and a consortium of industry (large and small, traditional and nontraditional companies), non-profit organizations, research institutes, and academic institutions. Consortia are open to any U.S. entity and, depending on the consortium, may include as many as 500 organizations. A consortium is typically focused on a specific technology area or problem set and is managed by a consortium management firm that acts as the primary interface between the government and consortium members. The consortium management firm will work with a government partner to collaboratively develop a needs statement that it will communicate to its members and will assist in the proposal development and contracting actions of its members.

Partnership Intermediary Agreements

Partnership Intermediary Agreements are agreements between the Federal Government and State or local government agencies or nonprofit entities (i.e., intermediaries) that provide intermediary services between a Federal Government organization and small businesses, educational institutions, and laboratories. These public-facing intermediaries help government organizations find, collaborate with, and contract with industry, labs, and academic partners to discover and develop innovative solutions to solve problems that the organization is trying to solve. A good example of a partnership intermediary is SOFWERX, established through a partnership intermediary agreement between the U.S. Special Operations Command and the non-profit Doolittle Institute. Other partnership intermediaries include the Virginia Tech Applied Research Corporation; Defensewerx; the Center for Technology, Research, and Commercialization; Innovation and Modernization Patuxent River; and the Wright Brothers Institute.

Tradeshows, Conventions, and Industry Associations

Tradeshows, conventions, and industry associations offer great opportunities to discuss problems and mission needs and to announce new requirements, interest in new technologies, and rapid acquisition plans for the procurement of commercial-off-the-shelf technology, prototypes, or services.

Social Media and On-Line Resources

Several SMEs recommended the use of social media and on-line resources as solicitation venues. Social media outlet user groups (e.g., LinkedIn, Facebook, etc.) offer great access to targeted technology networks where nontraditional innovators may have a presence. Posting solicitations to these groups may generate solutions from innovators who otherwise would not have responded to traditional DoD solicitation approaches. Another option is to use crowd-sourcing sites (e.g., InnoCentive) to generate innovative ideas. These approaches may produce some very viable innovative solutions, but they may also spawn unrealistic or unfeasible solutions.

Best Practices for Soliciting Prototyping Project Proposals

Reference Source: DoD Prototyping Handbook Section 5.3.3

Several SMEs offered the following best practices for soliciting solutions for warfighter gaps:

  • When developing solicitations, it is important to remember their intended audiences— especially when pursuing nontraditional defense contractors and academia. PMs should make an effort to translate the warfighter gaps from typical military terminology to technical needs that can be understood by all potential recipients. Any metrics included in the solicitation should also be drafted with the same vocabulary considerations.

  • Before publishing solicitations, operations security analyses and public release reviews should be conducted to protect For Official Use Only or other sensitive information from being released to the public.

  • Solicitations should include a request that respondents provide whatever data sets they have that support their solution or approach. These data sets will assist in determining the feasibility of the proposed solution.

  • If applicable, solicitations should include a statement addressing the possibility of follow-on production.

  • PMs should consider binning the needs into functional areas and having technical SMEs in those functional areas put together outlines of things they need to see from vendors for each functional area. These outlines can then be provided as part of the solicitation.

  • The targeted commercial marketplace should be researched to identify venues and techniques typically used by that marketplace for soliciting specific needs. Potential venues and techniques include catalogs, product directories, trade journals, seminars, professional organizations, contractor briefings, meetings and conversations with companies, in-house experts, on-line resources, social media, and vendor surveys