Software Acquisition

Each page in this pathway presents a wealth of curated knowledge from acquisition policies, guides, templates, training, reports, websites, case studies, and other resources. It also provides a framework for functional experts and practitioners across DoD to contribute to the collective knowledge base. This site aggregates official DoD policies, guides, references, and more.

DoD and Service policy is indicated by a BLUE vertical line.

Directly quoted material is preceeded with a link to the Reference Source.

Reference Source: USD(A&S) Guidance

A value assessment reveals how much impact the software has on the mission from the end user’s perspective. The value assessment measures the program’s progress toward meeting user needs and enables the program to determine if the cost of the software development effort is commensurate with the value it provides. The assessment is typically a combination of subjective and objective measures.

Guiding Principles of a Value Assessment

Programs should conduct thorough value assessments at least annually but should consider a streamlined value assessment regularly (i.e., at demonstration or release). The timing of the VA is negotiated between the sponsor and program office. The User Agreement template has a dedicated section to capture when the sponsor should provide inputs to the VA. In general, we have observed that program offices initiate the VA (which is common for all acquisition documentation) and then garner feedback from the user community. There is no rule that a highly-engaged sponsor cannot take ownership of the VA since it is fundamentally their assessment. In absence of a sponsor taking that role on, the onus is on the program office to initiate the process.

Assessments should be accomplished in a program-appropriate manner that leverages planned activities to the extent possible. E.g., information derived from developmental and operational tests can be used to inform a value assessment.  Field exercises, laboratory experiments, and/or operator-led capability demonstrations may also help determine whether value assessment factors have been satisfied or perhaps why the level of desired value was not achieved. Also, software pipeline and help desk tools may be sources of automated data collections that reflect software development performance and timelines for responding to problems and dynamic warfighter needs.

Notional Scenarios of Value Assessment Frequencies

Programs must consider several factors when determining the value or impact that software has on operations. These factors identify specific benefits that result from using the new software. Overall user satisfaction is one aspect of determining value, but programs should also consider other objective measures, such as:

• Increase in mission effectiveness: Mission effectiveness measures the extent to which the system contributes to mission accomplishment. Ideally, mission effectiveness is evaluated using quantitative or statistical measures, although in some cases it may be evaluated through a qualitative user assessment. Examples of mission-based metrics include lethality, survivability, accuracy, etc.
• Cost efficiencies: Cost savings apply to the system, related systems, and mission operations. Programs should use quantitative data if possible.
• User workload reduction: Workload reduction is the reduction in the amount of effort required by an operator to accomplish a mission task. Effort relates to the number of operations required to perform the task or the cognitive ability required to perform the task. This measure can be qualitative or quantitative.
• User manpower reduction: Manpower reduction is a reduction in the number of operators required to accomplish a mission task. This measure should be quantitative.
• Equipment footprint reduction: Footprint reduction is a reduction in the amount of equipment required in the field in order to accomplish the mission. Equipment includes computers, racks, servers, vehicles, support equipment, or any other items that deploy with the system. Programs should use quantitative data if possible.
• Number and significance of field reports (software trouble reports): Field reports document problems that the end users experience with the software. Problems can include software defects, missing features, inadequate performance, etc. Programs should use quantitative data.
• Software Development Performance: Enhancing responsiveness to warfighter needs can include increasing software deployment frequency to provide new capabilities, add features, or address defects on a faster cycle. Other metrics such as quality (based on change fail rate) can also be used with the intent of demonstrating if the program is meeting its software development potential as part of delivering a high level of value to the user. 
• Enhanced alignment with “to-be” architectures: Aligning to “to-be” architectures can enable   interoperability, update to commonly-used data standards, make product solutions more adaptable, and better support broader mission architectures and threads.  
• User adoption and user satisfaction: User adoption is a quantitative measure that considers how many military units use the software in the field. Some software applications are fielded but are not used, so user adoption measures how useful the military units find the software. User satisfaction is a subjective assessment by the end user that considers usability, suitability, reliability, and overall impact that the software has on performing the mission.
• Net promoter score (NPS): A simple and effective method to determine if customers/end-users are happy with the value generated by your product/service. Customers/end-users rate the product/service on a scale of 1-10 by answering the question, “How likely are you (the customer/end-user) to recommend this product/service to a friend or colleague?” 1-6 ratings are considered detractors (unhappy customers that may spread negative word of mouth), 7 or 8 ratings are considered passive users (satisfied but unenthusiastic customers) 9 or 10 are considered promoters (loyal champions who spread positive word of mouth), The NPS is then calculated as the # of promoters minus the # of detractors divided by the total # of respondents. For example, if you have 50 respondents with 25 promoters and 25 detractors, your NPS would equal (25-25)/50 or 0%. If you have 50 respondents with 35 promoters and 15 detractors, your NPS would equal (35-15)/50 or 40%. The goal over time is to deliver enough value that your NPS rises over time.     

Reference Source: USD(A&S) Guidance   

Overall Rating  

The value assessment should include an overall value rating for the determined cycle. The SWP Semi-Annual reporting proposes 5 categories: No Value, Low Value, Moderate Value, High Value, and Exceptional Value.  These can be defined in greater detail when establishing the program assessment approach for the assessment cycle.  Possible definitions include:  

Potential Objective Metrics Categories 

• Software Development Performance: Quantitative improvements in software development trends as determined by the program metrics (such as lead time, deployment frequency, change fail rates, mean time to restore etc). 
• Increase in Mission Effectiveness: Quantitative improvements to lethality, survivability, accuracy, etc. 
• Cost Efficiencies: Quantitative cost savings achieved for the system, related systems or mission operations.  
• User Workload Reduction: Quantitative efficiencies achieved by reducing the number of operations required to perform a task or the cognitive ability required to perform the task. 
• Manpower Reduction: Quantitative reductions achieved in the number of operators required to accomplish a mission task.  
• Equipment Footprint Reduction: Quantitative reductions in the amount of equipment required in the field to accomplish the mission.  
• User Adoption: Quantitative measurement of the number of military users/organizations that voluntarily adopt a fielded application.  This can be measured by comparing expected adoption rates based on communicated needs and the overall rate of adoption. 

Notional examples of objective and subjective assessment measures 

To assess achieved increases to mission effectiveness. 

To assess software development performance.  This measure is focused on evaluating the means with which operational value is generated. 

 To assess value achieved in areas where the program office is less able to directly influence but where the quality of the product plays a major role.   

To assess value across a wide swath of users based on user satisfaction ratings.  

To assess Return on Investment via a ratio of capability gained against resources expended. 

 To assess design improvements that add value and position the program for future capability while not immediately delivering user features in this assessment cycle.