The Evolution of Systems Engineering in the US Department of Defense

home > News & Events > News > The Evolution of Systems Engineering in the US Department of Defense

The challenge: As the defense budget continues to shrink and the need to innovate continues to grow, the US Department of Defense (DoD) must make better use of its resources. Historically, the DoD has employed systems engineering (SE) to deliver products within cost, schedule, and scope targets. But, the increasing complexity of defense systems makes reaching such targets a constant challenge. The question is, what can the DoD do today to maximize its investment in SE?

The approach: In order to appreciate the DoD’s use of SE, it’s helpful to understand its origins. According to the International Council on Systems Engineering (INCOSE), the term “systems engineering” stems from the practices employed by Bell Telephone Laboratory in the early 1940s.¹ While the DoD didn’t invent SE, it quickly started using the methodology during World War II. After the war, the nonprofit research institution RAND (its formal name was a contraction of Research and Development) was created to connect military planning with research and development decisions. According to RAND, “World War II revealed the importance of technology research and development for success on the battlefield… Forward-looking individuals in the War Department, the Office of Scientific Research and Development, and industry therefore began to discuss the need for a private organization to connect military planning with research and development decisions.”²

Over the course of the next several decades, RAND used system-based principles to develop strategic recommendations for aircraft, weapon and ship capabilities, and military basing locations, as well as to determine how to best implement an air defense campaign and how to develop life-cycle cost estimates for budgeting purposes, among other initiatives.³ While RAND uses a different name for the process, systems analysis (SA), the principles of holistic and system-based planning are very similar in nature to those of SE and have contributed to the body of knowledge we have today. The DoD continued to use these system-based principles to develop missile and missile-defense systems in an effort to stem Cold War aggression from the USSR. While the DoD cannot be credited with inventing SE, it was deeply involved in its evolution and continues to be at the forefront of developing its practices today.

As technology continues to advance, the DoD has evolved from procuring standalone systems to procuring complex and tightly integrated systems of systems. Today, tanks, ships, aircraft, satellites, and ground stations are collecting, processing, and disseminating real-time information to ensure military decision-makers receive the best data as quickly as possible. The interoperability requirements now imposed on project managers have reinforced the need for a disciplined approach to both SE and project management because an ever-increasing number of stakeholders across a wide range of domains must now be served. Figure 1 shows an example of the complex battle space in which the military must currently operate; each system has its own set of stakeholders, timelines, and programmatic risks.

Figure 1. Systems engineering principles are useful in managing military networks such as this one, which links information transmission among US aircraft, partner aircraft, ground stations, and space systems.⁴

The process: To improve the effective use of SE, the DoD must learn from past experience—both successes and failures.

Over the past 20 years, the DoD has had several successful, high-profile programs. This is largely thanks to adherence to sound SE principles. For example, according to a 2005 RAND study on the Navy’s F/A-18 E/F program, “The unparalleled success of the F/A-18 E/F acquisition program emerged from the Engineering and Manufacturing Development (EMD) functions meeting all of the products’ performance requirements, on budget, on schedule, and underweight by 400 pounds. All of this was confirmed in Operational Verification testing (the final exam) and described as an unparalleled success, passing with flying colors and receiving the highest possible endorsement.”

This same study stated that the F/A-18 E/F program’s success can be attributed to many factors, most related to good SE discipline. The report cited many examples of why the program was successful, including:

- “Structuring the contractor team according to prior experience on the F-18 A/B/C/D programs. Specifically, lines of responsibility were clearly defined, with a designated prime contractor ultimately responsible for contract performance.”
- “Cost and schedule estimates were relatively accurate and stable.”
- “The airframe weight had only minor increases, reflecting a stable design.”
- “Using the Navy’s evolutionary development approach for the moderately risky avionics technologies, which was funded outside of the Engineering and Manufacturing Development program.”⁵

By funding the new avionics outside the main development program, the program manager was able to compartmentalize his risk and undertake a new development project without impacting the rest of the program. If the new avionics failed, he could rely on the existing F-18 C/D avionics as a backup solution. While not all programs are designed in the same way, and risk tolerances vary across systems, understanding the advantages of these approaches can inform future development programs.

Despite increased emphasis on SE, the DoD has also learned important lessons from several significant failures to deliver weapons systems on time, on budget, and with the requisite capabilities.

In the cost domain, these failures can be measured in terms of Nunn-McCurdy breaches (Figure 2). According to the Government Accountability Office, “A Nunn-McCurdy breach occurs when a program’s unit cost exceeds certain thresholds. When that happens, DoD must notify Congress of the breach.”⁶ A significant breach is experienced when a program exceeds 15 percent of its current baseline cost (or 30 percent of its original cost), while a critical breach is when a program exceeds 25 percent of its current baseline cost (or 50 percent of its original cost). As part of the 2009 Weapons System Acquisition Reform Act, any program that experiences a critical breach is terminated unless it is certified by the secretary of defense. Programs that are certified typically undergo a restructuring, a revocation of previous milestone approvals, and require a written explanation as to the root cause of the cost growth.

Here are two recent and high-profile examples of Nunn-McCurdy breaches:

The F-35 Lightning II Program in 2010. This program saw significant cost growth in the per-unit price of the aircraft, causing it to exceed the 2002 baseline by more than 57 percent. One of the root causes was the discovery of a significant weight and design issue in the first prototype.⁸ If proper SE principles of risk management had been in place, this technical deficiency could have been caught before significant rework was required.
The Global Positioning System Next-Generation Operational Control System in 2016. “Air Force Secretary Deborah Lee James declared the breach on June 30 [2016] after quarterly reviews showed inadequate systems engineering at program inception, Block 0 software with high defect rates, and Block 1 designs requiring rework.”⁹ From 2012 to the time of the breach, program cost estimates rose 22 percent from $3.4 billion to $4.2 billion.¹⁰

In both cases, lack of proper SE played a central role in the cost overruns.

Figure 2. This chart shows the number of critical and significant Nunn-McCurdy breaches in the Department of Defense between 2007 and 2015. Such breaches reveal failures that could be addressed through systems engineering.⁷

The process and tools: Organizational changes have been made at the DoD to emphasize the importance of SE. To help manage increased complexity within DoD programs, the Office of the Deputy Assistant of Defense for Systems Engineering, ODASD(SE), was chartered in 2011 as “the point of contact for policy, practice, and procedural matters relating to DoD System Engineering and its key elements including technical risk management, software engineering, manufacturing and production, quality, standardization, and related disciplines.”¹¹ This office provides continued workforce development and ensures security across platforms and proper technical risk management. Additionally, it has ownership of the Systems Engineering Plan (SEP), a document required by all major defense acquisition programs that gives the project manager a framework for identifying the important SE components to execute a program. The SEP template addresses:

system architecture and interface control;
risk and opportunity management;
technical schedule and schedule risk assessment;
technical performance metrics and key performance indicators;
stakeholder management;
configuration and change management;
technical reviews and their associated entrance and exit criteria;
engineering tools; and
many other topics.

While templates and documentation are important in instilling SE discipline across an organization, it is equally important to ensure that their intent is carried out by the project team.

To accomplish this, ODASD(SE) oversees education, training, and competency screening. The office reviews the content of classes offered by the DoD’s source for project management and SE training, Defense Acquisition University, and staff members serve as subject matter experts in updating core competencies and the experiential requirements necessary to successfully execute core SE activities across the DoD. By establishing this office, the DoD is making a concerted effort to acquire, train, and retain the best SE talent possible.

Instilling sound systems engineering principles across a large enterprise requires both structural and cultural change. In addition to the organizational change mentioned above, the DoD established an initiative called Better Buying Power (BBP) in 2010.¹² The intent of this initiative is to improve acquisition efficiency in the face of declining defense budgets. Architected by the former assistant secretary of defense for acquisition, technology, and logistics, this initiative consists of 23 principles aimed at increasing DoD efficiency and productivity. BBP has been revised twice since then, with BBP 3.0 expanding to focus on 36 goals in eight core areas:

achieve affordable programs;
achieve dominant capabilities and controlled lifecycle cost;
incentivize industry and government productivity;
incentivize industry and government innovation;
eliminate unproductive bureaucracy and processes;
promote effective competition;
improve tradecraft in services acquisition; and
improve professionalism of total acquisition workforce.

Several of these core areas are dependent on the development and execution of SE principles, specifically: controlling lifecycle cost, spurring and incentivizing innovation, removing unproductive processes, and improving the tradecraft and professionalism of the workforce.

Results: According to the 2016 Annual Report on the Performance of the Defense Acquisition System, “The Department of Defense (DoD) is making continuing progress in improving acquisition. The overall series [of reports] presents strong evidence that the DoD has moved—and is moving—in the right direction with regard to the cost, schedule, and quality of the products we deliver. There is, of course, much more that can be done to improve defense acquisition, but with the 5-year moving average of cost growth on our largest and highest-risk programs at a 30-year low, it is hard to argue that we are not moving in the right direction.”¹³

Specifically, from 2011 to 2015, the growth of contracted costs for major development acquisition programs (MDAP) shrunk from 9 percent to 3.5 percent, its lowest growth in 30 years. Additionally, it was mentioned in the discussion above that Nunn-McCurdy breaches are an indicator of cost growth that could be attributed to poor systems engineering discipline. In the years since BBP was implemented, these breaches have significantly declined. Figure 3 shows the decreasing trend in the percentage of breaches vs. the overall number of MDAP programs.

Figure 3. The Department of Defense (DoD) has seen a decrease in Nunn-McCurdy breaches since its Better Buying Power effort was initiated in 2011.This DoD chart shows the downward trend in breaches attributed to quantity changes.¹⁴

While cost growth seems to be moving in the right direction, schedule growth metrics show mixed performance in the report. The outcome of the metrics is largely based on the data that’s included (completed vs. active programs, etc.). In some cases, there is a decrease in schedule growth, while in other cases there is zero, or even an increase in schedule growth. This emphasizes the fact that continual improvement is required, and the DoD must continue to develop its workforce to instill the systems engineering discipline needed for success. It should be noted that independent of schedule overruns, the DoD has seen a significant increase in planned schedule duration, from an average of three years in 1980 to an average of six and a half years in 2016. This data directly correlates with the complexity of the systems the department is procuring. As the DoD moves from independent systems toward systems of systems, this duration will continue to grow, reinforcing the need for proper program control.

Next steps: While there have been many successful acquisition programs throughout the DoD’s history, there have also been numerous examples of programs that failed to deliver their product within cost, schedule, and scope targets. By implementing structural, cultural and strategic changes, the DoD can gain significant returns on its investment in SE—from basic research, to weapon development, to the integration of systems of systems. It is up to us, the future leaders, to help transform this vision into a reality.

The views expressed in this article are solely those of the author and do not reflect the official policy or position of the US Air Force or Department of Defense.

About the Author

Austin Page is a major in the US Air Force. He has served as a program manager on various research and development, avionics, electronic warfare, and weapons projects, culminating in his work as deputy program manager for weapons integration in the F-35 program. He is currently a master’s degree student in MIT System Design & Management. He has a master’s degree in electrical engineering from Wright State University and a bachelor’s degree in electrical engineering from the University of Maryland, College Park.