Risk Identification: Integration & Ilities (RI3) Guidebook

There are many reasons for the cost and schedule difficulties associated with problematic projects - insufficient “upfront” funding, inadequate schedule, technical immaturity (including both technology immaturity and the “state” of system level design), funding instability, lack of logistics support, improper training, etc. Arguably the first three of these contributors have the most impact in the early phases of a project – even at the point of conceptual development. Cost, schedule, and technology are in fact the elements encompassed by the term “resources” in the GAO’s assertion regarding lack of resources creating significant project cost and schedule growth.

The issue of technology immaturity has been so often indicated in cost and schedule overruns that the DOD has adopted the Technology Readiness Levels (TRLs), originally developed by NASA4. TRLs provide a widely-accepted common language and measurement scale to enhance communication within and between the DOD science and technology (S&T) and acquisition communities, both in government and industry. The use of TRLs is becoming an essential element in the overall knowledge based acquisition decision process. Within the Air Force, SAF/AQR has developed a systematic process by which Critical Technology Elements (CTEs) in a system can be identified and their TRLs rated by an Independent Review Team (IRT). By law and policy, all Critical Technology Elements (CTEs) in a system are to be at or above TRL 6 at Milestone B and TRL 7 at Milestone C.

There is often a tendency for persons outside a program to believe that if the CTEs of a system are all at TRL 6, then the technical risks related to those technologies have been paid down. Unfortunately, this is not the case. While a TRL assessment addresses the issue of immaturity in the CTEs themselves, it is not a risk assessment. In particular, it does not deal with the risks associated with integration of those CTEs to each other or to other elements of the system. Some of the other elements of the system, while not new technology, become critical to the functioning of the system during the process of integrating or interoperating with the CTEs. These could be referred to as “high value non-CTEs.” Nor does a TRL assessment necessarily deal with the sustainability and other “ilities” required for successful program implementation. Unfortunately, historically, integration and the “ilities” have been significant stumbling blocks for many programs, even those with supposedly mature technologies. Reference 6 shows that, between 2001 and 2005, supportability issues accounted for $38.1 billion of DoD cost growth. An example of integration problems can be found in Reference 7, but the issues continue to this day in other programs.