The third step was to define the methods used to assess the application of SE practices to projects. While various SE models, standards, and so forth can inform this decision (such as CMMISE/ SW, EIA 632, MIL-STD-499B, IEEE-STD-1220, ISO/IEC-15288, and others), this effort was hampered by the fact that a widely accepted definition of what constitutes SE does not exist. To overcome this obstacle, the SEEC chose to survey specific activities that would normally be regarded as elements of SE. The survey analysis then examines the relationships between these activities and overall Project Performance.
Thus, for any activity that did not fit a particular reader’s preferred definition of SE, the analysis results for that activity could be ignored. In general, the
focus of the SE practice assessment was placed on identifying tangible artifacts of SE activities. The SEEC chose to base this assessment primarily upon the Capability Maturity Model Integration (CMMI) due to the SED’s sponsorship of this model, as well as the SEEC’s familiarity with it. Starting with the CMMI-SE/SW/IPPD Model v1.1, we identified the work products that, in the judgment of the SE experts on the committee, result from Systems Engineering tasks. The presence of these work products provides an indication of the magnitude of the Systems Engineering activities performed on the project. Questions were worded to search for the content of these suggested work products, rather than the specified work products themselves, thereby enabling the reporting project to accurately represent their system engineering activities, regardless of the titles
or format of their specific work products.
This approach enabled us to analyze relationships between Project Performance and Systems Engineering work products both individually and in ensemble, searching for those work products most closely tied to project success.
Defines standard
Replaced/Superseded by document(s)
Cancelled by
Amended by
| File | MIME type | Size (KB) | Language | Download | |
|---|---|---|---|---|---|
| NDIA SE Effectiveness Study-FINAL-08sr034.pdf | application/pdf | 1.52 MB | English | DOWNLOAD! |
Provides definitions
Cover images
Abstract
This survey quantifies the relationship between the application of Systems Engineering (SE) best practices to projects and programs, and the performance of those projects and programs. The survey
population consisted of projects and programs executed by defense contractors who are members of the Systems Engineering Division (SED) of the National Defense Industrial Association (NDIA). The deployment of SE practices on a project or program was measured through the availability and characteristics of specific SE-related work products. Project Performance was measured through typically available project measures of cost performance, schedule performance, and scope performance.
Additional project and program information such as project size, project domain, and other data was also collected to aid in characterizing the respondent’s project. Analysis of the survey responses revealed moderately strong statistical relationships between Project Performance and several categorizations of specific of SE best practices. Notably stronger relationships are apparent by combining the effects of more than one the best practices categories. Of course, Systems Engineering Capability alone does not ensure outstanding Project Performance. The survey results show notable differences in the relationship between SE best practices and performance between more challenging as compared to less challenging projects. The statistical relationship between Project Performance and the combination of SE Capability and Project Challenge is quite strong
for survey data of this type.