Home>Usage of the COSMIC method>COSMIC benchmark data

COSMIC benchmark data

Contents:

ISBSG and COSMIC benchmark data
COSMIC project data in the ISBSG repository
Screenshots of ISBSG data and reports
Accessing COSMIC benchmark data
Submitting COSMIC benchmark data
Information in the Knowledge Base

ISBSG and COSMIC benchmark data

Benchmark data can be used for software project estimation, benchmarking, contracting, supplier performance measurement, Request For Proposal management, contract management, procurement and other crucial processes in the organization.

Although accumulating enough project performance measurements in your organization to establish internal benchmarks is highly desirable, this usually takes a long time. Meanwhile, using ISBSG data and services provides an economical solution.

The International Software Benchmarking Standards Group (ISBSG) is a not-for-profit organization that collects data from the industry of software projects and application maintenance and support. The mission of ISBSG is:

“To improve the management of IT resources by both business and government, through the provision and exploitation of public repositories of software engineering knowledge that are standardized, verified, recent and representative of current technologies”

COSMIC supports the mission of ISBSG to improve decision making in the software industry by providing relevant and current data.

The ISBSG repository ‘New Developments & Enhancements’ contains data of over 8000 projects, releases and sprints. Over 500 of these were measured in the COSMIC FSM method. This is an invaluable source of data, especially when

  • you don’t have your own dataset yet, or
  • you wish to compare your metrics against industry data.

All ISBSG data is

  • validated and rated in accordance with its quality guidelines
  • current
  • representative of the industry
  • independent and trusted
  • captured from a range of organization sizes and industries

COSMIC project data in the ISBSG repository

As noted above, there is now data on hundreds of projects where the delivered software size has been measured using the COSMIC method. This number is growing steadily. A report is available from the ISBSG containing a detailed analysis of these projects, showing how project productivity (size/effort) and speed (size/duration) vary with, e.g.

  • Software domain (business application, real-time, software component)
  • New development versus enhancement projects
  • Programming language level and type (e.g. 3GL vs 4GL, COBOL, Java, etc)

The report also contains

  • An analysis of how effort is distributed over the major activities of a project by percentages
  • An analysis of the characteristics of COSMIC functional size measurements, e.g. the proportions of Entries, Exits, Reads and Writes and how these vary with software domain
  • Examples of how existing ISBSG benchmark data, obtained from projects using the IFPUG FSM method, compare with benchmarks obtained with COSMIC-measured project data


The report should be of major value to all organizations using the COSMIC method or thinking of adopting the method.

Screenshots of ISBSG data and reports

The ISBSG data is provided in an MS Excel sheet, containing over 8250 rows and over 250 data attribute columns.

A screenshot of the MS Excel sheet:

ISBSG Data Sheet

Free Benchmark reports compare your project against some relevant peer groups.

A few screenshots of Benchmark reports:

ISBSG Quality Benchmark ISBSG Productivity Benchmark

Accessing COSMIC benchmark data

All ISBSG data on COSMIC-measured projects in the ISBSG repository can be accessed via the ISBSG data portal. Please check www.isbsg.org/data  for more information about the data and how to license it.

For a no-cost trial, you can establish an account on the portal and get 10 free credits. You can then experiment with searching and filtering the data. You can then buy more credits if you wish to take a license to continue accessing the COSMIC data.
Instructions for searching the COSMIC projects in the repository are given below. The system will tell you how many projects are available and how many credits are needed. If you choose to license the data you will get a set of reports for your dataset and free use of the OLAP tool to do your own analysis of your dataset. Your dataset will be saved for you and can be recalled at any time.

  1. After you login, the Search screen will be displayed
  2. In the Add Filters section from the drop down list boxes, select: Count Approach = COSMIC
  3. Click on the blue Search button (lower right)
  4. You will get a message telling you that the search will cost you 2 credits (2 of your free credits)
  5. Click OK
  6. The system will search the ISBSG data and then report the number of COSMIC projects that were found
  7. You still have 8 free credits left to experiment with your own searches
  8. If you run out of credits and want to do more searches or license a dataset, click on Buy Credits at the top of the screen

If you want to refine your search, for example search for COSMIC, Real-Time projects, then after step 2 above:

    • Click the green + button
    • In the Add Filters section from the drop down list boxes select: Application Group = Real-Time Application
    • Then continue from step 3 above.

Submitting COSMIC benchmark data

ISBSG invites the international software community to contribute by submitting data and get a free benchmark report in return! The data will be processed by ISBSG anonymously.

For adding project data a COSMIC questionnaire is available: the COSMIC/ISBSG Concise Data Collection Questionnaire.

Information in the Knowledge Base

Further information on benchmark data and references to ISBSG material can be found in the benchmark section of the Knowledge Base.