Benchmark: measure of performance

Benchmarks are databases with measures of performance such as productivity (= software size/effort) on completed software projects.

Why Benchmark?

Benchmarks are valuable for two reasons.

First, if you have just finished a software project and have measured its work-output (= software size) and effort, you can compare its productivity against data on similar projects in a benchmark database. By ‘similar’, we mean projects of the same type (new development or enhancement) that used the same or similar technology (hardware platform and languages), are of similar size, etc. Knowing how your project performed against typical industry data is key to understanding your competitiveness as a software provider and is the first step towards improving performance.

Second, if you can measure the functional size of a new piece of software that must be developed, this can be used with benchmark data to estimate the effort and elapsed time for the new development project.

The ISBSG benchmarking database

COSMIC strongly recommends use of the database of the International Software Benchmarking Standards Group (www.isbsg.org), and urges users of the COSMIC method to submit data on their completed projects to the ISBSG. The ISBSG repository now contains data on hundreds of projects where the delivered software has been measured using the COSMIC method.

Submitted data are anonymized so that there is no risk of your organization’s performance becoming public knowledge. And for every set of project data submitted, you will receive from ISBSG a private report (at no cost) showing how your project’s performance compares against the ISBSG benchmark data. Further, you can access the ISBSG data on-line for ad hoc analyses (e.g. for estimating a new project) or you can purchase the whole database of all project measurements for your own analyses and for future project estimating.

The alternative of accumulating enough project performance measurements in your own organization to establish your own internal benchmarks is highly desirable but can take a long time. Meanwhile, using ISBSG data and services provides an extremely economical solution. The process of submitting your data to ISBSG is easy. COSMIC strongly urges you to join this common effort, in your own and the software community’s interest.

Submitting data to the ISBSG repository

Use the Concise Data Collection Questionnaire (DCQ) which can be found under Downloads. It is vital to submit only good-quality data. Do check the ISBSG Glossary of Terms’ within the questionnaire. Data entry is really very easy.