September 26th, 2008 chris
Move over Heidi, Kate and Cindy, if we are talking about software reliability, this model will surely dominate the runway. Supermodel Rayleigh.
A model of software reliability, the Rayleigh model is a parametric model in the sense that it is based on a specific statistical distribution. When the parameters of the statistical distribution are estimated based on the data from a software project, projections about the defect rate of the project can be made based on the model.
The Rayleigh model is a special case of the Weibull distribution family, which has been widely used for reliability studies in various fields. Supported by a large body of empirical data, software projects were found to follow a life-cycle pattern described by the Rayleigh curve, for both resource and staffing demand and defect discovery/removal patterns. The Rayleigh model is implemented in several software products for quality assessment. It can also be implemented easily via statistical software packages, such as the example provided in this chapter.
Compared to the phase-based defect removal model, the Rayleigh model is a formal parametric model that can be used for projecting the latent software defects when the development work is complete and the product is ready to ship to customers. The rationale behind the model fits well with the rationale for effective software development. Specifically, while the defect removal effectiveness approach focuses on defect removal, the Rayleigh encompasses both defect prevention (reduction in defect rates) and early defect removal.
In addition to quality projection, another strength of the Rayleigh model is that it provides an excellent framework for quality management.
Posted in Software Metrics
September 26th, 2008 chris
Kaoru Ishikawa, a Japanese University professor and influential quality management innovator, developed seven basic tools for quality control.
They are the following:
- Checklist (or Check Sheet)
- Pareto diagram
- Histogram
- Scatter Diagram
- Run Chart
- Control Chart
- Cause-and-effect Diagram
Posted in Software Metrics
September 26th, 2008 chris
Customer satisfaction is often measured by customer survey data via the five-point scale:
-
Very satisfied
-
Satisfied
-
Neutral
-
Dissatisfied
-
Very dissatisfied.
Satisfaction with the overall quality of the product and its specific dimensions is usually obtained through various methods of customer surveys. For example, the specific parameters of customer satisfaction in software monitored by IBM include the CUPRIMDSO categories (capability, functionality, usability, performance, reliability, installability, maintainability, documentation/information, service, and overall); for Hewlett-Packard they are FURPS (functionality, usability, reliability, performance, and service).
Based on the five-point-scale data, several metrics with slight variations can be constructed and used, depending on the purpose of analysis. For example:
(1) Percent of completely satisfied customers
(2) Percent of satisfied customers (satisfied and completely satisfied)
(3) Percent of dissatisfied customers (dissatisfied and completely dissatisfied)
(4) Percent of nonsatisfied (neutral, dissatisfied, and completely dissatisfied)
Usually the second metric, percent satisfaction, is used. In practices that focus on reducing the percentage of nonsatisfaction, much like reducing product defects, metric (4) is used.
In addition to forming percentages for various satisfaction or dissatisfaction categories, the weighted index approach can be used. For instance, some companies use the net satisfaction index (NSI) to facilitate comparisons across product. The NSI has the following weighting factors:
NSI ranges from 0% (all customers are completely dissatisfied) to 100% (all customers are completely satisfied). If all customers are satisfied (but not completely satisfied), NSI will have a value of 75%. This weighting approach, however, may be masking the satisfaction profile of one’s customer set. For example, if half of the customers are completely satisfied and half are neutral, NSI’s value is also 75%, which is equivalent to the scenario that all customers are satisfied. If satisfaction is a good indicator of product loyalty, then half completely satisfied and half neutral is certainly less positive than all satisfied. Furthermore, we are not sure of the rationale behind giving a 25% weight to those who are dissatisfied. Therefore, this example of NSI is not a good metric; it is inferior to the simple approach of calculating percentage of specific categories. If the entire satisfaction profile is desired, one can simply show the percent distribution of all categories via a histogram. A weighted index is for data summary when multiple indicators are too cumbersome to be shown. For example, if customers’ purchase decisions can be expressed as a function of their satisfaction with specific dimensions of a product, then a purchase decision index could be useful. In contrast, if simple indicators can do the job, then the weighted index approach should be avoided.
Posted in Software Metrics
September 4th, 2008 chris
Software metrics can be classified into three categories: product metrics, process metrics, and project metrics. Product metrics describe the characteristics of the product such as size, complexity, design features, performance, and quality level. Process metrics can be used to improve software development and maintenance. Examples include the effectiveness of defect removal during development, the pattern of testing defect arrival, and the response time of the fix process. Project metrics describe the project characteristics and execution. Examples include the number of software developers, the staffing pattern over the life cycle of the software, cost, schedule, and productivity. Some metrics belong to multiple categories. For example, the in-process quality metrics of a project are both process metrics and project metrics.
Posted in Software Metrics
September 4th, 2008 chris
In software, the narrowest sense of product quality is commonly recognized as lack of “bugs” in the product. It is also the most basic meaning of conformance to requirements, because if the software contains too many functional defects, the basic requirement of providing the desired function is not met. This definition is usually expressed in two ways: defect rate (e.g., number of defects per million lines of source code, per function point, or other unit) and reliability (e.g., number of failures per n hours of operation, mean time to failure, or the probability of failure-free operation in a specified time). Customer satisfaction is usually measured by percent satisfied or nonsatisfied (neutral and dissatisfied) from customer satisfaction surveys. To reduce bias, techniques such as blind surveys (the interviewer does not know the customer and the customer does not know the company the interviewer represents) are usually used. In addition to overall customer satisfaction with the software product, satisfaction toward specific attributes is also gauged. For instance, IBM monitors satisfaction with its software products in levels of CUPRIMDSO (capability [functionality], usability, performance, reliability, installability, maintainability, documentation/information, service, and overall). Hewlett-Packard focuses on FURPS (functionality, usability, reliability, performance, and serviceability). Other companies use similar dimensions of software customer satisfaction. Juran calls such attributes quality parameters, or parameters for fitness for use.
Posted in Software Metrics