A Theory in Measurement
In science, there are a lot of theories, and one of the most popular is Albert Einstein’s theory of gravitation called “General relativity”. According to general relativity, the observed gravitational attraction between masses results from the warping of space and time by those masses.
In software development, particularly on estimation, there are also theories that exist. One is a theory to define measurements and metrics, and it is called the “Measurement Theory”.
Measurement theory allows us to validly define measurements and metrics and to use statistical analysis on our metrics data.
Measurement theory is a branch of applied mathematics. The specific theory we use is called the Representational Theory of Measurement. It formalizes our intuition about the way the world actually works. In this theory, intuition is the starting point for all measurement. Any data manipulation of the measurements must preserve the relationships that we observe between the real-world entities that we are measuring.
Measurement theory allows us to validly analyze and manipulate our measurement data.
As an example, consider a customer satisfaction survey. Your users were asked what they thought of your customer support. The possible answers were:
1 Excellent
2 Good
3 Average
4 Inadequate
5 Poor
The result was that you ended up with a score of 3. So, you have average support, and it means you just need to improve a bit, right? Well, maybe. Or maybe not. When you looked at the data, you found out that your scores were 50% Excellent and 50% Poor. No one thought your customer support was average. Measurement theory dicates that taking the average of this kind of data is invalid and can give misleading results.
Measurement theory allows us to use statistics and probability to understand quantitatively the possible variances, ranges, and types of errors in the data.
Assume that you are responsible for estimating the effort and schedule for a new project. You use a prediction model for development effort, and it predicts that, on average, the effort will be 10 staff years with a duration of 1 year. What would you then tell your boss as your estimate? Maybe you would pad it by one staff year, just to be safe. What if you knew that the standard deviation is 2 staff years? A standard deviation of 2 means that 68% of the time, you expect the result to be between 8 and 12 staff years. This means that 16% of the time it will be over 12 staff years. Now what would you estimate? 12 staff years? 13? Or 10? What if, instead, you knew that the standard deviation was 4 months? Then what would you estimate? 10.5 staff years? (By the way, if we were the boss, we would prefer an answer that included the range with probabilities. We would want the additional information to better trade-off the risks and the rewards.)