No Access

MEASUREMENT IN SOFTWARE ENGINEERING: FROM THE ROADMAP TO THE CROSSROADS

CARLO GABRIEL PORTO BELLINI

Universidade Federal da Paraíba at João Pessoa, Brazil

Search for more papers by this author

RITA DE CÁSSIA DE FARIA PEREIRA

Universidade Federal da Paraíba at João Pessoa, Brazil

Search for more papers by this author

and

JOÃO LUIZ BECKER

Universidade Federal do Rio Grande do Sul at Porto Alegre, Brazil

Search for more papers by this author

https://doi.org/10.1142/S021819400800357XCited by:15

Abstract

Research on software measurement can be organized around five key conceptual and methodological issues: how to apply measurement theory to software, how to frame software metrics, how to develop metrics, how to collect core measures, and how to analyze measures. The subject is of special concern for the industry, which is interested in improving practices — mainly in developing countries, where the software industry represents an opportunity for growth and usually receives institutional support for matching international quality standards. Academics are also in need of understanding and developing more effective methods for managing the software process and assessing the success of products and services, as a result of an enhanced awareness about the emergency of aligning business processes and information systems. This paper unveils the fundamentals of measurement in software engineering and discusses current issues and foreseeable trends for the subject. A literature review was performed within major academic publications in the last decade, and findings suggest a sensible shift of measurement interests towards managing the software process as a whole — without losing from sight the customary focus on hard issues like algorithm efficiency and worker productivity.

Keywords:

References

A. Abran et al. (eds.) , Guide to the Software Engineering Body of Knowledge — Trial Version 1.00 ( IEEE Computer Society Press , Los Alamitos , 2001 ) . Google Scholar
A. Abran , A. Sellami and W. Suryn , Metrology, measurement and metrics in software engineering , Proc. 9th Int. Software Metrics Symposium ( IEEE ) . Google Scholar
P. J. Ågerfalk and B. Fitzgerald, Commun. ACM 49(10), 27 (2006). Google Scholar
L. M. Applegate and J. L. King, MIS Quarterly 23(1), 17 (1999), DOI: 10.2307/249404. Crossref, ISI, Google Scholar
D. E. Avison and G. Fitzgerald, Rethinking Management Information Systems: An Interdisciplinary Perspective, eds. W. L. Currie and B. Galliers (Oxford University Press, New York, 1999) pp. 250–278. Google Scholar
E. Babbie , Métodos de Pesquisa de Survey ( Universidade Federal de Minas Gerais , 1999 ) . Google Scholar
R. P. Bagozzi, Y. Yi and L. W. Phillips, Administrative Science Quarterly 36(3), 421 (1991), DOI: 10.2307/2393203. Crossref, ISI, Google Scholar
L. Bardin, Análise de Conteúdo, 70th edn. (Lisbon, 1977). Google Scholar
M. O. Barros, C. M. L. Werner and G. H. Travassos, J. Systems and Software 70(1–2), 21 (2004), DOI: 10.1016/S0164-1212(02)00155-3. Crossref, ISI, Google Scholar
V. R. Basili , Is there a future for empirical software engineering? , Proc. ISESE'06 ( ACM ) . Google Scholar
V. R. Basili and H. D. Rombach, IEEE Trans. Software Engineering 14(6), 758 (1988), DOI: 10.1109/32.6156. Crossref, ISI, Google Scholar
S. Beechamet al., J. Systems and Software 76(3), 251 (2005), DOI: 10.1016/j.jss.2004.06.004. Crossref, ISI, Google Scholar
S. Beecham, T. Hall and A. Rainer, Empirical Software Engineering 8(1), 7 (2003), DOI: 10.1023/A:1021764731148. Crossref, ISI, Google Scholar
C. G. P. Bellini, M.E.T.R.I.C.S. — Model for Eliciting Team Resources and Improving Competence Structures. A Socio-technical Treatise on Managing Customer Professionals in Software Projects for Enterprise Information Systems, Ph.D. dissertation, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil, 2006 . Google Scholar
C. G. P. Bellini, J. L. Becker and D. Borenstein, Int. J. Computers, Systems and Signals 5(1), 16 (2004). ISI, Google Scholar
I. Benbasat, D. K. Goldstein and M. Mead, MIS Quarterly 11(3), 369 (1987), DOI: 10.2307/248684. Crossref, ISI, Google Scholar
I. Benbasat and R. W. Zmud, MIS Quarterly 23(1), 3 (1999), DOI: 10.2307/249403. Crossref, ISI, Google Scholar
J. H. Blackstone Jr., L. R. Gardiner and S. C. Gardiner, Int. J. Production Research 35(3), 597 (1997). Crossref, ISI, Google Scholar
J. T. Boardman, IEEE Trans. Systems, Man and Cybernetics 25(7), 1150 (1995), DOI: 10.1109/21.391296. Crossref, ISI, Google Scholar
J. Bosch and L. Lundberg, J. Systems and Software 66(3), 183 (2003), DOI: 10.1016/S0164-1212(02)00075-4. Crossref, ISI, Google Scholar
M. C. Boudreau, D. Gefen and D. W. Straub, MIS Quarterly 25(1), 1 (2001), DOI: 10.2307/3250956. Crossref, ISI, Google Scholar
P. Bourqueet al., IEEE Software 16(6), 35 (1999), DOI: 10.1109/52.805471. Crossref, Google Scholar
L. C. Briand, K. El Emam, and S. Morasca, Theoretical and empirical validation of software product measures, Technical Report ISERN-95-03, 1995 . Google Scholar
L. C. Briand, K. El Emam and S. Morasca, Empirical Software Engineering 1(1), 61 (1996), DOI: 10.1007/BF00125812. Crossref, Google Scholar
L. C. Briand, S. Morasca and V. R. Basili, IEEE Trans. Software Engineering 28(12), 1106 (2002), DOI: 10.1109/TSE.2002.1158285. Crossref, ISI, Google Scholar
M. E. Bush and N. E. Fenton, J. Systems and Software 12(3), 223 (1990), DOI: 10.1016/0164-1212(90)90043-L. Crossref, ISI, Google Scholar
J. K. Chhabra, K. K. Aggarwal and Y. Singh, Information and Software Technology 45(8), 539 (2003). Crossref, ISI, Google Scholar
G. A. Churchill Jr., J. Marketing Research 16, 64 (1979), DOI: 10.2307/3150876. Crossref, ISI, Google Scholar
D. S. Corbin, J. Systems Management 42(9), 28 (1991). Google Scholar
W. L. Currie and B. Galliers (eds.) , Rethinking Management Information Systems: An Interdisciplinary Perspective ( Oxford University Press , New York , 1999 ) . Google Scholar
W. L. Currie and I. A. Glover, Rethinking Management Information Systems: An Interdisciplinary Perspective, eds. W. L. Currie and B. Galliers (Oxford University Press, New York, 1999) pp. 417–443. Google Scholar
T. H. Davenport and M. L. Markus, MIS Quarterly 23(1), 19 (1999), DOI: 10.2307/249405. Crossref, ISI, Google Scholar
C. Dekkers and P. McQuaid, The dangers of using measurement to (mis)manage: Measuring the software process, Proc. ASQ's Annual Quality Congress (American Society for Quality) pp. 551–560. Google Scholar
R. F. DeVellis , Scale Development — Theory and Applications ( Sage , Newbury Park , 1991 ) . Google Scholar
J. A. Díez, Studies in History and Philosophy Sciences 28(1), 167 (1997). Crossref, Google Scholar
L. Donaldson, Handbook of Organization Studies, eds. S. R. Clegg, C. Hardy and W. R. Nord (Sage, London, 1996) pp. 57–76. Google Scholar
L. M. Duvall, J. Systems and Software 31(2), 109 (1995), DOI: 10.1016/0164-1212(94)00091-Z. Crossref, ISI, Google Scholar
T. Dybå, Empirical Software Engineering 7(4), 387 (2002). Crossref, Google Scholar
H. Eisner , Essentials of Project and Systems Engineering Management ( John Wiley and Sons , New York , 1997 ) . Google Scholar
K. El Emam, Guide to the Software Engineering Body of Knowledge — Trial Version 1.00, eds. A. Abranet al. (IEEE Computer Society Press, Los Alamitos, 2001) pp. 137–154. Google Scholar
W. M. Evanco and R. Lacovara, J. Systems and Software 26(1), 77 (1994), DOI: 10.1016/0164-1212(94)90098-1. Crossref, ISI, Google Scholar
S. Faraj and L. Sproull, Management Science 46(12), 1554 (2000), DOI: 10.1287/mnsc.46.12.1554.12072. Crossref, ISI, Google Scholar
A. V. Feigenbaum, Quality Progress 34(5), 26 (2001). Google Scholar
N. E. Fenton and M. Neil, J. Systems and Software 47(2–3), 149 (1999), DOI: 10.1016/S0164-1212(99)00035-7. Crossref, ISI, Google Scholar
N. E. Fenton and M. Neil, Software metrics: Roadmap, Proc. 22nd Int. Conf. on Software Engineering (ACM) pp. 357–370. Google Scholar
N. V. Flor, Commun. ACM 49(10), 57 (2006), DOI: 10.1145/1164394.1164421. Crossref, Google Scholar
J. M. Fordet al., Social Science Computer Review 18(3), 258 (2000). Crossref, ISI, Google Scholar
M. B. Franzen and C. G. P. Bellini, Revista Eletrônica de Administração 11(3), (2005). Google Scholar
A. Fuggettaet al., ACM Trans. Software Engineering and Methodology 7(4), 411 (1998), DOI: 10.1145/292182.292197. Crossref, Google Scholar
R. A. Gardner, Quality Progress 34(3), 51 (2001). Google Scholar
D. Gefen, D. W. Straub and M.-C. Boudreau, Commun. AIS 4, 1 (2000). Google Scholar
C. Ghezzi , M. Jazayeri and D. Mandrioli , Fundamentals of Software Engineering ( Prentice-Hall , Englewood Cliffs , 1991 ) . Google Scholar
R. L. Glass, I. Vessey and V. Ramesh, Information and Software Technology 44(8), 491 (2002), DOI: 10.1016/S0950-5849(02)00049-6. Crossref, ISI, Google Scholar
A. Gopalet al., IEEE Trans. Software Engineering 28(9), 863 (2002), DOI: 10.1109/TSE.2002.1033226. Crossref, ISI, Google Scholar
J. F. Hair Jr. et al. , Multivariate Data Analysis ( Prentice Hall , Upper Saddle River , 1998 ) . Google Scholar
N. Hanebutte, C. S. Taylor and R. R. Dumke, Empirical Software Engineering 8(1), 43 (2003), DOI: 10.1023/A:1021716832057. Crossref, ISI, Google Scholar
W. Harrison, J. Systems and Software 72(2), 225 (2004), DOI: 10.1016/S0164-1212(03)00092-X. Crossref, ISI, Google Scholar
W. Hetzel, Applying Software Metrics, eds. P. Oman and S. L. Pfleeger (IEEE, Los Alamitos, 1997) pp. 72–93. Google Scholar
G. H. Hofstede , Cultures and Organizations ( Harper Collins , London , 1994 ) . Google Scholar
R. Hoving, MIS Quarterly Executive 2(1), 29 (2003). Google Scholar
J. Hussey and R. Hussey , Business Research ( Palgrave Macmillan , New York , 1997 ) . Crossref, Google Scholar
J. Iversen and L. Mathiassen, Information Systems Journal 13(1), 3 (2003), DOI: 10.1046/j.1365-2575.2003.00136.x. Crossref, ISI, Google Scholar
M. C. Jackson and P. Keys, J. Operational Research Society 35(6), 473 (1984), DOI: 10.2307/2581795. Crossref, ISI, Google Scholar
M. Jørgensen, Advances in Engineering Software 30(12), 907 (1999). Crossref, Google Scholar
M. Jørgensen, J. Systems and Software 70(1–2), 37 (2004). Crossref, Google Scholar
M. Jørgensen, K. H. Teigen and K. Moløkken, J. Systems and Software 70(1–2), 79 (2004). Crossref, Google Scholar
R. S. Kaplan and D. P. Norton , The Balanced Scorecard ( Harvard Business School Press , Boston , 1996 ) . Google Scholar
P. Katerattanakul and B. Han, European Journal of Information Systems 12(1), 60 (2003), DOI: 10.1057/palgrave.ejis.3000447. Crossref, ISI, Google Scholar
M. Keil, J. Mann and A. Rai, MIS Quarterly 24(4), 631 (2000), DOI: 10.2307/3250950. Crossref, ISI, Google Scholar
T. M. Khoshgoftaar, E. B. Allen and D. L. Lanning, J. Systems and Software 36(2), 103 (1997), DOI: 10.1016/0164-1212(95)00063-1. Crossref, ISI, Google Scholar
T. M. Khoshgoftaar and N. Seliya, Empirical Software Engineering 8(3), 255 (2003), DOI: 10.1023/A:1024424811345. Crossref, ISI, Google Scholar
S. R. Kirk and S. Jenkins, J. Systems and Software 72(2), 179 (2004), DOI: 10.1016/S0164-1212(03)00217-6. Crossref, ISI, Google Scholar
B. Kitchenham, Procedures for performing systematic reviews, Joint Technical Report — Keele University Technical Report TR/SE-0401 and National ICT Australia Ltd., Technical Report 040011T.1, 2004 . Google Scholar
A. S. Lee, MIS Quarterly 23(1), 29 (1999), DOI: 10.2307/249407. Crossref, ISI, Google Scholar
H. K. N. Leung, Information and Management 38(3), 137 (2001), DOI: 10.1016/S0378-7206(00)00060-4. Crossref, ISI, Google Scholar
M. Lindvall, R. T. Tvedt and P. Costa, Empirical Software Engineering 8(1), 83 (2003), DOI: 10.1023/A:1021772917036. Crossref, ISI, Google Scholar
F. Lopes and P. Morais, J. Information Technology Education 1(2), 103 (2002). Crossref, ISI, Google Scholar
K. Lyytinen, MIS Quarterly 23(1), 25 (1999), DOI: 10.2307/249406. Crossref, ISI, Google Scholar
S. G. MacDonell and A. R. Gray, Guide to the Software Engineering Body of Knowledge — Trial Version 1.00, eds. A. Abranet al. (IEEE Computer Society Press, Los Alamitos, 2001) pp. 121–135. Google Scholar
L. Mari, Measurement 17(2), 127 (1996), DOI: 10.1016/0263-2241(96)00022-X. Crossref, ISI, Google Scholar
L. Mari, Measurement 34(1), 17 (2003), DOI: 10.1016/S0263-2241(03)00016-2. Crossref, ISI, Google Scholar
J. G. McDaniel, Computers in Biology and Medicine 32(3), 127 (2002), DOI: 10.1016/S0010-4825(02)00010-0. Crossref, ISI, Google Scholar
S. Morasca, Foundations of a weak measurement-theoretic approach to software measurement, Proc. 6th Int. Conf. on Fundamental Approaches to Software Engineering pp. 200–215. Google Scholar
J. Münch and J. Heidrich, J. Systems and Software 70(1–2), 3 (2004). Crossref, Google Scholar
N. A. Mylonopoulos and V. Theoharakis, Commun. ACM 44(9), 29 (2001), DOI: 10.1145/383694.383701. Crossref, Google Scholar
R. Nelson and S. G. Winter , An Evolutionary Theory of Economic Change ( Belknap Press , Cambridge , 1982 ) . Google Scholar
P. Oman and S. L. Pfleeger (eds.) , Applying Software Metrics ( IEEE , Los Alamitos , 1997 ) . Google Scholar
J. S. Osmundsonet al., Information and Management 40(8), 799 (2003), DOI: 10.1016/S0378-7206(02)00114-3. Crossref, ISI, Google Scholar
P. Palviaet al., Commun. AIS 11, 289 (2003). Google Scholar
S. C. Palvia, R. S. Sharma and D. W. Conrath, Industrial Management and Data Systems 101(5), 237 (2001). Crossref, ISI, Google Scholar
D. Parnas, Commun. ACM 49(10), 29 (2006). Google Scholar
R. Pavur, M. Jayakumar and H. Clayton, Industrial Management and Data Systems 99(1), 5 (1999). Crossref, ISI, Google Scholar
J. M. Pearson, A. Pearson and J. P. Shim, Information Resources Management Journal 18(3), 50 (2005). Crossref, Google Scholar
A. Peled, Team Performance Management 6(1/2), 6 (2000), DOI: 10.1108/13527590010731934. Crossref, ISI, Google Scholar
S. L. Pfleeger, J. Systems and Software 31(2), 143 (1995), DOI: 10.1016/0164-1212(94)00094-4. Crossref, ISI, Google Scholar
M. T. Pich, C. H. Loch and A. De Meyer, Management Science 48(8), 1008 (2002), DOI: 10.1287/mnsc.48.8.1008.163. Crossref, ISI, Google Scholar
R. S. Pressman , Software Engineering: A Practitioner's Approach ( McGraw-Hill , New York , 2001 ) . Google Scholar
A. Rainer and T. Hall, J. Systems and Software 66(1), 7 (2003). Crossref, ISI, Google Scholar
B. Rameshet al., Commun. ACM 49(10), 41 (2006), DOI: 10.1145/1164394.1164418. Crossref, ISI, Google Scholar
T. Ravichandran and A. Rai, MIS Quarterly 24(3), 381 (2000), DOI: 10.2307/3250967. Crossref, ISI, Google Scholar
L. D. Richards and S. K. Gupta, J. Operational Research Society 36(9), 833 (1985), DOI: 10.2307/2582172. Crossref, ISI, Google Scholar
G. B. Rossi, Measurement 34(2), 85 (2003), DOI: 10.1016/S0263-2241(03)00026-5. Crossref, ISI, Google Scholar
F. Ruiz et al. , A proposal of a software measurement ontology , Proc. 4th Argentine Symposium of Software Engineering ( SADIO ) . Google Scholar
W. S. Sarle, Disseminations of the International Statistical Applications Institute, ed. W. S. Sarle (ACG, 1995) pp. 61–66. Google Scholar
H. Scarbrough, Rethinking Management Information Systems: An Interdisciplinary Perspective, eds. W. L. Currie and B. Galliers (Oxford University Press, New York, 1999) pp. 474–496. Google Scholar
F. L. Schmidt and J. E. Hunter, Intelligence 27(3), 183 (1999), DOI: 10.1016/S0160-2896(99)00024-0. Crossref, ISI, Google Scholar
A. H. Segars, Omega 25(1), 107 (1997), DOI: 10.1016/S0305-0483(96)00051-5. Crossref, ISI, Google Scholar
N. G. Shaw, Commun. AIS 11, 155 (2003). ISI, Google Scholar
H. J. Smith and M. Keil, Information Systems Journal 13(1), 69 (2003), DOI: 10.1046/j.1365-2575.2003.00139.x. Crossref, ISI, Google Scholar
I. Sommerville , Software Engineering ( Addison-Wesley , Harlow , 2001 ) . Google Scholar
I. Stameloset al., Information and Management 40(8), 729 (2003), DOI: 10.1016/S0378-7206(02)00099-X. Crossref, ISI, Google Scholar
P. Stein, Quality Progress 34(5), 72 (2001). Google Scholar
D. W. Straub, MIS Quarterly 13(2), 147 (1989), DOI: 10.2307/248922. Crossref, ISI, Google Scholar
D. W. Straubet al., Information Systems Research 13(3), 227 (2002), DOI: 10.1287/isre.13.3.227.80. Crossref, ISI, Google Scholar
P. A. Todd, J. D. Mckeen and R. B. Gallupe, MIS Quarterly 19(1), 1 (1995), DOI: 10.2307/249709. Crossref, ISI, Google Scholar
P. S. Tolbert and L. G. Zucker, Handbook of Organization Studies, eds. S. R. Clegg, C. Hardy and W. R. Nord (Sage, London, 1996) pp. 175–190. Google Scholar
P. Wagstrom and J. Herbsleb, Commun. ACM 49(10), 55 (2006), DOI: 10.1145/1164394.1164420. Crossref, Google Scholar
D. Wallace and L. Reeker, Guide to the Software Engineering Body of Knowledge — Trial Version 1.00, eds. A. Abranet al. (IEEE Computer Society Press, Los Alamitos, 2001) pp. 165–183. Google Scholar
I. Wieczorek, Empirical Software Engineering 7(2), 177 (2002), DOI: 10.1023/A:1015206216560. Crossref, Google Scholar
C. Wohlin and A. A. Andrews, Empirical Software Engineering 8(3), 285 (2003), DOI: 10.1023/A:1024476828183. Crossref, ISI, Google Scholar
M. H. Zack, Sloan Management Review 40(4), 45 (1999). Google Scholar
K. Zhu and K. L. Kraemer, Information Systems Research 13(3), 275 (2002), DOI: 10.1287/isre.13.3.275.82. Crossref, ISI, Google Scholar

Remember to check out the Most Cited Articles!
Check out our titles in C++ Programming!

Vol. 18, No. 01

Metrics

Downloaded 72 times

History

Received 25 December 2005

Revised 2 September 2006

Accepted 30 November 2006

Keywords

PDF download

System Upgrade on Mon, Jun 21st, 2021 at 1am (EDT)

MEASUREMENT IN SOFTWARE ENGINEERING: FROM THE ROADMAP TO THE CROSSROADS

Abstract

References

Recommended