Evaluating the Impact of Pair Testing on Team Productivity and Test Case Quality – A Controlled Experiment

Nosheen Qamar, Ali Afzal Malik


One of the main objectives of software testing is to uncover the maximum number of faults while consuming the least amount of resources. This research is an attempt to investigate the utility of an unconventional testing technique called pair testing in achieving this goal. In pair testing, two individuals sit together at one keyboard to test the software. An empirical study was designed and conducted to evaluate the performance of pair testing vis-à-vis conventional testing. Six pairs of testers divided into two different groups - one using pair testing and the other using conventional testing - participated in a controlled experiment involving three separate projects. The productivity of the groups and the quality of their work were quantitatively evaluated and compared. The results of comparison revealed that the group using pair testing spent more effort but the quality of its work was better.

Full Text:



Sommerville, I. (2011). Software Engineering, Addison-Wesley, USA.

Myers, G.J., Badgett, T., Thomas, T.M., Sandler, C. (2004). The Art of Software Testing, John Wiley & Sons, USA.

Kaner, C., Batch, J.,Pettichord, B. (2002). Lesson Learned in Software Testing, Addison-Wesley, New York.

Nidhra, S., Dondeti, J. (2012). Black box and White box Testing Techniques - A Literature Review, International Journal of Embedded Systems and Applications, Vol. 2, No. 2, pp.29-50.

Wong, W.E., Sugeta, T., Li, J.J., Maldonadoc, J.C. (2003). Coverage Testing Software Architectural Design in SDL, Computer Networks, Elsevier, Vol. 42, No. 3, pp. 274 –283.

Irena, J. (2009). Software Testing Methods and Techniques, IPSI BgD Transactions on Internet Research, Vol. 5, No. 1, pp. 30-41.

Bach, J. (2004). Exploratory Testing, The Testing Practitioner, UTN Publishers.

Beizer, B. (1990). Software Testing Techniques, Van Nostrand Reinhold.

Vaga, J., Amland, S. (2002). Managing High-Speed Web

Testing, Software Quality and Software Testing in Internet Times, Springer.

Lyndsay, J., Eeden, N.V. (2016). Adventures in Session-Based Testing . Available:http://www.workroom-productions.com/papers/ AiSBTv1.2.pdf.

Wood, B., James, D. (2003). Applying Session-Based Testing to Medical Software, Medical Device & Diagnostic Industry, Vol. 25, No. 5, pp.90.

Tuomikoski, J., Tervonen, I. (2009). Absorbing Software Testing into the Scrum Method, Proc. 10th Int’l Conf. Product-Focused Software Process Improvement, pp.15-17.

Jones, C. (2011). Software Quality in 2011: A survey of the State of the Art. Available: http://www.asq509.org/ht/a/GetDocumentAction/id/62711.

Lonetti F., Marchetti, E. (2018). Emerging Software Testing Technologies, Advances in Computers, Vol. 108, No. 1, pp. 91-143.

Santhanam P., Hailpern, B. (2002). Software Debugging, Testing, and Verification, IBM Systems Journal, Vol. 41, No. 2, pp.4-12.

Biscione F., Garzuoli, M. (2016). Pair Programming and Other Agile Techniques: An Overview and a Hands-on-Experience, Proc. 4th International Conference in Software Engineering for Defence Applications, Springer, pp. 87-101.

Nosek, J.T. (1998). The Case for Collaborative Programming, Communications of the ACM, Vol. 41 No. 3, pp. 105-108.

Williams, L., Kessler, R.R., Cunningham, W., Jeffries, R. (2000). Strengthening the Case for Pair Programming, IEEE Software, Vol. 17, pp. 19-25.

Lui, K.M., Chan, K.C.C. (2003). When Does a Pair Outperform Two Individuals, Proc. International Conference on Extreme Programming and Agile Processes in Software Engineering, pp. 225-233.

Müller, M.M. (2003). Are Reviews an Alternative to Pair Programming, Proc. 7th International Conference on Empirical Assessment in Software Engineering, pp. 335-351.

Williams, L. (2001). Integrating Pair Programming into a Software Development Process, Proc. 14th Conference on Software Engineering Education and Training, pp.19-21.

Williams, L., Shukla, A.,Antón, A.I. (2004). An Initial Exploration of the Relationship between Pair Programming and Brooks' Law, Proc. Agile Development Conference.

Jensen, R.W. (2003). A Pair Programming Experience, CrossTalk, Journal of Defense Software Engineering, Vol. 16, pp. 22 - 24.

Cockburn, A., Williams L. (2001). The Costs and Benefits of Pair Programming, Extreme Programming Examined, ACM Press, 2001.

Nawrocki, J., Wojciechowski, A. (2001). Experimental Evaluation of Pair Programming, Proc. 12th European Software Control and Metrics Conference, UK.

Bellini, E., Canfora, G., García, F., Piattini, M.,Visaggio, C.A. (2006). Pair Designing as Practice for Enforcing and Diffusing Designs Knowledge, Journal of Software Maintenance and Evolution, Vol. 17, No. 6, pp. 401-423.

Canfora, G., Cimitile, A., García, F., Piattini, M., Visaggio, C.A. (2006). Performances of Pair Designing on Software Evolution: AControlled Experiment, Proc. 10th European Conference on Software Maintenance and Reengineering, IEEE, pp. 198-205.

Canfora, G., Cimitile, A., Garcia, F., Piattini, M., Visaggio, C.A. (2007). Evaluating performances of Pair Designing in Industry, Journal of Systems and Software, Elsevier, Vol. 80, No. 8, pp. 1317–1327.

Ambler, S.W. (2002). Agile Modelling: Effective Practices for extreme Programming and the Unified Process, John Wiley & Sons.

Waqas, M., Malik, A.A., Qamar, N. (2016). Empirical Study on Pair Documentation, FAST-NU Research Journal, Vol. 2, No. 2, pp. 1-8.

Kohl, J. (2004). Pair Testing: How I Brought Developers into the Test Lab, Better Software Magazine. Available: http://www.kohl.ca/articles/pairtesting.pdf.

Harry, M.S. (2018). Measuring the Effectiveness of a Test. Available: http://www.mathematik.uni-ulm.de/sai/mayer/soqua04/slides/sneed.pdf.

Copyright (c) 2019 Pakistan Journal of Engineering and Applied Sciences

Powered By KICS