Methodologies for Evaluation

The question of how to (or how not to) evaluate a new technique can be daunting.  LSMR has addressed this question directly in several ways.

 

Publications

  • Robert J. Walker and Reid Holmes. Simulation to evaluate recommendation systems for software engineering. Chapter 12 in Martin Robillard, Walid Maalej, Robert J. Walker, and Thomas Zimmermann, editors, Recommendation Systems for Software Engineering, Springer, 2014. doi: 10.1007/978-3-642-45135-5_12
  • Mehrdad Nurolahzade, Robert J. Walker, and Frank Maurer. An assessment of test-driven reuse: Pitfalls and promises. In Safe and Secure Software Reuse [Proceedings of the International Conference on Software Reuse (ICSR)], volume 7925 of Lecture Notes in Computer Science, pages 65–80, 2013. doi: 10.1007/978-3-642-38977-1_5 (Talk)
  • Bradley E. Cossette and Robert J. Walker. Seeking the ground truth: A retroactive study on the evolution and migration of software libraries. In Proceedings of the ACM SIGSOFT International Symposium on Foundations of Software Engineering (SIGSOFT/FSE 2012), Article 55, 11 pages, 2012. doi: 10.1145/2393596.2393661
  • Robert J. Walker, Shreya Rawal, and Jonathan Sillito. Do crosscutting concerns cause modularity problems? In Proceedings of the ACM SIGSOFT International Symposium on Foundations of Software Engineering (SIGSOFT/FSE 2012), Article 49, 11 pages, 2012. doi: 10.1145/2393596.2393654
  • Alessandro Garcia, Phil Greenwood, George Heineman, Robert Walker, Yuanfang Cai, Hong Yul Yang, Elisa Baniassad, Cristina Videira Lopes, Christa Schwanninger, and Jianjun Zhao. Assessment of contemporary modularization techniques—ACoM'07: Workshop report. Software Engineering Notes, 32(5), September 2007. 8 pages. doi: 10.1145/1290993.1291005
  • Robert J. Walker, Lionel C. Briand, David Notkin, Carolyn B. Seaman, and Walter F. Tichy. Empirical validation: What, why, when, and how. In Proceedings: 25th International Conference on Software Engineering (ICSE-25), pages 721–722, 2003. doi: 10.1145/776816.776922
  • Gail C. Murphy, Robert J. Walker, and Elisa L. A. Baniassad. Evaluating emerging software development technologies: Lessons learned from assessing aspect-oriented programming. IEEE Transactions on Software Engineering, 25(4):438–455, July/August 1999. doi: 10.1109/32.799936