19303401
Lecture
SoSe 15: Empirische Bewertung in der Informatik
Lutz Prechelt
Additional information / Pre-requisites
Comments
Description
As an engineering discipline, Informatics is constantly developing new artifacts such as methods, languages/notations or concrete software systems. In most cases, the functional efficiency and effectiveness of these solutions for the intended purpose is not obvious -- especially not in comparison to other already existing solutions for the same or similar purpose.
For this reason, methods for evaluating the efficacy of these solutions must be a routine part of Informatics -- a fact which unfortunately only slowly has become recognized. Evaluation is needed by those who create new solutions (that is in research and development), but also by the users, as these need to evaluate the expected efficacy specifically for their situation. These evaluations need to be empirical (that is based on observation), because the problems are nearly always too complicated for an analytical (that is a purely thought-based) approach.
This lecture presents the most important empirical evaluation methods and explains where these have been used (using examples) and should be used, how to use them and what to consider when doing so.
Homepage
http://www.inf.fu-berlin.de/w/SE/VorlesungEmpirie2015 closeSuggested reading
- Jacob Cohen: The Earth Is Round (p > .05). American Psychologist 49(12): 997003, 1994. Darrell Huff: How to lie with statistics, Penguin 1991.
- John C. Knight, Nancy G. Leveson: An Experimental Evaluation of the Assumption of Independence in Multi-Version Programming. IEEE Transactions on Software Engineering 12(1):9609, January 1986.
- John C. Knight, Nancy G. Leveson: A Reply to the Criticisms of the Knight and Leveson Experiment. Software Engineering Notes 15(1):24-35, January 1990.
- Audris Mockus, Roy T. Fielding, James D. Herbsleb: Two Case Studies of Open Source Software Development: Apache and Mozilla. ACM Transactions of Software Engineering and Methodology 11(3):309-346, July 2002.
- Timothy Lethbridge: What Knowledge Is Important to a Software Professional? IEEE Computer 33(5):44-50, May 2000.
- David A. Scanlan: Structured Flowcharts Outperform Pseudocode: An Experimental Comparison. IEEE Software 6(5):28-36, September 1989.
- Ben Shneiderman, Richard Mayer, Don McKay, Peter Heller: Experimental investigations of the utility of detailed flowcharts in programming. Commun. ACM 20(6):373-381, 1977.
- Lutz Prechelt, Barbara Unger-Lamprecht, Michael Philippsen, Walter F. Tichy: Two Controlled Experiments Assessing the Usefulness of Design Pattern Documentation in Program Maintenance. IEEE Transactions on Software Engineering 28(6):595-606, 2002.
- Lutz Prechelt. An Empirical Comparison of Seven Programming Languages: Computer 33(10):23-29, October 2000.
- Lutz Prechelt: An empirical comparison of C, C++, Java, Perl, Python, Rexx, and Tcl for a search/string-processing program. Technical Report 2000-5, March 2000.
- Tom DeMarco, Tim Lister: Programmer performance and the effects of the workplace. Proceedings of the 8th international conference on Software engineering. IEEE Computer Society Press, 268-272, 1985.
- John L. Henning: SPEC CPU2000: Measuring CPU Performance in the New Millennium. Computer 33(7):28-35, July 2000.
- Susan Elliot Sim, Steve Easterbrook, Richard C. Holt: Using Benchmarking to Advance Research: A Challenge to Software Engineering. Proceedings of the 25th International Conference on Software Engineering (ICSE'03). 2003.
- Ellen M. Voorhees, Donna Harman: Overview of the Eighth Text REtrieval Conference (TREC-8).
- Susan Elliott Sim, Richard C. Holt: The Ramp-Up Problem in Software Projects: A case Study of How Software Immigrants Naturalize. Proceedings of the 20th international conference on Software engineering, April 19-25, 1998, Kyoto, Japan: 361-370.
- Oliver Laitenberger, Thomas Beil, Thilo Schwinn: An Industrial Case Study to Examine a Non-Traditional Inspection Implementation for Requirements Specifications. Empirical Software Engineering 7(4): 345-374, 2002.
- Yatin Chawathe, Sylvia Ratnasamy, Lee Breslau, Nick Lanham, Scott Shenker: Making Gnutella-like P2P Systems Scalable. Proceedings of ACM SIGCOMM 2003. April 2003.
- Stephen G. Eick, Todd L. Graves, Alan F. Karr, J.S. Marron, Audris Mockus: Does Code Decay? Assessing the Evidence from Change Management Data. IEEE Transactions of Software Engineering 27(1):12, 2001.
- Chris Sauer, D. Ross Jeffrey, Lesley Land, Philip Yetton: The Effectiveness of Software Development Technical Reviews: A Behaviorally Motivated Program of Research. IEEE Transactions on Software Engineering 26(1):14, January 2000.
13 Class schedule
Additional appointments
Mon, 2015-07-20 10:00 - 12:00Klausur
Wed, 2015-10-14 16:00 - 18:00
Klausureinsicht Empirie
Regular appointments
Mon, 2015-04-13 10:00 - 12:00
Mon, 2015-04-20 10:00 - 12:00
Mon, 2015-04-27 10:00 - 12:00
Mon, 2015-05-04 10:00 - 12:00
Mon, 2015-05-11 10:00 - 12:00
Mon, 2015-05-18 10:00 - 12:00
Mon, 2015-06-01 10:00 - 12:00
Mon, 2015-06-08 10:00 - 12:00
Mon, 2015-06-15 10:00 - 12:00
Mon, 2015-06-22 10:00 - 12:00
Mon, 2015-06-29 10:00 - 12:00
Mon, 2015-07-06 10:00 - 12:00
Mon, 2015-07-13 10:00 - 12:00
Description
As an engineering discipline, Informatics is constantly developing new artifacts such as methods, languages/notations or concrete software systems. In most cases, the ... read more