Spring 2002





Instructor: Dr. Leonard Adelman

            Office: S&T II, Room #325; Phone # 993‑1624

            Office Hours: Tuesdays, 6:30 ‑ 7:10 (or by appointment)

            E‑Mail Address: ladelman@gmu.edu



Adelman, L., & Riedel, S.L. (1997). Handbook for Evaluating Knowledge-Based Systems: Conceptual Framework and Compendium of Methods. Boston: Kluwer Academic Press.

Yin, R.K. (1994, 2nd ed.). Case Study Research: Design and Methods. Thousand Oaks, CA: Sage Publications.


Prerequisite: STAT 344 and 354 or equivalent or permission of instructor.


Evaluation is a critical control mechanism for system development. Its purpose is to provide feedback that developers can use to ensure that they provide a well-built system that is easy to use and that meets the users’ needs. To ensure that this purpose is met, evaluation needs to be integrated into the system development process, for different evaluation methods are required to answer the different types of questions inherent at different stages of development. This course overviews evaluation methods requiring the collection of empirical data. These methods include, for example, questionnaires, experiments, and case studies. These empirical methods also are used in many research studies testing hypotheses about how to use information technology to improve user performance.


There is a mid‑term exam, a final exam, and a student project. Each of the two exams is worth 30% of your grade; the student project is worth 20%. The exams will be based on questions that I handout in class. The questions will cover material presented in the texts and class. Since I will be teaching the class using a seminar format, class participation is critical to its successful implementation. Therefore, I will grade class participation after each class session. Class participation is worth 20% of your grade.


The purpose of the student project is to give students an opportunity to apply what they have learned in class to a real world problem. Feel free to use material from work and/or other classes. Just make sure that I can clearly see how you are effectively applying what you have learned in this class to your selected problem. The result of the project will be a 15-minute presentation (with viewgraphs) on the last day of class. There will not be a term paper unless there are too many students for presentations.






Week  1  (1/22)    Introduction to Course (Adelman & Riedel, Preface & Ch 1)


Week  2  (1/29)    Develop. Models, Evaluation Dimensions, & Eval. Models (A&R, Ch. 2)


Week  3  (2/5)      Evaluation Framework (A & R, Ch. 3)


Week  4  (2/12)    Requirements Validation Methods (A & R, Ch 4, pp 93-114)


Week  5  (2/19)    Requirements Validation continued (rest of Chapter 4)


Week  6  (2/26)    Knowledge Base Validation (A & R, Ch. 5, pp. 149-175)


Week  7  (3/5)      Knowledge Base Validation  cont. (rest of  Ch. 5) & Review for Midterm


Week  8  (3/12)    No Class - Spring Break


Week  9  (3/19)    Midterm Exam


Week  10 (3/26)   Review Midterm Exam and Usability Evaluation (A & R, Chapter 7)


Week 11 (4/2)      Performance Evaluation (A & R, Ch. 8)


Week  12 (4/9)     Case Studies: Introduction & Design Issues (Yin, Foreword and Chs.1 & 2)


Week  13 (4/16)   Conducting Case Studies (Yin, Chs. 3 & 4)


Week  14 (4/23)   Analyzing Case Study Evidence (Yin, Chs 5) and Review for Final Exam


Week  15  (4/30)  Student Presentations


Week  16  (5/14)  Final Exam (only on material after the mid‑term)

                             (Note: Tuesday, May 7th, is a “Reading Day.”)