HCI 460 Usability Evaluation Methods
This course describes a wide array of methods for evaluating the user experience of products and interfaces. We will discuss and practice methods such as heuristic and expert evaluations, cognitive walkthroughs, exploratory, ethnographies, usability testing (formative and summative), surveys, eye tracking, contextual inquiries and hybrid focus groups.
Goals to learn how to:
- Understand the role of usability evaluation
- Establish appropriate evaluation objectives
- Select evaluation methods that address evaluation objectives and take into account existing constraints
- Articulate advantages and disadvantages of usability evaluation methods
- Properly use various usability evaluation methods
- Present results and prepare effective report
Gavin S. Lew
Email: Gavin.Lew@boldinsight.com (both instructors will receive emails sent to this address)
Web page: http://boldinsight.com/hci460
Office Hours: Wed 5pm - 5:45pm and 9pm - 9:45pm in classroom (Loop Campus) scheduled via email
Handbook of Usability Testing by Rubin (first edition: ISBN 0-471-59403-2 or second edition: ISBN 0-470-18548-1)
Task-Centered User Interface Design: A Practical Introduction by Lewis and Rieman (online text)
- Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics by Tullis and Albert (ISBN 0-123-73558-0)
- Usability Inspection Methods by Nielsen and Mack (ISBN 0-471-01877-5)
- EYE TRACKING THE USER EXPERIENCE, Author: BOJKO, Publisher: ROSENFLD M, Edition: 13 (ISBN: 9781933820101)
- Various papers on eye tracking by Bold Insight
- Various papers on usability testing by Bold Insight
15% Project 1: Expert evaluation
25% Project 2: Formative usability study
15% Project 3: Quantitative comparison study
10% Take-home midterm quiz
25% Final exam
10% Individual contribution to projects
||100 - 93%
||92 - 90%
||89 - 87%
||86 - 83%
||82 - 80%
||79 - 77%
||76 - 73%
||72 - 70%
||69 - 67%
||66 - 60%
||59 - 0%
Elementary statistics (e.g., IT 223 or PSY 240)
Changes to Syllabus This syllabus is subject to change as necessary during the quarter. If a change occurs, it will be thoroughly addressed during class, posted under Announcements in D2L and sent via email.
Online Course Evaluations Evaluations are a way for students to provide valuable feedback regarding their instructor and the course. Detailed feedback will enable the instructor to continuously tailor teaching methods and course content to meet the learning goals of the course and the academic needs of the students. They are a requirement of the course and are key to continue to provide you with the highest quality of teaching.
The evaluations are anonymous; the instructor and administration do not track who entered what responses. A program is used to check if the student completed the evaluations, but the evaluation is completely separate from the student?s identity. Since 100% participation is our goal, students are sent periodic reminders over three weeks. Students do not receive reminders once they complete the evaluation. Please see https://resources.depaul.edu/teaching-commons/teaching/Pages/online- teaching-evaluations.aspx for additional information.
Academic Integrity and Plagiarism This course will be subject to the university's academic integrity policy. More information can be found at https://offices.depaul.edu/oaa/faculty-resources/teaching/academic-integrity/Pages/default.aspx.
Academic Policies All students are required to manage their class schedules each term in accordance with the deadlines for enrolling and withdrawing as indicated in the University Academic Calendar. Information on enrollment, withdrawal, grading and incompletes can be found at: http://www.cdm.depaul.edu/Current%20Students/Pages/PoliciesandProcedures.aspx
Incomplete Grades An incomplete grade is a special, temporary grade that may be assigned by an instructor when unforeseeable circumstances prevent a student from completing course requirements by the end of the term and when otherwise the student had a record of satisfactory progress in the course. All incomplete requests must be approved by the instructor of the course and a CDM Associate Dean. Only exceptions cases will receive such approval. Information about the Incomplete Grades policy can be found at http://www.cdm.depaul.edu/Current%20Students/Pages/Grading-Policies.aspx
Students with Disabilities DePaul University is committed to ensuring equal access to its educational and extracurricular opportunities for students with disabilities. The Center for Students with Disabilities (CSD) offers reasonable academic accommodations and services to support our students. We also serve as a resource to the many university departments that have a responsibility to accommodate students. 4 Please see https://offices.depaul.edu/student-affairs/about/departments/Pages/csd.aspx for Services and Contact Information.
Proctored exams for OL courses (if applicable) If you are an online learning student living in the Chicagoland area (within 30 miles of Chicago), you will need to come to one of DePaul's campuses to take an exam. Online learning students outside of the Chicagoland area are required to locate a proctor at a local library, college or university. You will need to take the exam within the window your instructor gives. Students should examine the course syllabus to find exam dates and the instructor's policy on make-up exams. Detailed information on proctored exams for online learning students can be found at http://www.cdm.depaul.edu/onlinelearning/Pages/Exams.aspx Online office hours for OL courses (if applicable) Faculty should be accessible to online students via phone, email and/or Skype.
Course Policies as Suggested by the Dean of Students Office Attendance: Students are expected to attend each class and to remain for the duration.
Class Discussion: Student participation in class discussions will be measured in two ways. First, students are highly encouraged to ask questions and offer comments relevant to the day?s topic.
Participation allows the instructor to ?hear? the student?s voice when grading papers. Secondly, students will be called upon by the instructor to offer comments related to the reading assignments. Students must keep up with the reading to participate in class discussion.
Attitude: A professional and academic attitude is expected throughout this course. Measurable examples of non-academic or unprofessional attitude include but are not limited to: talking to others when the instructor is speaking, mocking another?s opinion, cell phones ringing, emailing, texting or using the internet whether on a phone or computer. If any issues arise a student may be asked to leave the classroom. The professor will work with the Dean of Students Office to navigate such student issues.
Civil Discourse: DePaul University is a community that thrives on open discourse that challenges students, both intellectually and personally, to be Socially Responsible Leaders. It is the expectation that all dialogue in this course is civil and respectful of the dignity of each student. Any instances of disrespect or hostility can jeopardize a student?s ability to be successful in the course. The professor will partner with the Dean of Students Office to assist in managing such issues.
Cell Phones/On Call: If you bring a cell phone to class, it must be off or set to a silent mode. Should you need to answer a call during class, students must leave the room in an undisruptive manner. Out of respect to fellow students and the professor, texting is never allowable in class. If you are required to be on call as part of your job, please advise me at the start of the course.
WEEK 1: Course overview, overview of usability evaluation methods, heuristic evaluation, cognitive walkthrough, expert evaluation
WEEK 2: Expert evaluation continued, fundamentals of study design
How to write good findings and recommendations
Rubin ch. 1, Lewis and Rieman ch. 4 (except 4.2)
Project 1a - Expert evaluation: individual notes due
WEEK 3: Formative usability testing: How to prepare a study
Rubin ch. 2, 3, and 5
Project 1b - Expert evaluation: final report due
WEEK 4: Formative usability testing: How to conduct a study
Project 2a - Formative usability study: test plan due
WEEK 5: Formative usability testing: How to analyze findings, formulate recommendations, and write reports
Project 2b - Formative usability study: participant screening questionnaire and moderator's guide due
Take-home midterm distributed
WEEK 6: Teams will be conducting usability testing
Sample size. Intro to stats. Summative usability testing
Rubin ch. 10
Project 2c - Formative usability study: conducting the test
Take-home midterm DUE
WEEK 7: Summative usability testing
WEEK 8: Other usability evaluation methods:
Eye tracking, contextual inquiry, surveys, remote usability testing, focus groups
Project 2d -Formative usability study: report due
Project 3a - Quantitative comparison study: test plan
WEEK 9: Eye tracking. contextual inquiry, surveys, remote usability testing, focus groups, personas, etc.
WEEK 10: Overview of material for final exam (and get a proctor for the final exam
Human factors and medical devices and global research with some country examples
Project 3b - Quantitative comparison study: report due
Final exam on Week 11
NOTE: DL Students must secure a proctor or come in person to take the final. See course policies above. This schedule is tentative and subject to change as the course progresses.
This syllabus is subject to change as necessary during the quarter. If a change occurs, it will be thoroughly addressed during class, posted under Announcements in D2L and sent via email.
Evaluations are a way for students to provide valuable feedback regarding their instructor and the course. Detailed feedback will enable the instructor to continuously tailor teaching methods and course
content to meet the learning goals of the course and the academic needs of the students. They are a requirement of the course and are key to continue to provide you with the highest quality of teaching. The
evaluations are anonymous; the instructor and administration do not track who entered what responses. A program is used to check if the student completed the evaluations, but the evaluation is completely
separate from the student’s identity. Since 100% participation is our goal, students are sent periodic reminders over three weeks. Students do not receive reminders once they complete the evaluation.
Students complete the evaluation online in CampusConnect.
This course will be subject to the university's academic integrity policy. More information can be found at http://academicintegrity.depaul.edu/ If you
have any questions be sure to consult with your professor.
All students are required to manage their class schedules each term in accordance with the deadlines for enrolling and withdrawing as indicated in the University Academic Calendar. Information on enrollment, withdrawal, grading and incompletes can be found at http://www.cdm.depaul.edu/Current%20Students/Pages/PoliciesandProcedures.aspx.
Students who feel they may need an accommodation based on the impact of a disability should contact the instructor privately to discuss their specific needs. All discussions will remain confidential.
To ensure that you receive the most appropriate accommodation based on your needs, contact the instructor as early as possible in the quarter (preferably within the first week of class), and make sure that
you have contacted the Center for Students with Disabilities (CSD) at:
Lewis Center 1420, 25 East Jackson Blvd.
Phone number: (312)362-8002