HCI 541 Survey Design and Analysis
Summary
Students design and analyze surveys with an emphasis on user experience research. Benefits and costs of using surveys are compared to other UX methods. Students identify threats to validity and factors for the appropriate use of surveys. Course activities include survey question design, pilot testing, sampling, implementation, and analysis. Students use statistical software and survey software such as Qualtrics.
Major expectations of the course include weekly participation in ad-hoc group exercises and contribution to a team-based project. Participation and contributions may be conducted in-class (in-person or by zoom), by zoom outside of class, or asynchronously.
Course Goals
At the end of the course, students will be able to do the following:
- Identify user research scenarios where survey methods are appropriate.
- Identify threats to validity.
- Design surveys following best practices.
- Pilot test surveys.
- Implement and administer a survey using software tools such as Qualtrics
- Perform statistical analysis and report findings.
Texts
Required Texts
Survey Research Methods, 5th Edition (2014)
by Floyd J. Fowler Publisher: Sage
Selected readings
Grading
Grade Determination
30% |
2 Individual Projects (15% each) |
20% |
2 Team Projects (10% each) |
10% |
Quizzes (5% each) |
20% |
Summary Report (Individual submission) |
10% |
Weekly Participation |
10% |
Team Contribution |
Assignments and projects will be generally graded with a letter grade and converted to a 10-point scale (10 for an A, 9.5 for A/A-, 9 for an A-, 8.5 for a B, 8 for a B-, etc). Students receiving more than 95% of possible points are guaranteed at least an A, more than 90% at least an A-, more than 80% at least a B-, more than 70% at least a C-, and more than 60% at least a D.
The summary report is an individual submission due during finals week. It includes original insights, reflections or experiences for each of the phases in the survey design and implementation process. It takes the place of a final exam.
The score for weekly participation is based on work completed within an ad-hoc group. Every week, the class has a hands-on exercise involving group discussion. Group discussion may occur in-person, by zoom, or asynchronously online. Credit is achieved by being included in a group-based summary of the experience, which must be submitted by Monday noon of the following week for full credit. Generally, only group-based participation will receive credit. Credit for individual participation requires prior authorization and a rationale documenting exceptional circumstances. This participation is generally not graded but formally noted on a weekly basis. Each weekly participation counts as a point in a 10-point scale. A point may be added for individuals taking exceptional initiative in leading discussions and posting summaries. A point will be deducted for those showing no intiative. Supplemental contribution points can be earned by participating in the CDM subject pool (1 additional point for each whole credit of participation, up to 2 additional points but capped to an overall score of 11 points).
The score for team contribution is based on documented contributions in team reports, observed team activities (in class, email or group message boards) and team member reports at the end of the quarter. Students who make solid contributions to their team projects receive at least an A- (9 points) of the contribution points. Exceptional contributions may receive an A or an A+ (11 points).
Team presentations occur in the last week of class. All students should contribute to the presentation, although all members do not need to be physically present for the presentation. At the end of the quarter every student is expected to submit a completed evaluation form for each group member.
Team projects will generally be completed in groups of three, four or five. To form groups, students review student-submitted proposals and submit preferences to work on them. Groups will then be formed based on the preferences.
For team projects, all team members are expected to contribute to the project's activities and summary report. Generally, each team member will receive the same score on each project. However, in some cases, additional credit may go to those who make an exceptional contribution to a project and reduced credit to those who contribute little to a project. Any adjustment will be based on a variety of indications including group participation in class, contribution summaries in reports, and student feedback at the end of the quarter.
Policies
Students are expected to attend every class or watch the asynchronous recording.
Late assignments and projects will be accepted up to 3 days late, with a half point penalty. Assignments submitted later will generally not be accepted. Exceptions require prior authorization and a rationale documenting exceptional circumstances.
Additional assignments for extra credit will not be offered.
All grade challenges must be submitted in writing and include an explanation why the given score or grade should be reconsidered.
Prerequisites
HCI 445 or DSC 443; IT 403 or comparable statistics course is assumed for this course.
Tentative Schedule
Week |
Topic |
Reading |
Activity |
Assignment |
Jan 6 |
Course Overview; Survey properties |
Fowler ch. 1 |
Discuss Participation Survey |
|
Jan 13 |
Survey development process |
Reading on cognitive interviews |
Discuss examples |
Assignment 1 |
Jan 20 |
Threats to validity |
Fowler ch. 2 |
Cognitive Interview Practice |
|
Jan 27 |
Sampling; non-response bias |
Fowler ch. 3 and 4 |
Validity Examples |
Assignment 2 |
Feb 3 |
Method and question design |
Fowler ch. 5 and 6 |
Bias exercise |
Quiz 1 |
Feb 10 |
Survey evaluation |
Fowler ch. 7 |
Question design |
Team Project 1 |
Feb 17 |
Survey implementation and administration |
Fowler ch. 8 |
Evaluation lab |
|
Feb 24 |
Survey analysis |
Fowler 9 and 10 |
Qualtrics Lab |
Team Project 2 |
Mar 3 |
Variants to survey methods |
TBD |
Analysis exercise |
Quiz 2 |
Mar 10 |
Project presentations, review |
Course notes and presentations |
Informal presentations |
|
Mar 17 |
|
|
|
Final Report |
This syllabus is subject to change as necessary during the quarter. If a change occurs, it will be thoroughly addressed during class, posted under Announcements in D2L and sent via email.
Evaluations are a way for students to provide valuable feedback regarding their instructor and the course. Detailed feedback will enable the instructor to continuously tailor teaching methods and course
content to meet the learning goals of the course and the academic needs of the students. They are a requirement of the course and are key to continue to provide you with the highest quality of teaching. The
evaluations are anonymous; the instructor and administration do not track who entered what responses. A program is used to check if the student completed the evaluations, but the evaluation is completely
separate from the student’s identity. Since 100% participation is our goal, students are sent periodic reminders over three weeks. Students do not receive reminders once they complete the evaluation.
Students complete the evaluation online in CampusConnect.
This course will be subject to the university's academic integrity policy. More information can be found at http://academicintegrity.depaul.edu/ If you
have any questions be sure to consult with your professor.
All students are expected to abide by the University's Academic Integrity Policy which prohibits cheating and other misconduct in student coursework. Publicly sharing or posting online any prior or current materials from this course (including exam questions or answers), is considered to be providing unauthorized assistance prohibited by the policy. Both students who share/post and students who access or use such materials are considered to be cheating under the Policy and will be subject to sanctions for violations of Academic Integrity.
All students are required to manage their class schedules each term in accordance with the deadlines for enrolling and withdrawing as indicated in the University Academic Calendar. Information on enrollment, withdrawal, grading and incompletes can be found at http://www.cdm.depaul.edu/Current%20Students/Pages/PoliciesandProcedures.aspx.
Students who feel they may need an accommodation based on the impact of a disability should contact the instructor privately to discuss their specific needs. All discussions will remain confidential.
To ensure that you receive the most appropriate accommodation based on your needs, contact the instructor as early as possible in the quarter (preferably within the first week of class), and make sure that
you have contacted the Center for Students with Disabilities (CSD) at:
Lewis Center 1420, 25 East Jackson Blvd.
Phone number: (312)362-8002
Fax: (312)362-6544
TTY: (773)325.7296