Design a study to assess the efficacy of your interactive system. The components of the study include: tasks, variables, data to be collected, and data analysis method. Begin the study design process by re-articulating the research questions you are asking.
Some components of the study are formative, that is, their goal is to inform the next iteration of the design. Here, you may compare, for example, people's ability to use different interfaces to accomplish the same task.
Other study components will be summative, that is, their goal is to assess the system as a whole. Is it effective? Why? How? Develop one more hypotheses that you will seek to prove through the user study. To assess summative components, you may devise a control condition, which involves participants in some conditions. conducting a task without your system at all. The design of the tasks/activities, themselves, is crucial.
Consider carefully: what data will you collect? Will you instrument your program to produce logs? (Interface Ecology Lab logging framework can help.) Will you measure the time it takes people to perform a task? Their errors? Will you record them using the system and somehow code the data? Will you gather qualitative data, through observation and interviews? How will you analyze it?
Work with your graduate student mentors to refine a study plan. Focus your resources to conduct the best practical investigation. Make sure they approve it before you begin.
Turn in a clear statement of:
- apparatus: briefly, what did the participants use to perform the study?
- participants: how many? what background?
- what data was collected? how?
- results: what does the data show?
- analysis: what do the results mean? what are the implications for design?
- future work: what further study would you conduct if you had more time?
Formulate the research questions as a team. Then, divide up the tasks.