FHTW Berlin Fachbereich 4 Internationale Medieninformatik MMA I: Aktuelle Themen multimedialer Anwendungen Sommersemester 2001

Today is evaluation day. You will be conducting a formal evaluation today. Your report is due on July 26, 2001, the week after the exam. All groups must be present and submit to me a printed copy of their final reports. Each group will then report verbally on the results that they had. When this is finished, we will distribute and discuss the exams.

1. Conduct the test according to your preparations! Take turns being the test person, the test facilitator and the test scribe. Take extensive notes during the test, don't forget to debrief the test person before switching roles.
2. Compile and analyse your data:
   • Compile and summarise data, for example: Mean, median, range, and standard deviation of completion times.
   • Percentage of users performing successfully.
   • Bar chart of preference scores. etc.
   • Identify errors and difficulties which arose.
   • Diagnose the source of each error.
   • Prioritise problems by their severity or criticality
3. Prepare your final report with the following sections:
• Title Page
• Description of Test Environment
  • Hardware, software version, test room, dates when tests were performed.

• Executive Summary
  • Concise summary of major findings, no more than a few pages.

• Description of Test
  • Updated test plan, method, training, and tasks.

• Test Person Data
  • Tabular summary of age, occupation, experience.

• Results
  • Tabular and graphical summaries of times taken, number of errors made, questionnaire responses, etc.
  • Discussion and analysis, amusing quotations.

• List of Positive Findings
• List of Recommendations List of problems discovered, in descending order of severity, and recommended improvements. For each recommendation:
  • diagnose why the problem occurred
  • illustrate it with a screen shot
  • rate its severity (0...4 scale)
  • indicate exactly how many test users experienced the problem
  • include a reference to timestamp(s) on the video tape
  • possibly include an appropriate user quotation
  • describe your suggested improvements

• Appendices (raw data and tables).
  • Background questionnaires, consent forms, orientation script, data collection forms, transcripts, etc.

• Time-Warner