FHTW Berlin

FHTW Berlin
Fachbereich 4
Internationale Medieninformatik
MMA I: Aktuelle Themen multimedialer Anwendungen
Sommersemester 2001
Exercise 5:
Test Preparation

We will continue today with preparations for the evaluation. The authors have prepared some comments to you as you begin the test, please read them. The software is available unter http://vfh.informatik.fh-ge.de/module/infophysik/look . The user code and password will be given to you in class.

  1. Look through the part of the software that you want to be testing. Note that you are not doing the test right now, but you are preparing for the test!.What sorts of things do you want to test? Write them down! For example, there are some Mini-Quiz boxes (Lerneinheit 2.1.1 "Geschwindigkeit und Beschleunigung", page 2/7). This box asks you a question that you need to answer. The correct answer is 60, but the box instructs you to always put in a decimal point and one digit after the decimal point. So you could want to test this with a wrong answer, with the right answer but in the wrong format, and the right format. This would be 3 little mini-tests that you conduct.
  2. Prepare a script for the test. What are you going to test? The script should be quite detailed and list the tasks that you will ask the tester to perform. You need to give a name to every concrete task that you will be doing. It is sufficient to have some sort of numbering system, T1, T2a, T2b, T3c, etc. For each task:

    This task list might look like the one Keith Andrews did for Harmony. The first test should be one that the user can do successfully, so that they don't think that they are stupid! The tasks should be in a realistic and typical order for using the software. If sequential ordering not crucial, then randomize the presentation order in order to counterbalance the learning effect. Put each task scenario on a separate sheet, you will hand them to the user one at a time! Example: Using the 3D landscape, navigate to the collection about the city of Graz and open it (please do not use a search query for this task). You will need at least as many task sequences as you have persons in your group. Prepare a few extra, just in case you are done early.

  3. What do you expect the reactions of the users to be for each concrete test? Assign shorthand codes to expected activities so that you have a list available. This will help you in recording what happens during the test.

  4. Now prepare your test plan. This is a written document that has the following sections:
    1. Purpose of the test
    2. Problem Statement
    3. Background Questionnaire (from last week)
    4. User Group Profile (you did this last week) and example personal scenarios
    5. Method (Test Design, what method did you decide on last week)
    6. Task List (this is from Nr. 2 today)
    7. Test Environment (from the last exercise)
    8. Data to be Collected (prepare a checklist and a form to fill out next time)

  5. When you are finished with a test series you will want to interview the test person. For this you will need either a questionnaire for the tester to fill out, or a debriefing topic guide, so that you don't forget to ask the user anything important. You will also need a checklist for the entire test - Prepare this now!

  6. Make sure that you have everything available that you will need next exercise:

    All materials should be printed and ready when you begin next week (please be on time!). This week I will be happy to just have the URL pointing to your materials. At least one member of the group needs to have a stopwatch or digital watch that counts seconds with them next time!

 

 

Results:

Debora Weber-Wulff (weberwu@fhtw-berlin.de)