FHTW Berlin / Fachbereich 4 Internationale Medieninformatik MMA: Web Usability Sommersemester 2005 Prof. Dr. Debora Weber-Wulff

We will continue today with preparations for the evaluation.

Week 1

1. Choose your library from the list below. Each team in each group will be doing one library. Note that you are not doing the test right now, but you are preparing for the test! You now have 2 weeks to obtain a library card for the library so that you can do the test online. Keep track of what you have to do in order to obtain a library card. What information to you have to volunteer? If you have to pay for the library card, get a receipt and give it to me, I will reimburse you for the costs that are incurred. Stabi groups only purchase a monthly card, not a full year!
2. What sorts of things do you want to test? We call these tasks. Write them down! For example, there might be field for logging in - what happens when you put in a wrong answer? This is a mini-test and is not the same as a scenario!
3. Prepare a script for the test. Which scenarios and mini-tests are you going to use? The script should be quite detailed and list the tasks that you will ask the tester to perform. You need to give a name to every concrete task that you will be doing. It is sufficient to have some sort of numbering system, T1, T2a, T2b, T3c, etc. For each task:
   
   • Define any prerequisites.
   • Define successful completion criteria.
   • Specify maximum time to complete each task, after which help may be given.
   • Define what constitutes an error.

   This task list might look like the one Keith Andrews did for Harmony. The first test should be one that the user can do successfully, so that they don't think that they are stupid! The tasks should be in a realistic and typical order for using the software. If sequential ordering not crucial, then randomize the presentation order in order to counterbalance the learning effect. Put each task scenario on a separate sheet, you will hand them to the user one at a time! Example: Using the 3D landscape, navigate to the collection about the city of Graz and open it (please do not use a search query for this task). You will need at least as many task sequences as you have persons in your group. Prepare a few extra, just in case you are done early.

Week 2

1. What do you expect the reactions of the users to be for each concrete test? Assign shorthand codes to expected activities so that you have a list available. This will help you in recording what happens during the test.

2. Now prepare your test plan. This is a written document that has the following sections:
   
   1. Purpose of the test
   2. Problem Statement
   3. Background Questionnaire (from last week)
   4. User Group Profile (you did this last week) and example personal scenarios
   5. Method (Test Design, what method did you decide on last week)
   6. Task List (this is from Nr. 2 today)
   7. Test Environment (from the last exercise)
   8. Data to be Collected (prepare a checklist and a form to fill out during the formal evaluation)
      
3. When you are finished with a test series you will want to interview the test person. For this you will need either a questionnaire for the tester to fill out, or a debriefing topic guide, so that you don't forget to ask the user anything important. You will also need a checklist for the entire test - Prepare this now!
   
4. Make sure that you have everything available that you will need next exercise:
   • Test Materials
   • Nondisclosure Form
   • Orientation Script
   • Training Script (if any)
   • Task Scenarios
   • Data Collection Forms
   • Post Test Questionnaire or Debriefing Topic Guide
   • Checklist

All materials should be printed and ready when you begin the formal evaluation (please be on time!). Send the URL pointing to your materials to the teacher or post them in the forums area of Moodle before the formal evaluation begins. At least one member of the group needs to have a stopwatch or digital watch that counts seconds with them for the formal evaluation!

Library list

Bibliothek	OPAC	URL
Staatsbibliothek Preußischer Kulturbesitz	StabiKat / BIBER	http://stabikat.staatsbibliothek-berlin.de/
Freie Universität Berlin	ALEPH	http://opac.fu-berlin.de
Technische Universität Berlin	ALEPH	http://opac.ub.tu-berlin.de/
Humboldt Universität zu Berlin	ALEPH	http://opac.hu-berlin.de/
Fachhochschule für Technik und Wirtschaft	SISIS	Long URL
Öffentliche Bibliotheken Berlin	aDIS/BMS	https://www.voebb.de/
Digitale Bibliothek Berlin Brandenburg	kobv	http://digibib.kobv.de/