Tuesday September 2, 1997
9:00 am - 12:30 pm

8. Systematic Evaluation of Hypermedia Applications

Franca Garzotto, Politecnico di Milano, and
Paoli Paolini, University of Lecce, Italy


  1. Course Objectives
  2. Intended Audience
  3. Course Description
  4. Course Structure

  1. Course Objectives
  2. This course will introduce the attendees to methods and criteria to critically read and systematically analyse, test and evaluate, hypermedia applications. After this course, participants will be able to look more rationally and critically at existing applications, rationalizing the good aspects of design choices, detecting inconsistencies and suggesting potential improvements; moreover attendees will be able to articulate more precisely requirements and design choices of hypermedia applications.

    The course is based upon the "reading" of a variety of hypermedia applications (approximately, from 4 to 6 commercial applications, 2 or 3 research prototypes, from 2 to 4 WWW applications) in a wide range of domains, including products for education and training, museums, tourism, entertainment, as well as hypermedia encyclopaedias, catalogues, interfaces to hypermedia data bases.

    Applications will be analysed according to a design-oriented evaluation method. Opposed to "user-oriented" evaluation commonly applied in usability testing, design-oriented evaluation primarily examines the internal strength of the design underlying an application. Design-oriented evaluation couples a systematic analysis of the application based on a hypermedia design model, with general usability criteria (such as consistency, richness, self-evidence, predictability), independent from the application area(s) and user tasks. Also relevant in the analysis is the correlation of the design choices with the "consumption" situations. In this course, HDM96 is used as design model, and the W5 model (Who, What, Why, When, Where) is used to describe fruition situations. The proposed method of analysis, however, is completely general, and other models could be used.

  3. Intended Audience
  4. Limited exposure to hypermedia applications is useful, but not strictly required.

  5. Course Description
  6. Initially some applications will be presented informally, to warm-up the audience and
    establishing a few general concepts. Next the state-of-art for evaluations of Hypermedia applications will be
    (very) briefly surveyed. Then usability criteria will be defined, and the necessary design concepts introduced. After explaining the W5 model, to describe fruition situations for hypermedia applications, the features of design oriented evaluations will be introduced.

    The core of the course will be the "reading" and analysis of hypermedia applications (from 8 to 13, according to the taste and reactions of the attendees). For each applications the different aspects of design will be analyzed, and the consistency with the fruition situation on one side, and the implementation on the other side, discussed. The goal is twofold: to detect good aspects and choices, possibly to imitate; to detect weaknesses and "bugs", certainly to avoid.

    An interesting exercise is to guess why weakness, inconsistencies and bugs appear so often, even in commercial application: the origin can often be traced to weakness in the design, development and testing process. A discussion with the attendees will close this course (that, whenever has been offered, has solicited the interest for "more reading" from the attendees).

  7. Course Structure
    1. warm-up demonstrations of hypermedia applications
    2. (showing good and bad design choices)
    3. state of the art in Hypermedia evaluation
    4. usability criteria for hypermedia applications
    5. design concepts and terminology
    6. the W5 model
    7. design oriented evaluation
    8. analysis of applications (from 4 to 6 commercial applications, 2 or 3 research prototypes and from 2 to 4 WWW applications); for each application analysis of : Hyperbase in-the-large, Access and Hyperbase in-the-small (structure, dynamics, interaction, lay-out) comments on the design choices and consistency between design and implementation (strong points and weak points or "bugs")
    9. application analysis revisited: interpreting the "bugs" in terms of the
    10. designer intention
    11. conclusions and discussion with participants

This page is located below www.archimuse.com/ichim97.
To request further information or send comments, contact info 'at' archimuse.com
Last Updated: July 31, 1997.