The Best of the Web Judges will review sites in nominated categories, and rate them according to the following Evaluation Criteria.
Version 1.0 - J. Trant, September 2, 2005
(pending input from the MW2006 Judges)
The Categories
The Best of the Web judges consider Web sites nominated in each of the following categories: On-line Exhibition, E-Services or E-Commerce , Educational Use, Innovative or Experimental Application, Museum Professional's Site, Research Site. In addition, the Judges select the Best Overall Museum Site, from all sites nominated.
The Judging Process
Each judge will review sites in a particular category (or categories) and then participate in the selection of the Best of the Web. Criteria for evaluation are outlined below.
First Stage: Preliminary Category Review
Deadline: January 31, 2006
- two judges review each nominated site, providing rankings for each of the identified criteria
- where there are a large number of sites nominated in a category, preliminary review will be divided among the judges for the category.
- each judge will review approximately 10 (ten) web sites.
In the Preliminary Review:
- sites are assigned points (from 0 to 5) in each evaluation category
- sites are flagged for the category short list
- a maximum of 5 sites can be short-listed for any category; these are the semi-finalists.
Second Stage: Semi-Final Category Winners
Deadline: February 28, 2006
- all category judges review the short list of sites
- all sites are ranked again
- a category winner is decided
- possible best of the web sites are identified (finalist)
Third Stage: Best of the Web
Deadline: March 15, 2006
- all judges review the best from each category and sites identified as possible bests
- all possible best of the web sites are finalists
- all judges rank all finalists in all categories.
- Best of the Web is identified
- honorable mentions are identified
Evaluation Criteria
All sites will be evaluated using the same set of criteria. Judges will assign a score from zero (0) to five (5) points in each of the following areas, to create a total score out of twenty-five (25). In addition, judges will offer written comments on the sites.
--Content--
Reflect on the information or experience delivered by the site. Was the content or experience offered:
- engaging?
- compelling?
- interesting?
- audience-appropriate?
- current?
- regularly updated?
- open to user contributions?
- relevant to and supportive of user goals?
--Functionality--
Assess the choice of technology and functions used to deliver the site's content and build the site's construction.
Was the technology chosen:
- well executed? (did it work?)
- accessible?
- appropriate to the content?
- appropriate to the user?
- supportive of interaction with the content?
- supportive of the user experience (search, print)?
--Interface: Visual Design and Usability--
Consider the way that the site was presented visually. How was the site designed? Was the visual presentation of the site:
- visually appealing?
- consistent?
- supportive of user tasks?
- understandable?
- sympathetic to the content?
- appropriate to the target user?
- accessible?
--Interactivity--
Review the ways in which the site took advantage of the Web, explored relationships between objects or ideas, and encouraged the user to engage with the content presented, with the sponsoring institution, and/or with other users. Did the site:
- provide appropriate links amongst related content and content areas?
- encourage user input?
- support interaction amongst users?
- enable contribution of user content?
- indicate how to contact the institution?
- remain accessible?
--Overall--
Consider the impact the site had on you.
- Was your experience memorable?
- Did you want to go back?
- Did you stay a long time?
- Did you have fun?
- Did it make you smile or think?
Sources Consulted
CIDOC Multimedia Working Group, Multimedia Evaluation Criteria, 199.7 Revised Draft, J. Trant, Chair. Available http://www.archimuse.com/cidoc/
Webby Awards, Judging Criteria. 2004. Available http://www.webbyawards.com/webys/criteria.php
Public History Resource Center.
. Evaluating Web Sites. 2000.Debra DeRuyver, Jennifer Evans, James Melzer and Emma Wilmer. Available
http://www.publichistory.org/evaluation/index2.html
. Rating System for Evaluating Public History Web Sites. Debra DeRuyver, Jennifer Evans, James Melzner and Emma Wilmer. April 30, 2000. Available
http://www.publichistory.org/reviews/rating_system.html
Carleton Center for Public History. Canadian History Website Reviews - Notes for contributors. (n.d.) Available: http://www.carleton.ca/canweb/notestocont.html
