/mw/














A&MI home

158 Lee Avenue
Toronto Ontario
M4E 2P3 Canada

ph: +1 416-691-2516
fx: +1 416-352-6025

info @ archimuse.com
www.archimuse.com

Join our Mailing List.
Privacy.

 

 

published: March 2004
analytic scripts updated:  October 28, 2010

Creative Commons Attribution-Noncommercial-No Derivative Works 3.0  License

 

Museums and the Web 2003 Papers

Making Learning Fun: Plimoth Plantation's On-line Learning Center

Lisa Neal, eLearn Magazine; Kim Van Wormer, Plimoth Plantation,  USA

Abstract

Plimoth Plantation, a living history museum, sought to provide on-line learning that, while not replicating a physical visit, provided a rich and compelling educational experience. Plimoth Plantation's You Are the Historian on-line learning center (OLC) teaches about the events that later became known in the US as The First Thanksgiving in order to help children and teachers meet social studies standards. In designing the OLC, we grappled with how to develop engaging, interactive materials which encouraged exploration and learning. We started off trying to find an appropriate balance between learning and fun. We soon realized, however, that it wasn't a question of balancing learning and fun, but rather making learning fun. We describe the goals for the OLC, the design and development, and the impact of formative and heuristic evaluations and usability testing.

Keywords: on-line learning, museums, history, children

Introduction

Museums increasingly use on-line technologies to help people prepare for a visit, view or learn about a collection, or take a virtual tour (Carliner; Maxwell; Tedeschi, 2003). Besides their collections, museums house extensive expertise used to plan exhibits, write publications, and guide tours. In order to harness this expertise, museums are more and more commonly providing on-line learning  through kiosks for use by physical visitors and/or over the Web for virtual visitors.

Many museum Web sites depict collections, and the few that venture to provide a more experiential visit usually do so through a virtual tour. Museums and other educational sites and software that provide on-line activities often include games that are fun but not especially educational or purposeful. For example, in addition to educational materials, a Web site about early US explorers includes a game in which children shoot at buffalo running across the Plains. It is not easy to strike a balance between learning and fun in the classroom, in museums, or on-line. When fun is overemphasized, children focus more on the gaming and little learning results. The optimal educational impact is achieved when learning becomes fun.

Plimoth Plantation

Plimoth Plantation, located in Plymouth, Massachusetts, is the living history museum of the English colonists, the Wampanoag Native People, and the groups' interaction in 1620s New England (see Figure 1). The museum's mission is to offer"powerful personal experiences of history, built upon thorough research of the Pilgrim and Wampanoag communities. The museum offers multiple learning opportunities to provide a deeper understanding of the relationship of historical events to modern America." To achieve this mission, the museum employs immersive, recreated environments where visitors can encounter colonial"interpreters" who are dressed in period costumes, speak in dialect, and talk about their lives, and Native American interpreters who talk about and demonstrate Native life in the past and today.

overall view of plimoth plantation

Figure 1: Plimoth Plantation

Design of the On-line Learning Center

Plimoth Plantation received Federal and private foundation funding to provide standards-based on-line education for children and teachers. The team of museum staff, teachers, and external consultants that was formed to accomplish this decided to focus the OLC on The First Thanksgiving, which the museum more accurately refers to as the 1621 harvest celebration. One goal for the OLC was to educate children and teachers about the origins and evolution of the Thanksgiving holiday, using new research taken from the multiple perspectives of English and Native American sources. Another goal was to show the abiding impact of 17th Century events on modern culture, in particular, how the Wampanoag People and the early Colonists who settled in the US met and lived together.

One of the biggest challenges we faced was how to bring the powerful, personal experience of physical visitors to the on-line experience to reflect the unique characteristics of Plimoth Plantation. Another challenge was to design a site that was appealing, engaging, fun, and educational for children. That was not always easy even in the physical museum, and was potentially more challenging on-line. Fogg (2003) finds the most credible products rank highly on both trustworthiness and expertise. It was essential that the OLC reflect the Plimoth Plantation 'brand' and offer credibility.

Teaching for Understanding

We decided early on to design the OLC using a pedagogical framework called Teaching for Understanding (TfU) (Perkins, 2003). TfU is a curriculum design tool developed at the Harvard Graduate School of Education. TfU is based on the philosophy that understanding is built over time through a series of active performances of understanding. Students move through Introductory, Guided Inquiry and, ultimately, Culminating Performances to reach identified Understanding Goals.

The OLC team, which included teachers trained in TfU, developed Understanding Goals that were tied to the 3rd and 5th social studies standards for elementary school children in the Commonwealth of Massachusetts and the US. The Understanding Goals drove the design of the OLC, ensuring that on-line activities were relevant and led to increased student understanding. The Understanding Goals were reported in an on-line Teacher's Guide and a companion print version. The Teacher's Guide included guidelines for incorporating the OLC into the classroom and computer lab as well as classroom activities and resources.

You Are the Historian

While it is important that children learn facts to meet social studies standards, it is arguably more important that they develop an understanding of how historians learn about and interpret history. The OLC team weighed the importance of this notion while noting the success of goal-based and scenario-based learning, where the learner is given a problem to solve (Schank, 1997). Future educational software will not present a stream of information to a student but will place the student in an environment where the information needed for success will be actively sought and learned (Downes, 1998).

The popularity of movies such as Spy Kids and Harriet the Spy indicated that children as spies and problem-solvers appealed to children. Therefore, we decided to challenge visitors to the OLC to become historians. In this role, they could investigate what really happened in 1621 by interpreting information about people and events, rather than just by reading text or performing isolated activities. This approach leads to personalized experiences for children (Miyagawa, 2002) and engages them in activities that provide an authentic (i.e., meaningful) setting for learning (Kearsley) .

As historians, children learn how historians use primary sources and oral histories, and how historical events are interpreted and reinterpreted over time. After completing the activities on the site, rather than taking the multiple choice quiz common in on-line learning, children are asked to develop an on-line museum exhibit of captioned images. This goal-based approach is motivating to children since they work toward a defined goal and have tangible results which can be printed and shared. This approach is advocated by TfU since children engage in a culminating performance to demonstrate their understanding.

Children as guides and interpreters

A visitor to Plimoth Plantation encounters colonial and Native American interpreters who provide information. Since our target learner population was children, we thought that children would identify more readily with and be more curious about other children. Further, we thought that photographed children would be more in keeping with a living history museum than the animated characters commonly used on Web sites for children.

The home page introduces two children, a Wampanoag boy and a colonial girl. These children act as guides in their modern attire and as interpreters in their 1621 attire. In the initial screen, the children morph from their modern (Figure 2) to their 1621 personas (Figure 3), providing a sense of mystery. The OLC team thought this sense of mystery, which continues in the site with audio, hidden information, and surprises and rewards, would be appealing to children, considering the phenomenal popularity of Harry Potter books and movies. Throughout the site, the modern children offer information and guidance in text on the left of the screen, including a historian's task for the user to accomplish. The 1621 children provide information about their lives as well as instructions about what a child can do on the page.

children in modern dress

Figure 2: Modern children on home page

children in native / historical dress

Figure 3: 1621 Children on home page

“'Visit the Expert'

“Visit the Expert" provides information that would rarely be available to an actual visitor, and provides the user with a deeper understanding of the work of historians. This feature uses short audio clips supplemented by text and a photograph (Figure 4).

Thanksgiving? Fact or Myth

Figure 4: Visit the Expert on 'Fact or Myth' page

Engagement and Fun

The OLC team critiqued many on-line courses and Web sites for children from museums and other sources, looking for what was appealing, engaging, fun, and educational. We also scoured the literature seeking guidelines, consulted with experts, and requested assistance on listservs, including on CHI-KIDS. In our discussions about which approaches to take, we factored in the importance of both representing the unique qualities of Plimoth Plantation and meeting the Understanding Goals. We also considered the multiple stakeholder perspectives: children were the target users and educators were the target customers since they would decide whether to adopt the OLC in the classroom. The other user populations, such as funders, parents, homeschoolers, and the general public, were less of a focus since we anticipated their needs would be met if we successfully met the needs of children and teachers.

There were some commonalities in the on-line courses and Web sites that we liked. Even though they varied considerably in their design, they drew the user in and had an emotional appeal. However, 'fun and pleasure are elusive concepts' (Norman, 2004), and there is no consensus on how to design enjoyable experiences (Monk et al., 2002). Norman discusses the importance of emotion in design, saying that you actually think differently when you are anxious than when you are happy…devices and software should be designed to influence the mood of the user; they will be more effective because they are more affective (Gibbs, 2004).

Engagement is accepted as important in on-line learning but is similarly elusive. A rubric used for evaluating the quality of Web sites for children specifies ten criteria: explanations are offered for all of them except for engaging, which is circularly defined as process engages the learner (Blue).

Some techniques evolved in addition to challenging children to conduct an investigation as an historian, using children as guides and interpreters, and including expert opinions. These included the extensive use of graphics, audio, and video to provide visual and auditory richness; purposeful and engaging activities; variety; and rewards and surprises. These techniques both borrowed from the best examples we found and avoided what seemed ineffective.

Specifically, we wanted to avoid the pitfall of much on-line learning: linear, heavily text-based courses offering little, if any, interactivity. Additionally, we found many activities that were educational but not fun, or ones like the buffalo-shooting game mentioned above, which was fun but neither educational nor seemingly relevant to the site's educational goals. A successful educational game leads to deeper understanding of the complexity of decision-making, enjoyment, and provides a context in which to teach specific topics after the game has been played (Klassen and Willoughby, 2003).

Even within the physical museum environment there is a struggle to balance learning and fun during school field trips. Scavenger hunts, for example, can easily turn into races. The discussions about how to find a balance between learning and fun led to a realization that our goal was ultimately to make learning fun. While this goal is important in the design of learning materials for all ages, it is especially important for children.

Extensive visual and auditory components were considered important to draw children in and to capture the essence of Plimoth Plantation. The site uses a rich color palette, a carefully chosen font, images, photographs, video, and extensive audio. The latter includes background sounds such as waves, sounds to accompany an action, and audio clips. Most text, other than instructions, has audio that plays automatically when a page is opened; this was important to help children who are not strong readers. Visit the Expert and other audio clips captured the voices of museum staff, and some audio was in authentic dialect.

Loading pages

Long loading time is generally considered detrimental since it breaks the flow of using a site. Because of the audio and video, an OLC user on a dial-up connection would have to wait while a page was loading. Instead of merely using a progress indicator, we decided to set the tone for the page being loaded with a relevant question or riddle. For example, the Primary Source loading page asks:

What do these four things have in common?

  • The spoken language of the Wampanoag
  • A letter written by Edward Winslow to a friend in 1621
  • A photograph of man's first step on the moon
  • A postcard about your vacation at the beach.

The answer, when selected, is They are all primary sources. An indicator shows the loading progress and, when it is complete, an Enter link appears. On the Primary Source loading page, the background text of Primary Source and the evidence? further introduces the theme.

Dix (2003) found that experiences that were neither fun nor engaging (basically boring!) could be made fun, even waiting for a kettle to boil. The loading pages evolved from a necessity into a beneficial, useful, and enjoyable feature of the site that encourages reflection.

Exploration

Much on-line learning is developed by converting print-based resources. Such courses are linear in design, neither taking advantage of the potential of a non-linear on-line teaching environment nor matching the emergent learning approaches of a generation who are comfortable in the on-line environment (Phelps, 2003). The OLC takes a more exploratory approach to encourage personalized learning, starting with the navigation (Figure 3), which, while linear in structure, does not mandate a linear path. Each OLC page provides layers of information as well as hidden information and directions that can be discovered by moving the mouse over text (known as mouseovers). While the text on a page offers guidance as to what can be done on that page, we realized that many children would not read the text, or not read it initially, and we wanted to encourage unguided and prescribed exploration.

The English Colonists page has the most layers and mouseovers, starting with a photograph of the Plimoth Plantation village (Figure 5). When one of the houses is selected (through a hidden link), the next screen shows the interior of a home (Figure 6), where mouseovers provide more information about objects and Q&As (Figure 7) provide further information about aspects of daily life. There is information about a limited number of topics; these were selected as the ones most likely to appeal to children's curiosity about life then and include information about food, sleeping arrangements, children's work, and even chamberpots.

Modern child becons towards the colonists village

Figure 5: English Colonists page initially

Historical child in bed chamber

Figure 6: English Colonists page inside house

Question and answer about english colonists

Figure 7: English Colonists page Q&A

Rewards and surprises

Some user actions, such as selecting or moving an object, or completing a task, are rewarded by audio, video, text, or animations. 'The need to add fun elements that relate to critical features of an existing activity' increases engagement (Dix, 2003) and contributes to a sense of accomplishment, as well as providing variety. Praise can encourage a user to continue (Fogg, 2003). Rewards are common, if not mandatory, in games, but are not often used in on-line learning.

The rewards differ on each page. For example, once the captions are all correctly placed in the Fact or Myth page, a turkey runs across the screen saying 'Gobble, gobble,' providing a humorous reward. (In Figure 4, the turkey is visible on the left.) In the Primary Sources page, one activity uses sticky notes to provide explanations of phrases in a primary source document. When all of the explanatory notes have appeared, a final sticky note appears with a smiley face that winks once. In the Wampanoag page (Figure 8), when a child has selected all twelve stones, he or she is rewarded by a video.

Native child, and  a circle of stones

Figure 8: Wampanoag page

Activities

Each page is interactive in the sense that there are links to select and activities to do. Each page differs in the type of activity provided, and some provide multiple activities. The Primary Source page includes a 'magic lens' that can be dragged over a primary source document, showing, within the lens a translation of the text to modern English. Fact or Myth (Figure 4) has captions and images which a child can match. With a correct match, a caption slides into place with a sound, and for incorrect matches, the caption returns to the center with a different sound. On the Wampanoag page (Figure 8), which emphasizes the importance of giving thanks as a part of daily life throughout the seasons, children select stones to learn about an aspect of Wampanoag life through text, audio, and pictures. Share What You've Discovered (Figures 9 and 10) was especially important because of the TfU framework. In this activity users develop an on-line museum exhibit of captioned images.

Modern children with their projects

Figure 9: Share What You've Discovered page

Lisa's project

Figure 10: Example of Share What You've Discovered

Measuring Learning

The Impact of Formative and Heuristic Evaluations and Usability Testing

A formative evaluation was done early in the process. An heuristic evaluation was undertaken once development of the site was nearly complete. Usability testing was done at the end.

Formative evaluation

A formative evaluation of the first version of the OLC, done in collaboration with the Harvard Graduate School of Education, led to significant redesign of the site. The formative evaluation was done by students using almost 100 3rd and 5th grade children and teachers, under the supervision of Dr. Ilona Holland. It was important that children (and teachers) be in the target grades since, 'when it comes to designing technologies for children, age matters' (Druin, 2002). The evaluation found that many aspects of the OLC were effective at achieving the Understanding Goals as well as our goals of developing an appealing, engaging, fun, and educational site.

Reactions were very positive to the visual richness and the use of audio. However, we found that children and teachers wanted more activities, more audio, and a stronger role for the children as guides and interpreters. Children who took part in the formative evaluation identified with the children in the OLC and were curious to find out more about them. They liked the juxtaposition of the 1621 and modern children. Children liked the audio, especially when they were not strong readers. They reported that the one game that was developed was fun but they wanted more games and more they could actually do.

The subsequent redesign incorporated this data. The two children were given more prominent roles on every page. Activity instructions originally on the loading pages were moved to the actual activities. During the formative evaluation, we discovered that users, especially children, did not always read instructions, but if they wanted to read or reread them once they were on an activity page, they had to return to the loading page. Putting them on the activity page made them available at any time. Text was reduced in length and professionally edited to increase readability for the target population of children. Moving text onto the activity page furthered the role of the children as guides who provided instructions. Figures 11 and 12 depict how the Wampanoag page evolved to include instructions.

the circle of stones, with little interpretation

Figure 11: Early version of Wampanoag page

the circle of stones with the native guide and some explanatory text

Figure 12: Next version of Wampanoag page

Additional audio was created, both to supplement actions and to make sure that students who were not strong readers could listen instead of, or in addition to, reading. More activities were designed so that each page had something that a student could do. These activities were carefully designed to allow children to be active, rather than passively reading text or listening to audio, and to be engaged in an activity or exploration that helped meet one or more of our Understanding Goals.

The Wampanoag page, which was the one with the greatest appeal in the formative evaluation, was considered a bit too complicated (Figure 11). We simplified it by reducing the number of stones and added the Wampanoag boy to provide information about oral tradition (Figure 12). A reward in the form of a video was added to the page. The final version (Figure 8) depicts the more active pose of the boy from the second photography shoot.

The formative evaluation also helped in deciding the best name for the OLC. The OLC team had brainstormed and selected a number of candidates, and wanted to know which sounded appealing and most accurately represented the site without being too unwieldy. The consensus was that You Are the Historian: Investigating the First Thanksgiving worked best.

Heuristic Evaluation

Further evaluation was necessary as the project neared conclusion to ensure that the redesigned site was usable by the targeted audience and that the learning objectives and goals were met. Three heuristic evaluators, Sabrina Liao, Allison Farber, and Sukanti Iyne, who had significant expertise with the design and testing of children's Web sites and software, looked at the individual components of the site as well as the overall site.

There was considerable overlap in the feedback from the heuristic evaluators. The features they were most positive about were the colors, layout, children, graphics, and audio. They made many minor suggestions for improvements, as well as a few more significant ones. Despite the redesign resulting from the formative evaluation feedback, the heuristic evaluators thought the children were not sufficiently integrated into the activities. They suggested the children explain themselves or the task more, for instance, saying 'You'll be an historian investigating…' or 'I'm …"'and giving a personal story. The text and children in the final version reflects those suggestions. The two children were given more prominent roles on every page and were photographed again in more active poses to further engage children. Figure 12 shows the original pose of the Wampanoag boy ,and Figure 8 shows the more active pose from the second photography shoot. The heuristic evaluator's in-depth understanding of how children think and learn, and how children approach Web sites, proved invaluable for improving, and especially for increasing, the interactivity of the OLC.

Usability Testing

Once the changes resulting from heuristic evaluation were made, final usability testing was done using children and teachers. Children were watched carefully to see what they did, how they reacted, and if they encountered problems. Behavioral signs are often more reliable than verbal responses in children (Hanna et al., 1997), making the visual feedback as important as the verbalizations during testing. This is especially true looking for indications of engagement and enjoyment, as well as learning.

As an education site, we were interested in what testers had learned, not just how they reacted or where they encountered difficulties. Before and after their use of the site, we asked children and teachers to tell us what they knew about the events of 1621 and to define terms such as myth, historian, Wampanoag, and primary source. We were pleased when a child demonstrated greater or deeper understanding following their use of the site, and especially gratified when a child went from talking about Pilgrims and Indians to early Colonists and Wampanoags. We had a few opportunities to do longitudinal testing and follow up with test subjects and found that both retention and satisfaction with the user experience were high.

Teachers were asked to use the site and respond to questions about their reactions to specific site features, as well as their overall reactions. Teacher feedback was important, not just in evaluating the usability of the site, but because teachers would be the adopters. One tester, Bill Cassell, a third grade teacher from North Reading, MA, commented on many features of the site as well as on his overall impressions. He concluded,

Next year your site will be a part of my curriculum, both in social studies and in language arts. I'll prepare linkages between what we study in class and what is on your site, so that my students will be able to make connections in both subjects. You have created a rarity in my estimation: a site with sound, serious scholarship and an engaging, child-friendly approach. If more educational materials could incorporate these two factors, teaching and learning would be more productive and more fun.

Conclusions

From the initial phases of design, we grappled with the best way to make learning fun for children. In redesigning the OLC following formative evaluation, we struggled with how to add the additional activities children requested and that we knew were needed. Some of their suggestions from the formative evaluation were skewed toward increasing interactivity rather than learning. For instance, one child wanted 'a runaway turkey game…you have to press forward and backward to run away from the bullets.' Another requested 'a game that teaches you to sail a ship or a game where you guide the ship through obstacles.' While these and other ideas were loosely related to Plimoth Plantation or Thanksgiving, they did not further the OLC's education goals.

The final version incorporated new activities that brought in the desired interactivity in a manner that was consistent with our Understanding Goals and with the goals for the site. Other enhancements, to further engage children, include increasing the sense that the child is investigating a mystery and drawing children in using an intriguing question, puzzle, or riddle. We avoided the template-driven, more linear approach typical of on-line learning sites: each page was different, providing activities, rewards and surprises, and layers of information. We feel that this approach draws the learner in, and does not sacrifice learning for fun or fun for learning. The act of learning, itself, becomes fun.

The OLC, available at http://www.plimoth.org/OLC, was released in Fall 2003. Feedback from the actual use of the OLC has been extremely positive to date. While we have received extensive feedback from educators, parents, and librarians, we especially enjoy the feedback from children. One child wrote in the feedback form: "This Web site rocks. I got a lot of information from it and it helped me with my report. Thank you!!!!!!!!!!!!!!!!!" Another wrote, 'I loved your Web site it was the best! I am 12 and I don't like history so it helped me enjoyed history! Thanks!"

The OLC has received awards, including the Massachusetts Interactive Media Council (MIMC) 2003 award for best education site and has been Site of the Day or Site of the Week in a number of places. While we appreciate these opportunities to have more people find out about the OLC, we are especially gratified since this recognition shows the perceived quality of the OLC.

Future directions include adding on-line learning communities to the OLC. We plan to give children the opportunity to share their on-line exhibits. We also plan to give teachers a mechanism for sharing lessons learned from their use of the site in the classroom or computer lab and any additional recommendations they have for classroom activities beyond what is included in the Teacher's Guide.

Acknowledgments

We thank the many people involved in the design, development, and evaluation of the OLC. This project was supported by a congressionally directed grant administered by the Institute of Museum and Library Services, by a grant from the Arthur Vining Davis Foundations, and by other generous donors.

References

Blue Web'n Site Evaluation Rubric, http://www.kn.pacbell.com/wired/blueWebn/rubric.cfm.

Carliner, S. Reflections on Learning in Museums: The Inspirational Power of Museums. http://saulcarliner.home.att.net/museums/inspirearticle.htm.

Dix, A. (2003) Being playful – learning from children. In Proceedings of Interaction Design and Children 2003.

Downes, S. (1998) The future of on-line learning. On-line Journal of Distance Learning Administration, 1(3), http://www.westga.edu/~distance/downes13.html

Druin, A. (2002) Age matters, SIGCHI Bulletin, September-October 2002.

Fogg, B.J. (2003) Persuasive Technology: Using Computers to Change What We Think and Do, Morgan Kaufman.

Gibbs, W. W. (2004) Why Machines Should Fear, Scientific American, Vol. 290, No. 1, Jan. 2004, P. 37

Hanna, L., Risden, K., & Alexander, K. (1997) Guidelines for usability testing with children. Interactions, 4(5), September + October, 1997, 9-14.

Horton, W. (2000) Designing Web-Based Training, Wiley, 2000, http://www.designingwbt.com/.

Kearsley, G. and Shneiderman, B. Engagement Theory. http://home.sprynet.com/~gkearsley/engage.htm.

Klassen, K. and Willoughby, K. (2003) In-Class Simulation Games: Assessing Student Learning, Journal of Information Technology Education Volume 2, 2003, http://jite.org/documents/Vol2/v2p001-013-59.pdf

Maxwell, E. and Bittner, M. Museums Spend Over a Billion, Commit Over 18 Million Hours To K-12 Education Programs: Study Finds 72% Use Web Sites To Teach. Institute of Museum and Library Services, http://www.imls.gov/whatsnew/current/012903.htm.

Miyagawa, S. (2002) Personal Media and the Human Community. Technos Quarterly, Summer 2002, Vol. 11 No. 2, http://www.technos.net/tq_11/2miyagawa.htm.

Monk, A., Hassenzahl, M., Blythe, M., and Reed, D. (2002) Funology: designing enjoyment, SIGCHI Bulletin, September-October 2002.

Norman, D. (2004) Emotional Design, Basic Books.

Perkins, D. (1993) Teaching for Understanding. American Educator, v17 n3, Fall 1993, pp. 8,28-35.

Phelps, R. (2003) Developing On-line From Simplicity toward Complexity: Going with the Flow of Non-Linear Learning, In Proceedings of NAWeb 2003, http://naweb.unb.ca/proceedings/2003/PaperPhelps.html.

Schank, R. (1997) Virtual Learning: A Revolutionary Approach to Building a Highly Skilled Workforce, New York, NY: McGraw-Hill.

Teaching for Understanding. http://learnWeb.harvard.edu/alps/tfu/index.cfm.

Tedeschi, B. (2003) Pixels at an Exhibition. New York Times, March 16, 2003.