Science Museum Learning Collaboratories: Helping to Bridge the Gap Between Museums' Informal Learning Resources and Science Education in K-12 Schools

Introduction: The Science Learning and Achievement Gap

"At the daybreak of this new century and millennium, we are convinced that the future well-being of our nation and people depends not just on how well we educate our children generally, but on how well we educate them in mathematics and science specifically."–Before It's Too Late, The Glenn Commission (National Commission on Mathematics and Science Teaching for the 21st Century, 2000).

"No one ever failed 'museum'"–Frank Oppenheimer (1912-1985), Founding Director of the Exploratorium, San Francisco

Museums all over the world are increasingly using the Web and the Internet to enable remote public access to their resources and to expand the variety, frequency and quality of their educational outreach programs. For example in its second national survey of museum-school relationships, True Needs, True Partners (IMLS, 2002), the federal Institute of Museum and Library Services found that the over 15,000 museums in the U.S. alone now commit more than a billion dollars and more than 18 million instructional hours annually to K-12 educational programs. Furthermore, more than 70% of the museums surveyed provide access to their web sites for learning and teaching resources for teachers, students and parents. The percentage of science-technology museums, aquariums, zoos, and other informal science learning centers, as well as university, foundation and government-sponsored science and medical research centers and professional development organizations for science teachers (e.g., NASA, NSTA, PBS) that provide web access and Internet-based exchanges with their members and communities is even higher. In the 21^st century, the Web, now joined with podcasting, digital video, blogs and wikis, has clearly become a major global resource for building of lifelong learning communities (Kahn,, 1999; Kahn & Rockman, 2002, 2003; Kahn, et al., 2004; Craney, et al./ HHMI, 2005; Kahn, 2005).

Over the past 40 years, the aggregated investment by the National Science Foundation (NSF), National Institutes of Health (NIH), NASA, Department of Energy, U.S. Department of Education, and other government programs, foundations and private donors to science-technology museums, PBS/NPR TV and radio, aquariums, zoos, and other informal science centers and related informal science educational programs has been in the billions of dollars. At the same time, U.S. student achievement in science has continued to be lag well behind that of students in many developed countries (Lemonick, 2006; National Academies, 2006), and the vast majority of U.S. K-8 teachers (including middle school science teachers–and even many high school science teachers) still lack even B.A. or M.A. degrees in science or science education (Horizon Research, 2002). Furthermore, the federal "No Child Left Behind" Act has placed increasing accountability pressures on teachers for standards-based instruction and to show adequate annual yearly progress in student achievement as measured by high stakes, standards-based tests. Despite the free availability of billions of dollars of the museum and other informal learning resources on the web, the majority of science teachers still doesn't know about or make integral use of this wealth of resources in their daily classroom instruction. Even the most experienced teachers seldom have time to search through thousands of web pages to find and identify the most exemplary, grade-level appropriate, and state content standards-aligned resources that fit their own science curricula and instruction, as well their students' learning needs (Kahn & Rockman, 2002, 2003; Kahn, 2005.

One Possible Solution: The Bay Area Science Education Collaboratory

Over the past five years, an initial online learning and teacher professional development environment for improving science education in the San Francisco Bay Area has been designed and built to address these issues. The concept approach that was used was one of co-designing a web-based, open source educational "collaboratory" (http://en.wikipedia.org/wiki/Collaboratory) with and for use by a regional community of teachers, parents, science and museum educators, and students. Most collaboratories, such as those documented by the NSF-funded "Science of Collaboratories Project" (http://www.scienceofcollaboratories.org/), have been designed to support geographically distributed communities of science or social science researchers sharing and contributing to essential databases (e.g., the Human Genome Project) or having shared remote/virtual access to and use of unique or rare scientific instruments or facilities (e.g., The Hubble Space Telescope, CERN).

However, the Bay Area Science Museum and Education Collaboratory has taken a slightly different approach to its design and use. First, the Bay Area Science Collaboratory has been specifically co-designed with and for use by science teachers to "bridge the gap" between museum and other informal science centers and formal science education (Hofstein & Rosenfeld, 1996; Hofstein, Bybee, & Legro, 1997; Bybee, 2001; IMLS, 2002; Kahn & Rockman, 2002). The Bay Area Collaboratory (Kahn & Rockman, 2003; Kahn, 2005) supports San Francisco Bay Area (as well as other California) science teachers and students, parents, museum educators, and other informal science providers by collectively helping teachers and other users easily find and aggregate the best standards-aligned, web-based learning and teaching resources from multiple Bay Area science-technology and children's museums, aquariums, zoos, and other "best of the Web" resources for use in formal science education in the classroom. The initial target audience for the Collaboratory has been San Francisco Bay Area middle grades science teachers (grades 4-8). This focus was catalyzed by extremely low levels of California student performance in science at the 5^th and 8^th grade levels on the National Assessment of Educational Progress in 2000.

Figure 1: Bay Area Science Education Collaboratory Home Page
Copyright © 2003, 2006 by DesignWorlds for Learning, Inc.,
http://www.designworlds.com/collab/

Standards and Accountability: The Impact of No Child Left Behind

The creation of national education science standards during the 1990's (National Research Council, 1996, AAAS, 1993) was followed by the passage of the No Child Left Behind Act in 2001. This major federal legislation has refocused schools and teachers' accountability to be very highly based on instruction that is based on state content standards, with annual student achievement testing with increasingly "high stakes" for showing higher levels of student achievement and annual improvement. For example, the California Department of Education's STAR testing program now administers annual California standards-based science tests to all students in grades 5, 8 and 10 (as mandated by No Child Left Behind), and these test results are factored into each school's Academic Performance Index (API). Failure to meet adequate Annual Yearly Progress (AYP) goals on these tests, as well as those in mathematics, reading and language arts, can have serious consequences for school funding and independence of individual teacher choices about classroom instruction. In the most extreme cases, a school's independence of curricular choices or even its continued existence, can be sacrificed if its API is not sufficiently high or its AYP goals are not met.

While teachers and student field trip groups are a major audience for science museums, none of these museums were designed around these national or state curriculum standards. However, because American students' achievement in science now lags far behind that of students in many other developed countries and because there is national concern over the decreasing number of American students pursuing undergraduate and graduate study in science, technology and engineering fields (as compared to major increases in India, China, Korea, and other countries), a number of leading science educators have argued that museums should make strong efforts to show how their exhibits and other educational resources can complement and supplement the national science education standards (e.g., Bybee, 2001). The second IMLS "True Needs, True Partners" survey of museum-school partnerships also documented that the vast majority of museums now either have staff or have hired consultants dedicated to showing teachers and educators the alignment and connections between the museums' exhibits and programs with state curriculum standards (IMLS, 2002).

The Bay Area Science Collaboratory was thus designed to provide a "light switch easy" visual Web interface to link the wealth of over $1 billion invested in multiple Bay Area science-technology museums and other informal science centers with California's state science curriculum standards (California Board of Education, http://www.cde.ca.gov/be/st/ss/scmain.asp). In our initial phase of Collaboratory design, we discovered that each museum has its own way of presenting its learning and teaching resources and assets on the Web, and there is no one standard interface or way of finding the answer to a typical teacher's question: What resources do you have that are appropriate for sixth grade and that directly address these California science content standards about plate tectonics?

Some might ask, "Why can't teachers just use Google, Yahoo or other search engines to find these resources?" The simple answer is:

1. most teachers lack the time for doing such searches;
2. the information overload factor has now become overwhelming (For example, in 2002, a Google search on the terms "middle school plate tectonics" resulted in over 17,000 hits; the same search performed in early 2007 now results in over 330,000 hits!) and
3. Web searches cannot easily determine grade level appropriateness of the resources, pedagogical fit with the curriculum, or effectiveness of these resources in addressing different scientific content or processes of inquiry and experimentation. However, these are exactly the kinds of knowledge that museum staffs have, especially those who have work directly with teachers for many years, but this kind of "pedagogical/standards metadata" is rarely included with web resources of most science museums.

The Collaboratory is one of a growing number of "open source" web-based collaborations to co-design and use community aggregated, created and modified educational resources that are changing the landscape of web-based lifelong learning (Tapscott & Williams, 2006). In this way, it is both a growing shared "community knowledge and learning garden" for science education, as well as an ongoing professional development experience in both science content and "best practices" in science pedagogy for its participants. The Bay Area Collaboratory is based on a highly collaborative action research and participatory design model, in which teachers have both been central members of the design team, as well as its users, contributors and advocates, informal trainers and coaches for other teachers. It also represents an alternative professional development model in science education at a time when fewer than 45% of Bay Area middle school science teachers (and even fewer elementary teachers) have academic degrees or adequate formal backgrounds in science.

Initial needs assessment and potential stakeholder analysis research in 2001-2002 (funded by the Hewlett Foundation) identified a wealth of existing exhibits, lesson plans, "recipes" for hands-on demos/activities, and web-based educational outreach activities in Bay Area science-technology and youth museums, aquariums, zoos, and nature parks. However, we also found that these resources are still highly under-utilized by the majority of teachers in the Bay Area–and especially those in middle schools in large urban districts and even less in underserved communities.

A summary of our findings from our initial needs assessment research in 2001-02, as well as the formative evaluation during our first two years of development, is as follows:

• Bay Area elementary and middle school teachers, especially those in the elementary grades, are generally aware of and have occasionally visited at least some of the many museums and informal science centers in the Bay Area.
• However, most teachers are not aware of the diverse kinds of science learning and teaching resources available from these museums, especially now that these are available on the museum Web sites.
• Until recently, few of these museums provided easy ways for teachers to find grade level appropriate, standards-aligned resources, either on the museum exhibit floor or via their web sites. Since few teachers have the time to seek out and make decisions about which of these resources they can most easily use in their classrooms, much of the major wealth of informal learning and teaching assets remain unused except by the most experienced science teachers.

These findings have also since been verified by many other participating teachers, as well.

What is a Science Education Collaboratory?

Figure 2: Bay Area Science Education Collaboratory 6^th Grade Home Page
Copyright © 2003, 2006 by DesignWorlds for Learning, Inc.
http://www.designworlds.com/collab/collab-6th/

Here's a typical example of the rationale and need for the Collaboratory: Suppose you are (or are working with) a San Francisco Bay Area 6th grade science teacher or you are a parent of a 6^th grader. California's 6^th grade science standards emphasize earth sciences (http://www.nexist.org/hf/Standards and http://www.cde.ca.gov/be/st/ss/scgrade6.asp). We've already seen that just doing a typical web search produces far too many hits to be useful to most teachers. For specific museums, some of them have search engines that can ameliorate this situation, but many of the best resources available in these museums may still not be found this way (Note: The best solution is to personally know a museum staff member or a teacher who was been part of an ongoing teacher fellow or teacher institute program at a museum, but there are very few such teachers available on demand!). Constraining this search further may cut down the total number of hits, but almost no middle school teacher has the time to even search through more than one or two pages of these hits to find those that are most appropriate to teaching needs.

The Collaboratory was designed to help address just these needs. It has been designed as more of a "community knowledge/learning garden" than a portal. By this, we means that it needs to be seeded, tended and nurtured, harvested and "weeded" by a community of users in order to be healthy and to continue to serve the needs of the middle grades science teaching community.

Using the community garden metaphor, the Collaboratory consists of four major parts or components:

1. Seeds

The seeds of the Collaboratory come from multiple Bay Area science museums, zoos, aquariums, and science research centers with web resources, as well as other informal science learning centers. Each museum tends to have its own pedagogical framework, as well as a particular focus for its educational goals and programs.

The Exploratorium in San Francisco (http://www.exploratorium.edu) is one of the oldest, largest and best-known "hands-on" science museums, with a strong focus on inquiry-based learning, especially for elementary school students.

The Tech Museum of San Jose (http://www.thetech.org), which also has many hands-on and interactive exhibits, focuses much more on how technology is designed to utilize scientific principles to solve problems or give birth to whole new areas of business, industry and research.

The University of Caliornia's U.C. Museum of Paleontology (http://www.ucmp.berkeley.edu) is a relatively small museum on the Berkeley campus and thus, it was not designed for major physical public access or visits; rather, it is one of the university's natural history research museums. However, UCMP was one of the first museums in the world to develop a web site and its web site now houses both an extensive portal for paleontology, as well as one of the most extensive sets of resources about evolution and science in the U.S. (http://evolution.berkeley.edu).

Similarly, Chabot Space and Science Center (http://www.chabotspace.org) in Oakland has a focus on observation of objects that may be light years away from Earth, as well as timescales that can be in the millions or billions of years; hence, its pedagogical focus is more on observation of patterns and evidence from other worlds, coupled with similar observations on Earth, to help us understand our place in the Milky Way Galaxy – how were the Earth and Solar System born, where else might there be life or planets quite similar to Earth, and what the relationship is between astronomy, physics, biology and cosmology (and cosmic evolution)?

The Lawrence Hall of Science at the University of California, Berkeley (http://www.lhs.berkeley.edu) not only has many hands-on exhibits, but is has been the home of research, development and creation of many of the nation's major science and mathematics curriculum reform projects for nearly 40 years (e.g., SCIS, OBIS, HAP, EQUALS, Family Math, GEMS, FOSS, CHEM, SEPUP and many others).

Finally, the Children's Discovery Museum of San Jose (http://www.cdm.org) serves primarily younger students than those of our Collaboratory focus, but its BioSITE project is a major educational outreach program which is directly relevant to middle grades environmental science and education (http://www.cdm.org/biosite/).

2. Soil

The CA state science standards are the "soil" into which the science content "seeds" from different museums need to be planted so they can grow into learning resources teachers can harvest. Figure 2 above shows a visual interface to the CA 6^th grade science content standards (with thanks to the visual creativity of Eileen Clegg, http://www.visualinsight.net). Each standard is subdivided into subtopics or specific "benchmarks" which are designed to indicate how much students have learned, all tied to grade-level appropriate standards. For example, we might pick 6^th Grade Standard #1, Plate Tectonics, from the Collaboratory's 6^th Grade Home Page (Figure 2 above). This would take us to the web page where all the subtopics and benchmarks for this standard are linked to different kinds of museum web-based learning and teaching assets:

Figure 3: Bay Area 6th Grade Science Education Collaboratory, Standard #1 Web Page
Copyright © 2003, 2006 by DesignWorlds for Learning, Inc.
http://www.designworlds.com/collab/collab-6th/plate.html

Clicking now on any of these benchmarks will bring up a set of green buttons on the left side of the screen, which represent the "fruits or vegetables" to be harvested from our quest to find good resources. For example, if you click on "Volcanoes and Earthquakes," and you then click on "Projects," you will see the kind of student project resources some of our teachers have found and used from these museum and informal learning resources:

Figure 4: Exploratorium "Faultline" Webcast Archive and Resource
Copyright © 1999, 2006 by the Exploratorium
http://www.exploratorium.edu/faultline/

3. Harvesting Educational Fruits and Vegetables

The Collaboratory has several different kinds of resources that can be used by teachers, parents, and students depending on needs and curricular focus. These are shown as the buttons on the left side of each Collaboratory screen, and the gray buttons turn green once a user selects a specific subtopic and there have been resources found (and shared) by other teachers for any given resource type, as follows:

• Lesson and Unit Plans: These are the basis for most classroom teachers' work in schools. Many museums now provide a variety of lesson plans that address topics that are either drawn from exhibits on the floor of the museum or from themes and pedagogical focus of the museum as a whole. Teacher-designed, contributed and rated lesson plans are being created at our next level of Collaboratory development.
• Hands-on Demos and Activities: Most science and children's museums have a number of interactive exhibits that are the main draw for attending these museums. However, mini-versions of many of these exhibits or demonstrations may be created by teachers (or by students at home) if they are given sufficient guided instructions. For example, the Exploratorium has what it calls "Snacks" that are very low-cost hands-on experiments or demos that can be done by both teachers and parents (http://www.exploratorium.edu/snacks/).
• Homework: These are web and other resources for independent student exploration of a topic addressed in class, as well as "family learning" opportunities that make use of museum resources in conjunction with classroom curriculum.
• Projects: These can range from very short-term experiments and investigations (e.g., "kitchen science") to longer-term science fair projects, which require independent student research, and investigations (see, for example, A Schmahl Science Workshop http://www.schmahlscience.org).
• Field Trips and Virtual Tours: When classes can actually go to a museum for a field trip, the Collaboratory allows teachers to find and share pre-visit planning guides, resources for the actual museum field trip and post-visit follow-up and reflection, all aligned with one or more science curriculum standards for their grade level. The now common use of museum webcasting (first pioneered in the Bay Area by the Exploratorium in the 1990's) has become a major outreach vehicle for many museums and other science centers. The growth of lower cost, easy-to-use digital video cameras and video sharing sites, such as YouTube (http://www.youtube.com) and Google Video (http://video.google.com), now enable sharing of some of students and teachers interactions with exhibits so that others can see these virtually. Similarly, video podcasts can and will become common portable virtual museum field trips, as well as visitor orientation guides.
• Assessment: This is an area needing considerable development and collaboration with the museums over the coming years. Annual assessment of science learning and achievement is now typically limited to student performance on standardized multiple-choice tests, which tend to favor memory of facts over assessment of deeper understanding of scientific concepts, scientific thinking and applied scientific literacy. We are confident that over time, new and much better methods of diagnosing and assessing students' understanding of science will emerge, and museums may well become one of the major sources of new ways of assessing both informal and formal science learning.
• "Best of the Web" Resources: In addition to museums, there are, of course, many local, national and international institutions and other sources of exemplary science learning available on the Web for different grade levels. For example, NASA Ames Research Center and the USGS are both outstanding sources of science content, both are located in the San Francisco Bay Area, yet neither is a museum, per se. While we encourage teachers to first look for resources from the local museums, this category enables teachers to find, share and use any other resource on the Web that's appropriate to their needs.

4. Irrigation and Weeding: Nexist Wiki Topic/Subject Maps and Standards Matrix/Database

A major part of the benefit of both the technical and social architecture of the Collaboratory is in the connecting of different concepts, museum resources, and curriculum standards and pedagogical practice of a growing teacher-user community. The Bay Area Collaboratory has been fortunate to be able to use an open source wiki environment, called Nexist Wiki, developed by Jack Park, an expert in developing web-based topic and subject maps (Park & Hunting, 2002 http://www.thinkalong.com/Nexist/ and http://www.nexist.org/dev/)– soon to be merged and upgraded with his new TopicSpaces. The importance of web-based subject and topic maps is that will provide the connection between and hierarchy of concepts and topics that are not at all clear to teachers in the way that state curriculum standards are now listed. This area has been very well developed by AAAS Project 2061 in its Atlas of Science Literacy (AAAS & NSTA, 2003), but it is currently not available in electronic form.

Figure 5 shows one screen and link to one of the many Collaboratory 6^th grade topic/subject map web pages, showing links with other topics within this space:

Figure 5: Home Page for Nexist Wiki topic map of Sixth Grade Bay Area Science Education Collaboratory
http://www.nexist.org/hf/TopicMap?item=Home

The other critical part of this space is the MySQL database and wiki Standards Matrix where all new resources found by teachers and other community members are stored. Anyone who wants to go beyond the two most popular (highly voted) resources shown on the Collaboratory's main pages for any standard, subtopic and grade level may search through this space and protected password access is given to those actively participating teachers who are contributing new resources to the Collaboratory. Below is a subset of the 6^th grade Standards Matrix and database from the 6^th Grade Collaboratory:

Figure 6: Bay Area Science Education Collaboratory 6^th Grade Standards Matrix and Wiki Resource Database
Copyright © 2003, 2006 by DesignWorlds for Learning, Inc.
http://www.nexist.org/hf/Standards

A summary of the full community process of finding, seeding, using, voting and "weeding" resources for the Collaboratory as a community informal science knowledge/learning garden is show in Figure 7.

Figure 7: DesignWorlds for Learning Science Education Collaboratory Development Process and Feedback Cycle. Copyright © 2003, 2006 by DesignWorlds for Learning, Inc.

Early Evaluation Findings

First Year Evaluation. An external formative evaluation was conducted after the first year, based on in-depth interviews and questionnaires that were administered to the participating teachers. This evaluation found that the participating teachers were very diverse with respect to their teaching experience, science background, and experience with Bay Area science museums and online resources. All teachers increased their use of web-based resources in their classrooms especially web sites they had heard about at the workshops. Teachers began sharing information about web-based resources with other teachers in their schools and with each other. The major constraints in using web-based resources were related to time. Yet, teachers felt that the project's work has a great potential to save their time by helping them find the resources they need. The teachers' level of satisfaction with the web site also increased over the course of the project as the web site was modified to meet their needs. Finally, the teachers improved their knowledge of science content and science pedagogy (Kahn and Rockman, 2003).

Second Year Evaluation. This evaluation, similar in design to the first, found that as teachers became convinced that the time they spent using Collaboratory resources saved them time in the long run, they were more likely to be willing to invest that time up-front. Participating in the Collaboratory increased these teachers' knowledge of Bay Area informal science resources, science content and processes and, the California sixth grade science standards. Teachers with the lowest level of content knowledge learned the most. The evaluation also found that focusing the Collaboratory resources on the standards teachers are required to teach was an increasingly important feature of the environment, from the perspective of the teachers. The teachers also reported a wide variation in the quantity of material available for the various standards, but not in the quality.

The following quotes express two middle school teachers' enthusiastic experience in both being part of the Collaboratory co-design team, as well users of the Collaboratory's science learning resources they and their teacher colleagues found from the different museums:

"This experience feels wonderful. I feel very privileged to have access to it. I love it! Thank you to all who made this possible."

"Bay Area Science Education Collaboratory ... AWESOME!!!!! (to say the least)"

Outcomes: How do we define success for the Collaboratory?

Ultimately, the success of the Collaboratory involves the alignment of all the key stakeholders–learners, providers (in this case, the museums), payers, and policy-makers–in terms of engaged student learning and achievement in science. In our next phase of Collaboratory development, this translates into the following:

• Student Success is 10-15% improvement in student academic performance in science (more knowledge and demonstrated performance in the same time, or the same knowledge and performance in less time) determined by state, national and international standardized tests or other accepted measures, and by the portfolio of qualitative and quantitative assessment instruments that are available or designed for the assessment feature on the Collaboratory web site. In 5^th and 8^th grades, this can now be measured using the California Standards Tests (CST) in science. Student success also includes effective communication of science concepts in writing, multimedia presentations, and in other forms, not only in science classes, but also in integrating critical analysis, science-related-issues, inquiry and other science concepts in areas such as language arts, social studies, and problem-solving applications in mathematics. Student Success is also characterized by improved and sustained student interest and engagement in science and science-related activities. Following use of the the Collaboratory, teachers, parents and others should observe students asking more questions, reading more science-related content, engaging in conversation or writing in which science is included, doing science-related projects (e.g., science fairs), and expressing interest in going to science museums or participating in other science-related activities.
• Teacher Success is shown by positive and enthusiastic input from teachers saying the Collaboratory helped them help their students be even more successful learning science by using supporting quantitative AND qualitative assessment examples. It is also shown by enthusiastic teachers saying that their own knowledge of the science they are teaching has improved, and that their classroom preparation time is used much more effectively because of the Collaboratory.
• Parental Success is shown by positive feedback from parents, guardians, or mentors about HOW the Collaboratory helped them help their children/learners be even more successful learning science. Homework, science projects, discussions of science news events and national/international science-related issues, and museum visits are all areas in which parents can easily interact with their child in exactly the appropriate part of the curriculum. During these interactions, the learner and the parent can feel comfortable that the activities being supported by the parent are directly useful to the learner.
• Community Success is ultimately shown by an increase in scientific literacy and science awareness, including: (1) getting students to continue taking science-related classes, above and beyond those which are required to graduate; (2) an increased desire of the science and science education community to appreciate the value and validity of science enterprises and to ensure continued support of them; and (3) desire and demonstrated behavior (of the social science community, in particular) to increase desire for and active, direct public participation in the decision-making process on societal issues having a science-related basis

Assessment: How do we know that we have been successful?

The items above are all measurable conditions. Our plan is to involve our external evaluator in designing instruments and collecting data to capture the information needed to understand the level of success reached by the Collaboratory. While some of the proposed outcomes are longer term and cannot be captured in the brief time of the proposed effort, indicators of some of these goals will be assessed.

For each of the grade-level initiatives, we will establish the baseline or a range for measures that include both academic achievement and teacher performance. The data will be collected from the classrooms of participating teachers and, where possible, from classrooms of teachers in the same schools not participating in the project. At the end of each semester, a second data collection process will capture outcome data that can be related to levels of use of the Collaboratory materials. Post measures will also look at teacher and parent success indicators.

Among the specific issues to be explored in the evaluation are:

• Level of teacher participation (How much is the Collaboratory web site used, and do teachers remain active participants throughout the project? (i.e., level of attrition, levels of contribution.))
• Amount and quality of implementation (use of Collaboratory and the nature of the lessons implemented by participating teachers)
• Impact on curriculum and pedagogy (Does the Collaboratory change the content in the classrooms by supplementing or supplanting existing resources and how does the instructional process change when these resources are available.)
• Impact on student learning/achievement and engagement/interest (can include standardized tests in science (depending on what is being applied in the classrooms), participation in and quality of student science projects (as evaluated by peers, other faculty and external judges), use of scientific language and the ability to communicate science concepts, and other indicators of student knowledge of science concepts developed from Collaboratory materials.
• Parent participation in children's science education (Do they attend museums or discuss science-related exhibits/media shows with their children? Support science fair participation?)

The evaluation feedback will contribute to further knowledge about the elements in the Collaboratory and can help refine the aspects of the Collaboratory development process that work well and those that don't, as well as the strategies for its implementation that lead to success with underserved students. In addition, the evaluation will portray the levels of academic success reached by the project by comparing baseline data with outcome measures and will inform the interpretation by the recommendations made by the people who now have the wisdom of practice (students, teachers and parents). The academic outcome measures will include standardized measures administered by the state, district, and school, as well as national and international measures administered by assessment programs. There are also likely to be teacher-made assessments developed by the design team and by individual teachers who are participating in the project.

Future Plans and Prospects

The Bay Area Science Education Collaboratory now has all the graphical interfaces and databases for all California science curriculum content standards in grades 4-6 and will soon have completed those for 7^th and 8^th grade. Over 100 Bay Area K-12 science teachers, museum and science educators, and even community college faculty have used and contributed to its hundreds of resources over the past 4-5 years.

Like many collaborative open source web-based projects (e.g., Tapscott & Williams, 2006), the growth and success of this resource will depends on reaching a critical mass of active teacher and parent users, with demonstrated results and outcomes showing improvement in student science learning and interest, as well as standards-based science achievement. For the Collaboratory to also function as a central resource for a teacher professional development community, it will also need to support teachers in developing stages of need and level of activity. We see the following as the key stages of support:

• Users of existing Collaboratory content ("Consumers")
• Contributors of content resources and lesson plans
• Editors (of content or specific grade level standards, such as "Investigation and Experimentation")
• Mentors/coaches for support of other teachers
• Community advocates, builders and "evangelist" for recruiting other teachers

Our next plans involve the "seeding" of the 7^th and 8^th grade content of the Collaboratory, as well as a major field test and evaluation of the use of this resource to complement, supplement, or even replace current textbook-based science instruction taking place in many Bay Area elementary and middle schools. We are actively looking into collaborating with a number of open source education initiatives (many funded by grants from the Hewlett Foundation), and we are currently exploring how this resource might help bootstrap creation of content for open source science textbooks and other open source digital media/interactive learning in science in middle grades and high school.

Because the vast majority of the over 15,000 museums in the U.S. are not just science-technology museums, we feel this same model of a participatory design and practitioner community, committed to finding and using the wealth of other museum and informal learning resources for improving formal education, has great potential, especially in helping museums better communicate how their assets can be mapped and aligned with state curriculum standards. For example, a majority of the museums are visual arts and history museums, and the IMLS has funded several of these museums to find ways to co-create curriculum units with teachers that are aligned with national (or state) standards. Museums' co-development of new, easy to use content authoring and publishing tools, such as the San Francisco Museum of Modern Art and the NMC: New Media Consortium's Pachyderm 2.0 open source authoring system (http://www.pachyderm.org) will enable the creation of media-rich interactive lesson plans that can then be easily organized and aligned with curriculum standards for teachers, especially for art and history, but also, potentially for science and other content areas, as well.

Finally, because many areas of the U.S. (and other countries in the world) have their own multiple museums with rich learning and teaching resources on the web, we very much look forward to working with other communities in helping create new community open source Museum and Informal Learning Collaboratories to help teachers, parents, museum staffs and others be even more successful in helping students be engaged and successful learners in this age of global, free access to the world's wealth of educational resources.

Acknowledgements

The Bay Area Science Education Collaboratory has been supported by grants from the William and Flora Hewlett Foundation and the S. D. Bechtel, Jr. Foundation. We gratefully acknowledge the generous support of both of these foundations, and we wish to thank Dr. Marshall (Mike) Smith, Jorge Ruiz-de-Velasco, Laurie Bechtel-Dachs, and Susan Harvey. Additional support has also been provided by the San Jose Unifed School District, through their federal Magnet Schools Assistance Program grant, and from the Krause Center for Innovation at Foothill College, which has provided the means for online teacher professional development for over 100 Bay Area teachers in the co-development and use of the Collaboratory. We also wish to especially thank and acknowledge our non-profit educational partners, Saul Rockman and Dr. Beverly Farr of Rockman, et cetera:http://www.rockman-etc.org), Dr. Bill Erlendson and The San Jose Unified Educational Foundation, and Meril Smith and Belinda Lowe-Schmahl, A Schmahl Science Workshop: (http://www.schmahlscience.org). We deeply acknowledge our core Collaboratory team: Dr. Rick Berg, Eileen Clegg, Susanne Commisso, Frona Kahn, Jack Gottsman, Jack Park, and Dr. Sherman Rosenfeld, as well as the following consultants who have been part of our design team over the past five years: Alan Bernard, Bill Daul, Bob Kahn, Dr. Loretta Kelley, Dr. Valerie Landau. We also thank Barbara Glynn, Bob Peterson, and Jerry Kay for their excellent contributions to communicating and promoting the Collaboratory, as well as Deborah Bond-Upson, CEO of Galaxy Classroom/Best Practice Networks (http://www.bpn.net); our Educational Advisory Board members: Dr. Mike Berman, Herb Kohl, Krishen Laetsch, Dr. Ida Oberman, Professor Emeritus Larry Lowery, Dr. Bonnie Marks, Steve Mayer, Judy Scotchmoor, Dr. Art Sussman, Dr. Herb and Marlene Thier, Bernie Trilling, and Professor Mattanjah de Vries; the participating Bay Area science-technology and children's museums, and the over 100 dedicated Bay Area science teachers who are have been our design partners, contributors and users.References

AAAS (American Association for the Advancement of Science) and NSTA (National Science Teachers Association, 2003). Project 2061: Atlas of Science Literacy. Oxford: Oxford University Press.

AAAS (American Association for the Advancement of Science 1993). Project 2061: Benchmarks for Science Literacy. Oxford: Oxford University Press.

Bybee, R.W. (2001). Achieving science literacy: Strategies for insuring that free-choice science education complements national formal science education efforts. In J. H. Falk (ed.), Free-Choice Science Education: How we Learn Science Outside of School (New York: Teachers College Press) 22-63.

California State Board of Education (Califorina Department of Education), Science Academic Content Standards, Grades K-12. http://www.cde.ca.gov/be/st/ss/scmain.asp

CISC Science Subcomittee (2002). Strategic Science Teaching Grades K-12: A Sampler of Science Lessons Connecting Literature with the California Standards. San Diego: San Diego County Office of Education (Graphic Communications Dept.).

Craney, C., T.M. Kahn, N. Moreno and A. Thanukos. (HHMI, 2005). Invited panel on "Dissemination Strategies," 2005 Pre-College Science Education Directors Conference, Howard Hughes Medical Institute, Chevy Chase, MD, Oct. 25, 2005."

Falk, J.H., (Ed.), (2001): Free-Choice Science Education: How We Learn Science Outside of School (New York: Teachers College Press.

Hofstein, A., R. Bybee, and P. Legro (1997). Linking Formal and Informal Science Education through Science Education Standards. Science Education International, 8(3), 31-37.

Hofstein, A. and S. Rosenfeld (1996). Bridging the gap between formal and informal science learning. Studies in Science Education, 28, 87-112.

Horizon Research (2002). The Status of Elementary Science Teaching and The Status of Middle School Science Teaching. http://2000survey.horizon-research.com/reports/#statusteaching

IMLS (2002). True Needs, True Partners: Museums Serving Schools. Washington, D.C.: Institute of Museum and Library Services, http://www.imls.gov/pdf/m-ssurvey.pdf.

Kahn, T. M. (2005). The Bay Area Science Education Collaboratory. Invited presentation on Dissemination Panel at the 2005 Pre-College Science Education Directors Conference of the Howard Hughes Medical Institute, Chevy Chase, MD, Oct. 25, 2005.

Kahn, T. M., L. Kelley, S. Marvit, S. Rosenfeld, H. Thier, M. Thier, and C. Youker (2004). The Science Education and Musem Learning Collaboratory. A Collaborative Community Model of Bringing Exemplary Local Science Museum Learning Resources to Middle School Teachers via the Web. Panel presentation at the 2004 Annual Conference of the Asssociation of Science-Technology Centers (ASTC), San Jose, CA, Sept. 18, 2004.

Kahn, T.M. and S. Rockman (2003). The Bay Area Science Education Collaboratory, Unpublished final report to the William and Flora Hewlett Foundation, October, 2003. ROCKMAN, et cetera and DesignWorlds for Learning, Inc. http://www.designworlds.com/Hewlett/BA_ScienceCollab/index.html

Kahn, T. M. and S. Rockman (2002). Toward a Bay Area Science Learning Collaboratory: Leveraging the San Francisco Bay Area Science-Technology Museums and Other Informal Science Education Programs as a Key Educational Resource for Student Learning and Teacher Professional Development. San Francisco/San Jose: ROCKMAN, et cetera. and DesignWorlds for Learning, Inc. http://www.designworlds.com/Hewlett/BA_ScienceCollab/index.html

Kahn, T.M. (1999). Designing Virtual Communities for Creativity and Learning. Nicasio, CA: Edutopia, The George Lucas Educational Foundation. http://www.edutopia.org/php/article.php?id=Art_483

Kisiel, J. F. (2003). Teachers, museums and worksheets: A closer look at a learning experience. Journal of Science Teacher Education, 14(1), 3-21.

Kruger, A. and J. Sutton (2001). EDThoughts: What We Know About Science Teaching and Learning. Aurora, CO: McREL. (Mid-continent Research for Education and Learning).

Lemonick, M. D. (2006). Are we losing our edge? Time Magazine, Feb. 13, 2006. http://www.time.com/time/archive/preview/0,10987,1156575,00.html

National Academies of Science & Engineering and National Institute of Medicine (2006). The Gathering Storm: Energizing and Employing America for a Brighter Economic Future. Washington, D.C. National Academies Press, http://www.nap.edu/catalog/11463.html.

National Research Council (1996, 2001). National Science Education Standards. Washington, D.C.: National Academies Press.http://www.nap.edu/openbook/0309053269/html/index.html

Park, J. & Hunting, S. (2002). XML Topic Maps: Creating and Using Topic Maps for the Web. Boston: Addison-Wesley Publishing Company.

Rosenfeld, S., J. Griffin, S. Commisso, T. M Kahn, V. Kaul, L. Kelley, N. Maoz, S. Rockman, M. Storksdieck, and M. Werner (2006). Varieties of Formal-Informal Science Learning Environments: Perspectives of Different Stakeholders. Panel paper presentation, 2006 Conference of the National Association for Research on Science Teaching (NARST), San Francisco, CA, April 6, 2006.

Tapscott, D. and A. D. Williams (2006). Wikinomics: How Mass Collaboration Changes Everything. New York; The Penguin Group.