Bringing It All Together: Developing a User-Centered Search Experience on the SFMOMA Web Site

Dana Mitroff, Marla Misunas, and Susie Wise, San Francisco Museum of Modern Art, USA

http://www.sfmoma.org/

Abstract

The San Francisco Museum of Modern Art has begun a major cross-departmental initiative to bring its collections management database online and integrate the contents of the collections database within a new search engine. From media-rich, Flash-based multimedia content to public programs information stored in a SQL database, the new site-wide search engine will integrate data from several disparate sources into a seamless, web-based engine for the Museum's varied audiences.

Key words: Collections database, content development, content management, multimedia, online collection, search engine, usability, user interface, user testing

Overview

The San Francisco Museum of Modern Art (http://www.sfmoma.org) has begun a major cross-departmental initiative to bring its collections management database online and integrate the contents of the collections database within a new search engine. From media-rich, Flash-based multimedia content to public programs information stored in a SQL database, the new site-wide search engine will integrate data and interpretive content from several disparate sources into a seamless, web-based engine for the Museum's varied audiences.

In this paper, we will discuss how this project evolved and grew in scope, our commitment to user-centered design, and what we discovered in the development process.

Background

The project, informally called Collections Access Online, grew out of a National Endowment for the Humanities Challenge Grant and is a cross-departmental collaboration between the Web group in the Publications & Graphic Design Department, the Interactive Educational Technologies group in the Education Department, and the Collections Information and Access Department. The three authors are from the three respective departments, and their diverse perspectives inform the paper.

SFMOMA received an NEH Challenge Grant of $1 million to provide essential endowment resources for expanded use of new technologies to increase public access to SFMOMA's rapidly growing permanent collection. NEH support represents a crucial step in leveraging our collections' value as an interdisciplinary educational resource for the general public, students, teachers, and scholars. Subsequently we also received a $500,000 grant from the Institute for Museum and Library Services to further refine our use of multimedia and improve our ability to communicate with visitors and our school audience in galleries and online.

While SFMOMA received both the NEH and the IMLS grants based on its significant educational multimedia resources developed over the past seven years, we do not have searchable, object-specific data on the Web site. The Collections section of the site contains static HTML pages featuring only 45 selected works, and there is currently no way for a user to search any part of our collections database (http://www.sfmoma.org/collections/).

This lack of publicly accessible, digital collections information on the SFMOMA site was seen as a critical gap in our institutional offerings, and informed one of the key objectives of the NEH grant: to offer greater public and scholarly access to SFMOMA's Collections database through online accessibility of images and basic wall label information for the artwork that is most frequently utilized in exhibition, publication, and education programs via SFMOMA's Web site. The IMLS grant promises to go even further by streamlining and refining access to our collections database and educational materials.

When we first received the NEH grant, our goal was to put images and basic wall label information online by using EmbARK Web Kiosk, the Web component of our collections management software, EmbARK. We planned to modify the interface of EmbARK Web Kiosk so that it matched the look and feel of our existing site, and link to it from the Collections section of our Web site. However, we did not intend to integrate the collections data with our site-wide search engine. This meant that, in order for a user to search the SFMOMA collections, he/she would have to click through to the Collections section of the site. Simply typing in an artist's name or title of an artwork into the search box would yield no results.

As we began developing the functional specifications for this project, we quickly realized that merely putting EmbARK Web Kiosk online inside the Collections section of our site would not best serve our Web visitors. To us, it may be obvious that clicking on Collections in the navigation bar would lead one to the collections-specific portion of the site (where, presumably, one could then search the collections), but this puts the burden on the user to discover this. Burying the SFMOMA Collections information inside the Collections section of the site also assumes that all users are familiar with the basic concept of a museum collection. Furthermore, even if a user does finally click through to the Collections section of our site, he/she might never discover all of the rich, interpretive content about our collections in the Education section of our site.

From object-level information to interpretive multimedia content, the right solution would be to provide a simple, centralized way for users to discover all of the content available on the SFMOMA site. This solution should be a search engine that brings everything together in a simple, easy-to-use interface.

At the time of this paper's publication, the SFMOMA Web site uses a basic search engine that came with Microsoft's Internet Information Server. This search engine, a local file indexer, only finds the static HTML pages on our Web server. A local file indexer "finds" files by following the directory structure on the hard drive, usually starting with the Web server root directory. Local file indexers scan files based on their location in the directory structure, as opposed to following Web links, and they can't "see" dynamic data. The SFMOMA site has nearly 4,000 HTML pages, but we also have a significant amount of content that is stored in other formats, such as content stored in a SQL database and content stored in Flash files. The entire Calendar (http://www.sfmoma.org/calendar/), Exhibitions (http://www.sfmoma.org/exhibitions/), and Press Room (http://www.sfmoma.org/press/) sections of our site are database-driven. In addition, all of our rich, interactive educational content, such as Making Sense of Modern Art (http://www.sfmoma.org/msoma/), is Flash-based. And finally, the collections data stored in EmbARK Web Kiosk's database is not accessible by a local file indexer.

The scope of this project quickly grew beyond offering straightforward access to collections data; it became a larger project about user experience and search engine technology. These technical and user interface design challenges represented a unique opportunity for three departments to work together. As a team, the three departments could see the challenges and implications from several points of view, and keep the details as well as the big picture in focus. The Collections Information and Access Department (CIA) is responsible for the management and maintenance of the EmbARK system and the EmbARK Web Kiosk. The Interactive Educational Technologies group is responsible for the production of our rich, educational multimedia and represents the needs of our diverse audiences through the interpretation of our Collections for the public. And, finally, the Web group is responsible for bringing it all together on the museum's institutional Web site and representing SFMOMA's online institutional presence in a well-designed, user-centered manner. Also informing the project's perspective is the San Francisco-based Web design firm, Perimetre-Flux Design, the developers and designers of SFMOMA's Web site and our ongoing partners in other projects.

Data Preparation and Delivery

SFMOMA has been working with Gallery System's EmbARK software since its development phase in 1992. We began actively using EmbARK as our collection management system when we moved into our new building in 1995. More than 80 staff members at SFMOMA need desktop access to EmbARK, but the institution only has a 20-seat user license. Staff members in Education, Communications, Development, the Library, the Museum Store and other departments who don't have access are able to review collections information only through printed reports obtained from staff with EmbARK access.

With the release of Gallery Systems' customizable web component, EmbARK Web Kiosk, we decided to install the product on our intranet and make it available for staff. (Fig. 1: EmbARK Web Kiosk on SFMOMA intranet) This allowed us to decrease the number of desktop users of the EmbARK database, reduce the need for training and supporting these users, free up our user licenses, and test the ways in which people use online collections information. This was a baby step towards our larger goal of providing collections information on our Web site.

Figure 1

The staff of the Collections Information and Access Department (CIA) was excited about moving towards online, public access to collections information, but, like many museums, we were protective of our data. Concerns about misappropriation of images and text, publishing potentially incorrect information, and the public's potential disincentive for visiting the museum were among the issues. However, several years prior to the release of EmbARK Web Kiosk, members of the CIA staff were instrumental in the Museum's decision to join the Art Museum Image Consortium, or AMICO (http://www.amico.org), as a founding member.

The AMICO submission requirement of 500 images per year served as a catalyst to help us get our digital images, rights, and object data ready for online publication. As part of our participation in AMICO, we produced digital images, researched copyrights, and formalized in-house approval procedures for publishing information electronically. CIA hired an imaging specialist to create professional quality files and an intellectual property specialist with experience in electronic rights. Other CIA staff checked records, did basic research and got curatorial input. Conservation and Registration contributed physical object information and Education contributed didactic material from our interactive educational programs. Ultimately we streamlined the process and focused on three groups of objects (new accessions, exhibitions, and loans). This approach helped us capitalize on the attention these objects were already getting in the normal course of our work.

After we published these records through AMICO, our next step was to make them available to staff—and then the public—through EmbARK Web Kiosk. Our curators and senior staff approved the use of all permanent collection records for use on our public Web site, provided that certain caveats—such as "we are continually updating our information"—were published on our site. As we tested the EmbARK Web Kiosk in-house, we determined that it would have to be refined and modified for the public through a user-centered design process.

Research and Usability

From the start of the project, the team has been committed to focusing on user-centered design and usability. We thought this project would present us with a great opportunity to apply user interface design principles from the for-profit Web world to a museum site. Our research started with a report published by the Nielsen Norman Group, "E-commerce User Experience: Design Guidelines for Search." (Nielsen, et al, 2000) The members of Nielsen Norman Group are leaders in the field of user-centered design and usability, and although a few of the design guidelines in their report are specific to searching for products on e-commerce sites, most of the guidelines apply to all forms of Web searches and contain many valuable lessons for museums and cultural institutions.

The report can be ordered online through the NNG site, and we will not attempt to summarize the full report here. We will discuss three basic guidelines that we applied to our functionality design specifications for the project.

Make search visible

The design of the SFMOMA site at the time of this paper's publication requires users to click on the SFMOMA logo in order to reveal the search box. The only hint users have is the word "search" next to the logo (Fig. 2: Detail of SFMOMA navigation).

Figure 2

In addition, the current design does not feature a search button as on most sites; the user must hit the Enter key to submit a search. Although this design may look sexy and win us design awards, the project team agreed that when it comes to functionality as basic as search, we simply cannot afford to present an obtuse user interface. As searching is one of the most basic functions on a Web site, we redesigned the interface for the search box so that search is clearly visible and easy to use. (Fig. 3: Detail of redesigned SFMOMA navigation).

Figure 3

Make the default search simple to use

We debated whether or not our new site search should have scoping parameters such as "Search Collections," "Search Calendar," and "Search Entire Site," but we determined that simpler was better. Just because our back-end databases for our collections and our calendar of events are separate systems, the user should not have to worry about this. Many users do not understand the difference between these options and will end up failing in their searches. Our feeling is that narrowing a search should be a user's decision, not a requirement of the search interface.

Provide information about alternative ways of locating content.

This principle reminds us that users should be able to find more than that for which they were searching. Search results should be relevant and clear, but results pages should also contain what we call "value-added" content. Because we have so much rich content buried on the SFMOMA site, we know that users may not find everything we have to offer even after we have improved our user interface design and re-engineered our search engine. Therefore, as part of the design, we will surface our multimedia content on the search pages. This will allow users to "stumble upon" relevant content when searching, just as they would when exploring the stacks in a library.

Functionality Design

Our functionality design document has been an evolving outline of the project's requirements and scope. It maps out all new and existing screens. The functionality design process is important because it pushes the team to define all scenarios. Each scenario yields a set of screens and all the elements those screens will contain. After incorporating the guidelines of the NNG report into our functionality design document, we contacted one of the NNG user experience specialists who was kind enough to give us a full-day of pro-bono consulting services under the aegis of her own consulting firm. In this intensive one-day workshop, we reviewed search engines on commercial and museum sites, developed profiles of our users, mapped our goals to our users' needs, and further refined our functionality design documents.

Using data from our Web logs and emails to the webmaster, we were able to break our users into three distinct groups: people who want artist/artwork-specific information or thematic art-related information; people planning to visit the Museum who want logistical information; and, lastly, people seeking administrative-type information or wanting to interact with the institution. We then analyzed what the needs of these users are ("find out if SFMOMA has any works by Picasso," "get the Museum hours," "view job listings," etc.) and mapped our goals to our users' needs. Our most basic goal was to provide information these users can't currently get from our site (namely, collections information), but an equally important goal was to provide better access and more gateways to the wealth of information we do have.

Content Design

Our focus in content design is to surface otherwise-buried multimedia materials and provide users with multiple paths of exploration. Some of this content already exists outside of the Collections section on our site and is not searchable with our current search engine; other content is being developed or modified in the first phase of this project. Some of the ways in which we are surfacing content include creating object-specific links to Making Sense of Modern Art (http://www.sfmoma.org/msoma/) that will be searchable through EmbARK, designing thematic pages highlighting popular categories in our collections ("abstract paintings," "Pop Art," etc.), and adding links to multimedia offerings on search and Collections pages.

Our decisions about content development came from our desire to surface existing materials, but they are also grounded in the user patterns we discovered in our Web server logs. An analysis of our logs revealed that the number one search performed on our site was for specific artists or artworks in the SFMOMA collection. One user in particular, tried several times to locate information about Diego Rivera's The Flower Carrier with no results. It was obvious from the specificity of the query and the repeated attempts made that this person knew that we have this work in our collections and that it is almost always on view. Not only was this committed searcher unable to find an image or basic label information, he/she never found the 35 screens of rich content (including archival film footage of Rivera working on murals in the 1930s and video commentary by art historians) we have in our educational feature, Making Sense of Modern Art. Given that these rich interpretive materials live on our Web site, it is critical to make those connections accessible through the search function.

Making Sense of Modern Art is our on-going multimedia exploration of artworks in the permanent collection. It is designed to allow users to explore questions that surround works of art. All of its screens are designed in Flash to allow for maximum interactivity. The downside is that Flash is difficult to search and we have not yet invested in the action scripting work to make it easily searched. The Making Sense interface is organized by artworks, with each work having its own artwork screen (Fig. 4: Diego Rivera Artwork Screen, http://www.sfmoma.org/msoma/) from which a user can explore questions or zoom into the work.

Figure 4

One of our most basic goals is to ensure that artwork-specific screens from Making Sense of Modern Art will be searchable. Currently there is only one entry point for Making Sense and a user needs to navigate through the interface to reach individual artwork screen. We have modified the programming architecture so that each artwork screen has its own direct URL. These URLs can now be entered into EmbARK as basic links. This means that a user doing a search for Diego Rivera would find the Rivera data screen in EmbARK with a link directly to the Diego Rivera content in Making Sense of Modern Art. Paper prototyping verified that users are very interested in these links to multimedia content.

Our server logs showed us that 11% of our users perform what we call "thematic searches." That is, in addition to doing straightforward searches for specific artists or artworks, users are searching across larger themes in art, such as "abstract painters," "women artists," and "color field painting." We are currently reviewing data from our logs and data from similar institutions' logs to establish what might be the top 25 thematic queries on art museum Web sites. We plan to create new pages, grouping together the content we already have to address users' interest in those themes. We can thus showcase thematic multimedia content that has already been developed and may be of interest to browsing Web users.

Finally, another focus of our content development has been on what we are calling "value-added" content. This is a way to present compelling multimedia features for which users have not specifically searched. We can add links to this content at various steps along the way of a user's search process (on search results, advanced search, and Collections pages, for example) to intrigue those users who are open to browsing. During paper prototype testing one user noted that he expects "the museum to give me its best stuff." Users count on us to present our most relevant and compelling content. Presenting rich, deep content in conjunction with our search engine is one new way for us to do just that.

Our ultimate goal may be a complete restructuring of our Web site, and our current steps are heading us in the right direction. With these new ways of presenting content in place with the new search engine, we should be able to study our user logs and investigate visitor behavior to learn more.

Paper Prototyping

When our functionality design document contained enough details for us to create screen mock-ups, we were ready to do some paper prototyping with users. In consultation with the usability specialist from Nielsen Norman Group, we developed a user testing script and some paper "computer screens."

We recruited five users from our galleries and compensated each user with a catalogue, a poster, and free passes to the Museum in exchange for 30 minutes of their time. One of us acted as the facilitator while two of us observed and took notes. We gave each tester a pencil and told them to point at the screen with the pencil to "click" and to write on the screens with the pencil to "type."

We found our paper prototype testing to be extremely useful in examining our concepts and terminology. For example, almost every user had a different understanding of the terms "multimedia" and "interactive." We realized that we will need to do further terminology testing to chose the best terms in the appropriate places in our navigation.

Although our computer "screens" were print-outs of a Word document, we found the paper testing to also be useful in analyzing our page layouts. Many users didn't see links that we thought were very obvious, and noticed other links that we didn't think were very important in the information hierarchy. We plan to do several paper prototypes and tests with real HTML screens throughout the project.

Technical Approach

Now that our collections data is ready for the public and we have defined our interface and design goals, we are ready to identify a search engine solution. One option is to customize Internet Information Server's index server so it can search our SQL database and the EmbARK Web Kiosk. This solution, presented to us by a contractor, would be a fully custom solution requiring the development of search algorithms. Although the programming work for this is affordable, the long-term maintenance implications are too risky. This solution is also unnecessarily complex, requiring real-time querying of our SQL database.

The mission of the Museum is not to develop proprietary software and at this point we feel we will be better served by choosing an existing product. We have also determined that a spider indexer would better serve our needs than local file indexer solution. Spider indexers locate files by following links, indexing the linked pages, and then following each link from those pages until they have crawled through every page on the site. As long as long there are links to our database-driven content, it will be searchable by a spider.

We are currently examining two possible scenarios: purchasing a search software product that can be installed and configured to run on our Web server, or licensing a hosted search solution that will run on a third party server. Although a hosted solution requires a monthly maintenance fee, it will not require in-house maintenance and expertise. We do not have any web engineers on staff, and this is one of the reasons we are leaning towards a hosted solution at this point. In either case, our first choice for either scenario at this point is Google, as we feel that the Google engine is the most sophisticated engine available.

Conclusion

While the data and content development strategies are in place, the bulk of the implementation work is still ahead. Our next steps include completing all components of the interface design, and selecting and customizing the search engine—with iterative user testing throughout the process.

With the completion of this project, we will have a new set of tools to better serve our users, and a renewed commitment to do so.

References:

Jakob Nielsen, Rolf Molich, Carolyn Snyder, Susan Farrell, (2000). E-commerce User Experience: Design Guidelines for Search". Downloadable PDF Published by Nielsen Norman Group, orderable online at http://www.nngroup.com/reports/ecommerce/search.html.

General Search Engine Resources

http://www.searchtools.com

http://www.searchenginewatch.com