Archives & Museum Informatics

info @ archimuse.com

www.archimuse.com

published April 1998
updated Nov. 2010

A Virtual Interactive Art Gallery

António Ramires Fernandes, Interactive Systems and Multimedia Group
Hugo Castelo Pires, Universidade do Minho
Rui Rodrigues, Universidade do Minho

1 Abstract
2 Introduction
3 Modeling The Gallery
4 Augmenting Reality - Displaying Multiple Exhibitions
5 Using Visual and Audio Clues to Provide Information and Enhance Realism
6 The Exhibition Manager for the Gallery Staff

6.1 Adding Paintings to an Exhibition
6.2 Placing the paintings interactively

6.2.1 The Object/Placeholder Paradigm
6.2.2 Object Management and Information

7 Building Your Own Exhibition

7.1 Adding the Artworks and Information
7.2 Security measures

8 Conclusion & Further Work
Bibliography

Providing access through the Internet to a 3D environment of an Art Gallery is just one of the components of this project. 

This project has three major components. The first provides a 3D interactive editor for the gallery staff to help them previewing the disposition of the art objects in the gallery. With the help of the editor the gallery's staff can arrange not only the art objects, but also the lighting, sound, and some other details. A second component brings the virtual exhibition to the Web, with the virtual and the real exhibitions opening at the same time. The Web exhibition attempts to resemble as much as possible the real one, including sound conditions and the lighting according to the current time. 

It is however the third component of this project which is perhaps the most innovative: this is an interactive exhibition builder for the Web which provides net surfers the possibility to build their own exhibition. With this simulator the user will be able select the art objects to include amongst a selection from the gallery, or their own work, and their disposition, the lighting conditions including the number of lights, their position, orientation and intensity. Sound is also contemplated providing the user with the ability to change the ambient music. The users can then submit their own exhibition to the gallery for appreciation. The main goal of this component of the project is to bring the gallery closer to the community. Furthermore, this component allows the Art Gallery to open a "Exhibition Designer" competition for net surfers in which they would submit their own exhibition for appreciation. The selected virtual exhibition could then be replicated as a real one.

This project started out as a modeling experiment of a real Art Gallery using VRML 2.0 so that it could be seen on the Internet. 

VRML stands for Virtual Reality Modeling Language. This denomination can be misleading in the sense that Virtual Reality is usually associated with immersive systems which is not necessarily the case with VRML. Furthermore, the modeling primitives in VRML are not comparable in extent and flexibility to those present in 3D modeling packages. 

VRML is mainly a language which allows one to define a 3D world to be viewed on the Internet with real time navigation. Currently on version 2.0, besides providing modeling capabilities, VRML also provides primitives for interaction as well as defining animation. 

The shortage of graphical primitives in VRML 2.0 when compared to commercial 3D packages is due to the real time navigation feature in a 3D scene, i.e. only those primitives which were fast enough were included. Also some graphical primitive's features usually found in 3D packages to provide a high degree of realism were simplified for performance's sake. 

Performance is therefore a main issue when modeling 3D worlds with VRML. The performance problem is increased by the fact that, due to the availability of this project on the Internet, the 3D models can be viewed in a wide range of hardware architectures, from slow to super fast computers. There had to be a possibility for those with slower machines to enjoy the 3D model, and at the same time, for the better equipped ones, to take advantage of the faster hardware configurations. Some of the solutions found are presented in section 3. 

Although we tried to build a model which was as realistic as possible, at the same time we enjoyed the freedom of building a virtual world. Extending reality to provide multiple exhibitions was one of the options explored. In order to achieve this an 8 floor building was created, in each floor a different exhibition is presented. An elevator was added to the model, replacing the real door, to allow the user to travel between floors. This is described in more detail in section 4. 

Another objective of this project was to provide information to the user in a non textual way. The visual and audio clues defined not only provide information about the timetable, and the time of day but also enhance the realism of the virtual model. See section 5 for more details on this. 

The project, once concluded, is to be delivered to the Gallery staff. Therefore the process of creating a new Exhibition has to be as intuitive as possible, just like hanging paintings on a wall. The purpose of the Exhibition manager is two fold: to allow the Gallery staff to build their own virtual exhibitions for the web, and also to help the staff to build the real exhibition. 

To create an Exhibition an hybrid application using HTML, with Javascript, and VRML was developed. The Javascript deals with the information and exact positioning of the paintings and spotlights to be included in an exhibition, whereas the VRML allows the visual positioning of these items interactively in the gallery model. This process is described in detail in section 6. 

Finally we proposed ourselves, and the Gallery staff, a last challenge: to allow the gallery visitors to build their own exhibitions, using only an Internet browser with a VRML plug-in. Exhibitions can be built either using a pre-selected set of paintings from the Gallery, or with the visitor's own paintings or photos. In the later case, the only requirement to build an exhibition is to have the paintings or photos available on the Internet. This will allow the Gallery to open an "Exhibition Designer" contest for net surfers in which they could submit their own exhibition for appreciation. The selected virtual exhibition could then be replicated in a real one. Furthermore, this can be used to promote young and less know painters or photographers which would otherwise have no possibility to have a real exhibition at the gallery. This last component of the project is presented in section 7. 

Conclusions and further work are presented in section 8.

This project will soon be available at http://www.geira.pt/mns/ 3. Modeling The Gallery (toc) 

Our initial approach was to use a commercial 3D package with the ability to export to VRML for the modeling task. This approach turned out to be frustrating for two reasons:

• The VRML file size was huge when compared to a rough estimation of the file size needed to do the modeling from scratch in VRML.
• Adding interactivity later on proved to be quite difficult because we had to gain previous understanding of the structure of the automatically generated file in order to see how to program the required actions.
  
For instance consider the spotlight model we included in the gallery. Below a VRML and a commercial 3D package versions are presented, see figure 1. 

 

figure 1. Two versions of a lamp model. The left one was done in a commercial 3D package, the one on the right was done in a text editor.

The commercial package version translated the model into a VRML file with more than 23k as opposed to 1k for the version created in a text editor. Furthermore, the commercial package version uses 672 polygons, whereas the version created by hand uses only 49 polygons.  

The above example clearly shows that exporting to VRML is not always a good option. The file size has direct implications not only on the download time but in some cases, as in the example above, also on the number of defined polygons. 

For the above reasons this approached was abandoned. A specific VRML editor was also considered, but if animation and interaction is to be added to the model then the currently freely available editors for PC's known to us were not also up to the task. 

The interior of the building where the real exhibitions take place is quite geometric and simple, see figure 2. Although, as mentioned before, the graphical primitives in VRML are simplified versions of those present in 3D packages, they were sufficient to easily create a model of the interior of the gallery. Therefore we decided to create everything in VRML without the help of any graphical editors, just with a text editor. 

figure 2. A detail from the virtual gallery

The modeling process is relatively straightforward as far as the interior's architecture is concerned. However adding spotlights to the model involves some important considerations. The VRML specification requires VRML browsers to be able to display a minimum number of 8 simultaneous lights. Currently most browsers do only the bare minimum to comply for performance reasons. Some engineering was required to define the light's properties and positioning so that 8 lights sufficed. 

The main issue is however related to the way VRML treats light. The VRML lighting model is a gross approximation of the real world lighting. The adopted model emerged from a compromise between realistic rendering and speed, with a strong bias towards speed. For instance shadows and reflections were not contemplated in the adopted lighting model. Fortunately shadows and reflections were not critical in our model. 

Understanding the way the lighting of a polygon is computed is critical to modeling in VRML. From a theoretical point of view, the VRML specification defines a lighting model which requires lighting to be computed for each pixel. However, with this lighting model and today's hardware, performance would decrease drastically. In order to cope with standard hardware configurations the majority of the current VRML implementations only compute lighting for each vertex of a polygon. The lighting for the remaining pixels of the polygon is computed as an average of the vertex lighting computations. 

The implemented lighting model implies that for a particular polygon if no vertices are lit then the whole polygon is unlit. This has major implications in the way the spotlights lit the walls and paintings. If a spotlight is pointing at the center of a wall, and if the light cone does not include any vertices, then the whole wall will be unlit. In order to solve this problem either the angle of the light cone is increased or the wall is constructed as a set of tiled polygons. The former solution is unrealistic in the sense that real spotlights usually do not have a large enough aperture so that a vertex of the wall is lit. The latter solution on the other hand, while providing a more realistic lighting effect, has the disadvantage of increasing the number of computations required to compute the lighting factor, thereby slowing down performance. 
 
Figure 2 shows the lighting effect of a spotlight on two tiled walls. The smaller the tiles, the smoother the effect. 

figure 3. Different levels of lighting detail for two walls built built using tiled polygons

In order to provide a more realistic approach we implemented the later model, where each wall is build using a number of tiled polygons. This model turned out to be very demanding on the hardware, requiring a medium high level computer according to today's standards. To allow net surfers with medium low end hardware configurations to visit the gallery a lighter version, where each wall is a single polygon, was also implemented. In this version the lighting is not as realistic, on the other hand it provides a smoother real-time navigation, allowing more modest hardware configurations to be able to cope with the computational requirements. 

We wanted to have both the realistic and the simpler model online. We did not want to deprive users with high end hardware configurations of the more realistic model, on the other hand we wanted less equipped users to be able to enjoy the gallery's virtual environment. Figures 4 and 5 show the different renderings obtained. 
 

figure 4. The low level of detail version    figure 5. The high level of detail version

However we did not want to have the users deciding before they entered the gallery, for instance to make an option in a HTML page before actually entering the museum. Defining recommended hardware configurations for each version is difficult due to the number of hardware parameters involved like main processor, graphic cards and memory, not to mention the subjectiveness of the concept of smooth real-time navigation. Furthermore, we wanted to stick to 3D as much as possible. The approach taken was to have a light switch in the gallery to allow the visitor to switch between the two implemented models and choose the more appropriate for the visitor's hardware configuration in an interactive way. The device switches from one version to the other instantly, thereby allowing one to decide in real-time which version is more suitable to enjoy the visit. 

Another performance problem appeared when we started adding pictures to the Exhibition. VRML has a nice texture treatment with a considerable set of features and options to manipulate textures. However textures are performance killers. The higher the number of textures the slower and jerkier the navigation. 

The first action to deal with this problem was to reduce the file size for each picture. This doesn't sacrifice the quality of the pictures because the VRML window is relatively small to allow for smooth real-time motion. 

The second measure was to remove the texture from the walls of the gallery. The walls of the real gallery are painted but have a textured effect, so we did detach the model from reality. Nevertheless the difference is almost unnoticeable and the gain in performance clearly makes it worth while. 

It may seem that removing a single texture, when there are roughly 12-15 paintings in each exhibition would not affect the performance significantly. However the real time motion smoothness has to do with the number of textures which are visible from the current viewpoint, and not with the total number of textures. The wall was almost always visible, whereas, in average, there are only about 2 or 3 pictures in view at any time. Removing the wall texture therefore consisted of removing a quarter to a third of the number of visible textures in average. Furthermore the wall's area is much larger than the paintings area, which means that in average there were more pixels with the wall's texture than with the painting's textures.

There is no reason to limit the use of the gallery's 3D model to show just the current exhibition. One of the advantages of computer graphics is that once they are done multiple instances can be easily created. Therefore its only natural that one would use the model created to provide more than an exhibition. Possible candidates for this purpose are past exhibitions. There was also a request from the gallery staff: to use the gallery for their ancient photo collections, three photo collections from Braga, a city in Portugal, in the early century, the 30's and the 50's. These collections are not available to the public, so the virtual gallery was a suitable way to present these works which otherwise wouldn't be know to the public. 

Other possible candidates are exhibitions which can't take place in the real gallery due to its tight schedule, presently the gallery has already exhibitions scheduled up to the year 2000, that's 2 years from now! Previews of real exhibitions are also on the list of possible uses for virtual exhibitions. 

Traditionally the exhibition selection would be done in a 2D HTML page, with links to the several exhibitions. However, as mentioned before, we wanted to stick to 3D as much as possible. One could have a control panel somewhere inside the gallery with buttons to switch between exhibitions. This option was dismissed because it was too 2D like, not taking advantage of the freedom provided by 3D. We wanted to provide a true 3D interface. 

The approach taken had a clear and intentional departure from reality. An elevator was created, in the same location where the door is in the real gallery, which would allow the user to travel between exhibitions. In this way we used the freedom provided by 3D modeling to create a virtual 8 floor building to present multiple exhibitions. Figure 6 provides a skeleton of the virtual building. 

The floor organization is as follows: 
 

• 3 bottom floors, one for each of the ancient photo collections. These floors were placed underground to emphasize the notion of a deposit, as mentioned before the photo collections are not available to the public;
• The ground floor, which is the entrance to the virtual gallery, contains the current exhibition.
• The other 4 floors can be used at will by the gallery staff, for instance to display past exhibitions or to present virtual only exhibitions.
     figure 6. A Skeleton of the Virtual Building   The elevator was modeled to resemble a real elevator, both in appearance, see figures 7 and 8, as in behavior. Inside the elevator there are 8 buttons, just like in a real elevator, one for each floor, plus a button to close and open the doors. The doors are transparent to provide the movement sensation of a real elevator. 
    figure 7. View from Inside the Elevator     figure 8. View from Outside the Elevator

Some care has been put in order to have the virtual elevator's behavior as similar as possible to a real elevator. Some of the elevator's implemented behaviors are as follows: 
 

• When the user gets in the elevator the doors close automatically.
• Once the elevator arrives at a floor the doors open
• When the user leaves the elevator the doors will close.
• The button corresponding to the current floor is highlighted to help the user to keep track of his/her position in the virtual building.
  
For optimization reasons and to avoid keeping the user waiting for the download of all the exhibitions, only the paintings of the current exhibition are loaded initially. When the user goes to the elevator and presses a floor button the paintings of the respective exhibition are loaded. This may create a delay when the user arrives at a floor because there may be no paintings available at the moment. Nevertheless, in the worst case, when the user visits all exhibitions, the sum of the delays each time the user enters a floor for the first time will be equal to the delay required to download all the exhibitions initially. Furthermore the initial delay will be much shorter than if all the exhibitions were to download when the user enters the virtual gallery.

This was a challenging part of this project. The objective was to take advantage of being in a 3D environment to provide information resorting to visual and audio clues instead of direct or textual ways while at the same time enhancing the realism of the scene. 

The first task was to let the user know if the gallery was open or closed during the visit. The first approach was to put the timetable text somewhere inside the gallery. This would be a direct textual way of providing the information to the user. This approach was abandoned because this kind of presentation fits much better in a 2D HTML page than it would in a 3D environment. 

We adopted the local gallery time instead of the visitors local time, so if someone is visiting from Canada at 7 am the visitor will find the gallery open because the local time is at least 3 hours later, depending on where in Canada the visit comes from. 

To provide this information we resorted to the window shades. When the gallery closes the shades are closed as well. Note that the shades are only translated, not rotated, so the visitor can still see the street in front of the museum through the two large windows. 

The closing and opening of the window shades occurs when the gallery closes or opens, respectively, independently of the time the visitor entered the site and the duration of the visit. This was achieved using the TimeSensor provided by VRML to implement a real-time clock. 

Because the gallery has two large windows to the street it is usual to see people looking inside when the gallery is closed. The real shades are only fully closed, thereby not allowing one to look inside, 3 hours after the closing of the gallery to the public. The model also implemented this behavior, rotating the shades and blocking the view to the street at the same time the real shades are fully closed. Again this approach was preferred to having a textual information somewhere in the 3D model. Figure 9 shows the different positions for the shades. 
 
 



 
figures 9. The three positions for the shades   Furthermore different pictures, taken at different times of day, were used as textures for the window views, thereby providing a more realistic environment inside the gallery. The lighting used to simulate the light coming from the exterior through the two large windows is also decreased at night, and reduced to zero when the shades are fully closed to enhance the realism of the virtual lighting. 

Finally, there is a church just in front of the gallery. We thought that adding the chimes would be a nice touch to enhance this virtual experience. Therefore, using the timer defined previously for the shades, we set the chimes to play at each quarter of an hour, and at the hours. We could have provided a clock, either digital or with hands, to provide local time information to those visiting from outside Continental Portugal. 

Adding a clock however would imply to add elements which are not present in the real gallery without enhancing the mood of the scene. The chimes, together with the window views and the shades, on the other hand help to set the mood of the real gallery, with temporal information being provided as a side effect.

To help the gallery staff to create the exhibitions, an exhibition manager is provided, along with an exhibition editor to allow in a practical and intuitive way the design of an exhibition, i.e. the placement of the paintings, lighting, and sound. 

An exhibition is primarily defined by a name. Some additional information may be provided, namely a background image and text colors to be used for the automatically generated 2D exhibitions, and music for sound sources existent in the 3D exhibition.

As images are necessary to represent the paintings (and eventually other art forms), they are an important part of the information needed to create an exhibition. In order to ease the task of specifying the location of the images used, a special "image warehouse" is consulted, and paintings are added to the new exhibition by moving each image found to a WWW site containing all the exhibitions. This implies that the only images existing in the warehouse are the ones which can be used to create the exhibition.

The next step in creating an exhibition is setting up the placement of the artwork and lighting. 

This is accomplished by providing a 3D interactive model of the gallery in which the user can use drag and drop techniques to move objects around, taking some constraints into account, complemented with input forms which allow exact positioning of the objects.

To specify the constraints applied to the objects, the concept of placeholder is defined. Each type of object can only be placed in the respective type of placeholder. For instance, paintings can only be placed on walls and spotlights on ceiling gutters, see figure 10.  figure 10. Spotlight in the gutter

These placeholders are always considered to be rectangular portions of a plane, so objects can only dragged in a two-dimensional way, within the dimensions of the placeholder. The object orientation is independent of its position in the placeholder. 

This generic approach allows defining different object/placeholder pairs. Some have already been mentioned: painting/wall, spotlight/gutter. Others can be defined, for instance statue/floor.

Apart from the position and orientation of objects, each particular kind of object has additional information, like spotlights’ intensity or paintings names, author and dimensions. This information can be changed in appropriate forms of the exhibition editor, along with facilities for creating and removing objects and changing the placeholder where they are placed. 

As mentioned before, two interfaces for placing the objects are provided. With the interactive 3D interface, the exhibition designer can move the paintings within the walls in which they are placed by dragging the mouse over the former. One can also drag the base of a spotlight along the gutter, and rotate the lamp in order to point it wherever one wishes, in a realistic fashion. 

figure 11. Visual Editor 

The visual interface, see figure 11, has however a limitation inherent to this type of interfaces, the accurate placement of the objects is hard to accomplish. The exact positioning of objects can be required for some exhibitions. To workaround this limitation, the position of an object can also be controlled in the respective object information form, see figure 12, by directly changing the values, using keyboard input, that define the relevant information, for instance <x, y> position of paintings or spotlight angles. 

figure 12. javascript editor 

These two interfaces are interconnected so that changes on one of them are automatically reflected on the other, in order to ensure consistency and enable the preview of the exhibition. Also to maintain consistency, no object is allowed to move outside the limits of its placeholder.

This is what we consider the most challenging part of the project: to allow internet surfers to build their own exhibitions. The degree of interaction present on a tour in a three-dimensional exhibition is high. Allowing users over the net to build an exhibition themselves goes far beyond that. 

Besides providing for little known or resourceless artists a virtual space for presenting their own work, it also allows the promotion of exhibition contests: the winner would be able to see its virtual exhibition become real within the gallery walls. These contests could be made with art previously exhibited in the gallery, or original work from the exhibitor.

By letting ordinary users build their own exhibition the problem of where the resources needed for the exhibition should be located arises. In order to build an exhibition the images of the works must be available on the Internet. This would be a major restriction a few years ago, however presently with so many sites providing free web space this requirement is no longer a limitation. 

Using a set of forms similar to those defined for the gallery's own exhibition builder, the user can add paintings to his/her exhibition by providing the url of the respective image, plus the information about the works to be displayed. 

The user can create an exhibition in as many sessions as required. Images and respective properties can be added, removed or altered at any time during the construction of the exhibition. The same goes for the spotlights and music. 

When the exhibition is concluded it can be submitted to the gallery for approval.

7.2. Security measures (toc)

Due to the number of users that can have access to this kind of service, some security measures must be taken. These measures aim mainly to assure the following three points:

• no user can modify another user's exhibition;
• no exhibition can be visited by anyone besides the creator and the gallery staff until the staff approves it;
• after the approval by the gallery staff an exhibition can't be altered by anyone, even the author;
• the images and sounds must be available on the internet during the time the exhibition is made available.
  
To achieve the first goal, each user is assigned an identifier and a password. The password authenticates the user, and the identifier is then used to reference his work. 

The second goal is a very important one, since the gallery cannot be associated with any kind of activities other than cultural. As the real exhibitions must meet some requirements, so do the virtual ones. Being so, a user can create and edit an exhibition at will, in several sessions if he wants, but for the exhibition to be available to the web community, he must submit a request for evaluation. If the staff considers the exhibition to be appropriate, it classifies it as such. 

The third point guarantees that the exhibition that the staff approved is the one which is made available on the internet. It is imperative that the user must be prevented to alter the exhibition without the prior consent of the gallery staff to prevent content replacement, otherwise there is always the possibility that the new contents of the exhibition don't meet with the gallery staff's approval. If the creator wishes to change the exhibition at a later time, he must submit a request to the gallery, which will reverse the process, making the exhibition unavailable to the public and open to modifications. 

The fourth point is equally important, an exhibition without images is just an empty building. 

In order to comply with the third and fourth requirements, once the exhibition is approved by the gallery staff, all the images and sounds are copied to the gallery's Web server. In this way not only is their availability guaranteed, but also the contents are not replaceable.

Looking back at the project as a whole, modeling was probably the easiest part once the performance problems and VRML limitation were identified and dealt with. 

However this project is not exclusively about modeling, it has to do with exploring the potential of 3D to provide an environment where, although reality is kept to a certain extent, it is also augmented to provide options which are not available on the real gallery. Our attempt to achieve this was through the use of an elevator that allows visitors to visit not only the real exhibition but also past or completely virtual exhibitions. 

Providing visual and audio clues for the gallery visitors in a non direct and non textual way was also attempted using chimes and different window views instead of a real clock, and using the shades instead of a textual timetable. 

The exhibition manager was probably the hardest part; we wanted to provide an intuitive interface both for the staff museum and for net visitors to ease the process of creating an exhibition. This tool also helps the Gallery staff to create virtual exhibitions to preview the position of the paintings for the real exhibition. 

Allowing net surfers to build their own exhibition is definitely the boldest part of this project. Using this feature the Gallery can provide opportunities for young and less known painters or photographers to show their work to the community. Also the possibility of having an "Exhibition Designer" competition can help bringing the Gallery closer to the community. 

Bringing the community closer to the Gallery is what this project is about. The objective in modeling the Gallery in 3-D and presenting several exhibitions simultaneously is twofold: first to arise interest in those which can visit the real exhibition and have them visiting the real Gallery more often; secondly to eliminate geographical barriers allowing those that can't visit the real Gallery to appreciate the art work, which otherwise they wouldn't have the chance, in a richer environment than a simple 2D HTML page. 

This project can be further expanded to provide an even richer environment. For instance the gallery sometimes has exhibitions with statues, and other 3D pieces. This is an important issue that leads to some research in the area of 3D scanning. 3D artworks could use the floor as placeholder and allow the orientation of the piece by controlling only the rotation of the vertical axis or optionally any of the three axes, although this later option maybe neither necessary nor practical). 

An alternative to the 3D scanning of an object would be mapping a panorama of the object over a cylindrical or spherical surface. A panorama is composed of several images taken around the object at different angles of fixed intervals. Those images can then be "stitched" together to provide one single image of the object all around or become frames which are changed depending on the position of the observer. Depending on the surface used, the panorama could be made in one or more axes. 

Another possible enhancement would be to create a multi-user 3D world inside the gallery, so that people could enjoy the art works and simultaneously exchange ideas and opinions about it with each other. The author of the paintings or photos could provide a live worldwide presentation of his/her work and discuss it with the gallery's visitors. 

A virtual character could be created to guide visitors through the exhibition, providing the author's view of the paintings. Extra virtual worlds could be created to provide information and mood for each painting. 

The possibilities are endless and only limited by the imagination...and the hardware, and the bandwidth...

VRML Interavtive Tutorial, available online. 

The Annotated VRML Reference Book, available online. 

"Computer Graphics: Principles and Practice", Foley, van Dam, Feiner and Hughes, Addison Wesley, 1996. 

.   
 

 
Last modified: March 14, 1998. This file can be found below http://www.archimuse.com/mw98/ 
Send questions and comments to info@archimuse.com