Published: March 15, 2001.
Re-Purposing of Content and Digital Delivery Convergence: Implications for Interface Design
Slavko Milekic, M.D., PhD, The University of the Arts, USA
With the introduction of the digital medium, an abundance of content previously obtainable through other media became available in digital form. Digital delivery implied the necessity for some kind of content modification dictated by the specific characteristics of the digital medium.
The situation is further complicated by the convergence of different media used for the delivery of digital/digitized information. The same content has to meet the constraints of different display devices and bandwidth limitations. Furthermore, digital delivery makes possible adaptations of content not only in regard to characteristics of hardware used to display it, but also in regard to individual user's goals and needs.
In this paper I will present the implications for content re-purposing with the migration from the traditional to the digital medium as well as the problems that arise from convergence of different types of digital delivery mechanism.
The effects of the 'discovery' of the digital medium are global and profound. I have described some aspects of these changes elsewhere (Milekic, 1999). In this paper I will analyze two currently observable trends connected to the introduction of the digital medium. These trends are:
a) re-purposing of content previously available through other media for delivery through the digital medium;
b) convergence of means (physical devices) used for the delivery, display and manipulation of digital content.
Each of these trends creates complex constraints but also opens up new possibilities for content delivery/exchange. These possibilities can be explored only by modifying the traditional interface design practices of applications created for digital content delivery. However, before proceeding to the analysis of general effects of these trends, it is advisable to clarify the consequences and implications of each one separately.
According to the on-line version of Merriam-Webster dictionary, a literal definition of re-purposing would be that it is a new way (from Latin re-, again) of achieving the same outcome (from Latin proponere /to propose/: something set up as an object or end to be attained: INTENTION). In most general terms re-purposing can be described as manipulations of space, time and meaning within the new medium. Most often re‑purposing involves some kind of compression. Compression of data space (ultimately corresponding to the physical storage space necessary for storing it) in a medium currently constrained by bandwidth limitations is frequently proportional to time compression. However, with bandwidth limitations resolved, the space compression will be relevant only in regard to the characteristics of a user's display device. This does not mean that the need for time compression will disappear. In future, time compression will not be tied to how long it takes an image to be downloaded to one's hard drive, but to how quickly one is able to use (understand) the content. In other words, time compression will be related to the compression of meaning.
In regard to the digital medium, one can distinguish two different kinds of re-purposing: a) one that refers to re-purposing of content from another medium (that of print, audio or video, etc.) for delivery in the digital medium, and b) re-purposing that occurs within the digital medium. These two forms of re‑purposing can be labeled as inter-media and intra-medium re-purposing.
Inter-media re-purposing is historically not a new occurrence. A clear example is a shift from oral tradition to written transmission of myths and folk tales. In order for oral content to be successfully transferred into the new medium, it had to be adapted to the constraints imposed by the new medium. In this process some of the characteristics of the content and possible interactions with it were lost or changed, while new ones were gained. For example, oral traditions rely on a very undependable storage medium -- human memory. Historically it has often been the case that certain folk tales and myths were confined to the brain of a single individual, upon whose death they would be irretrievably lost. Committing these to print preserved the content and made it available for transition to any number of (literate) individuals in the future. The other side of this transition is that some possibilities were lost. Human memory may be unreliable as storage medium, but it is inherently flexible and creative. Storytellers freely added and embellished the story so that it suited particular circumstances. This led to the enrichment and the creation of different versions of popular myths. Transferring such material to print meant that it had to be 'edited'; that is, redundancies had to be removed, inconsistencies corrected, and one had to settle for a single 'true' version of the content (it is only much later that the value of the diversity of oral interpretations was recognized and is now extensively documented). It is important to note that these changes were mandated by the characteristics of the medium itself.
Transferring (re-purposing) content from any other to the digital medium will be affected in a similar way by properties of the digital medium. In order to anticipate the effects this transfer will have on content accessibility and usability (and user interface design), it is important to identify in advance the specific characteristics of the new medium. Properties unique to the digital medium are consequences of binary encoding of information and, as I argued elsewhere (Milekic, 1999), at the most general level they can be labeled as:
Reproducibility refers to the fact that any content transferred into the digital form becomes instantaneously reproducible. As a matter of fact, the very act of making it available to our senses (by displaying it on the monitor screen) involves reproduction. Reproduction in a digital medium is perfect; that is, reproduced code (even if it was reproduced millions of times) is indistinguishable from the 'original' one. Thus, the notion of the original in the digital medium is going to be very different from the one used for works in traditional media. Most likely, it will be defined temporally as the time of the first occurrence of a certain binary pattern, before digital 'originals' started multiplying.
As instantaneous as reproduction, transferability refers to re-creation of a binary code at another location, with the loss of the 'original' code. The ease with which any portion of binary code can be reproduced, transferred and modified led to the pervasiveness of the medium. We are already at the level where any bit of information stored in this medium is simultaneously accessible to anyone with access to a digital portal.
It is the level of control over manipulations in this medium that is unlike any other traditional medium. Not only can one manipulate information at the level of a single bit, but one can also selectively interact with the data specified at a level of a particular pattern. For example, it is possible (and trivially easy with modern rendering software) to change only those data bits corresponding to a certain shade of red in a digital photograph. Hyper-linking of data (which may be in different modalities) represents yet another level of manipulability, the one at the level of meta-structure. Thus, the information in the digital medium is infinitely personalizable, malleable, modifiable and adaptable. Because of its creative potential the manipulability of digitally encoded information has been the driving force behind the globalization of digitization.
However, blind use of recognized properties of the medium can also lead to less-than-perfect results. Good examples are early attempts to re-purpose text documents for use on the World Wide Web. Re-purposing was done with a knowledge that the medium allows for hyper-linking of any two points in the information space. Although the abundance of hyper-links made the text seem more usable, their net effect was not any greater than what would be achieved with an automatic page-turner.
Re-purposing within the digital medium is achieved by modification of certain aspects of the content or its delivery mechanisms. Depending on the motivation for re-purposing of the content, one can distinguish two broad categories which I will label
a) device-centered content adaptation
b) user-centered content adaptation
Very often the line between the two is blurred because the distinction is not made between the users who access certain content and the devices that they use for this purpose. Needless to say, equation of a user's goals with the characteristics of the device being used to reach it leads to serious flaws in interface design. For this reason, I will describe these two different categories separately.
a) Device-centered content adaptation
In the field of HCI, re-purposing of the content focused mostly on technological constraints of different devices used to access the content and not on user interests, goals. etc. Examples of device-centered content adaptation are provided in an excellent paper by Ma, Bedner, Chang, Kuchinsky and Zhang (Ma, Bedner, Chang, Kuchinsky and Zhang, 2000). They provide five general categories of content adaptation applicable to different data modalities within the digital medium: video, image, audio and text.
They label these categories as:
Data prioritization is achieved by assigning different levels of delivery priority to different content aspects. These could be related to content modality, for example, by assigning priority to delivery of text prior to the delivery of images or audio and video information. Or it can be achieved by prioritized progressive delivery within the same media type, for example, by delivering images first in low-resolution version, and then filling-in more detail if time and/or user display device allow.
Data transcoding is the process of converting data format. In their paper the authors (Ma et al., 2000) refer to data transcoding only in the context of client display device characteristics. This would include transcoding of original images to display-appropriate format; for example, GIF-to-JPEG or color-to-grayscale. However, it is important to note that some forms of data transcoding may also provide other forms of content adaptation, like information abstraction. This is demonstrated in Figure 1 which depicts the same information (photograph of a young woman) transcoded to formats compatible with different display and bandwidth characteristics.
Figure 1. Sequence of images with different levels of data (and hence time) compression, ranging from 232 kilobytes for uncompressed image in TIF format to less than 10 kilobytes for duo-tone image.
Notice, however, that image 4, although claiming the least amount of data space (and thus the least amount of download time) provides the viewer with much more information due to the addition of a "thought bubble" which provides relevant contextual information for the interpretation of the image.
Modality transformation is the translation of the information delivered in one modality into another modality. Examples include transforming digital video stream to a series of still images, delivering only the audio component of the video, or closed caption text. True modality transformation would not be characterized by loss of information, and therefore one should distinguish it from modality transformation-based reduction.
Information abstraction refers to the procedures that reduce bandwidth requirements by delivering extracted information to the user. Ma and his colleagues (Ma et al., 2000) provide examples such as video highlights, video frame and/or rate reduction, image dimension reduction, image data size reduction (by increasing compression rate), stereo-to-mono conversion, text summarization, font size reduction etc. However, these examples mix both semantic and non-semantic information abstraction -- reducing image data by using a better compression algorithm does not have to lead to perceptible loss of information and have an effect on meaning, while text summarization may.
Figure 2-3. Information abstraction by reduction of font size and reduction of white space allows the text to fit into the browser window without scrolling and without any change to the text itself.
Figure 3-4. Further compression is achieved by condensing the text and presenting it in a list form.
Another difference is that some of the given examples use automatic algorithmic procedures (like image compression, thumbnail generation, audio subsampling) while the other ones depend on human editing (text summarization, video description and such) to achieve data compression. Thus, it may be useful to divide this category into two separate ones: data compression achieved through different automatic procedures, and true information abstraction achieved (at least for the time being) through human intervention.
Purpose classification is the least clear category suggested in the above-mentioned article (Ma et al., 2000). While it is definitely true that "a typical Web page contains a lot of information and media objects that are redundant or may not be of interest to a user" by which they refer to advertisements, banners and promotional material, it is often the case that the very existence of a particular page depends on displaying this (redundant) information. Although the authors invoked this category with the goal of meeting low bandwidth characteristics of different user devices, they are coming very close to user‑centered content adaptation, by discussing the purpose and usefulness of ads for a user.
b) User-centered content adaptation
Adapting the content so that it can efficiently accommodate users with different needs, skills, backgrounds, knowledge level, etc., is another challenge in the research agenda for universal usability (Shneiderman, 1999). It is worth noting that tools for user-centered content adaptation focus on different kinds of problems compared to the device-centered content adaptation. In this case the problem is often not the network bandwidth but rather the cognitive bandwidth. The unique properties of the digital medium also play a much larger role in user-centered content adaptation by providing the opportunity for the development of new cognitive tools for knowledge sharing and discovery. For example, the potential of the digital medium to log all of the interactions that take place in it can be used for a qualitatively different level of the preservation of the past. In the following section I will briefly outline different user populations and possible strategies for re-purposing the content to suit their needs.
Adapting content for use by different age groups (from preliterate children to adults) has traditionally been resolved by creating a new genre; for example, children's books, movies etc. This approach has been implemented successfully on the Web on the NASA space mission sites that have sections designed for K-12 children. Ideally, the content should be made adaptable for different age groups (children, adults) by creating an interface mechanism that would enable the user to fine-tune the level of complexity of the displayed content.
The content should also be made available to fit a user's interest and immediate goals. Relative failure of algorithm-based search engines to satisfy user needs indicates that this challenge has not been satisfactorily resolved. More successful Web portals, like Yahoo, owe their success to the fact that segmenting of the content to fit different users' interests has been done by humans. However, recent advances of "data mining" techniques (also known as "knowledge discovery in databases" - KDD) hold promise that in very near future one will be able to "build knowledge" by searching vast amounts of text available on the World Wide Web or be able to search through video materials in the same way one searches text today. Other examples of making the content of other media (TV) more adaptive by transferring it into the digital form include ReplayTV and TIVo, devices which allow users automated recording and replay of preferred shows (and even learn from viewing behavior), slow motion replay, pause and resume of live TV broadcast.
On the side of content providers, re-purposing is necessary for achieving different goals. The same content has to be presented and interacted with in a different way depending whether it is going to be used for teaching/learning, information, or as a data base. This idea is still in its infancy, although some large content providers (NASA, NY Times, as well as some museum sites) are making an effort to present part of their content in form suitable for use in education.
Re-purposing of content should also address the issues involving different cultural expectations. Currently, English language and Western cultural models dominate the World Wide Web. Issues of culture-specific content delivery have potential broad economic implications, and there are already some services that provide automatic translations of Web pages into different languages (AltaVista). However, the quality of automatic translation is severely limited, and commercial Web sites catering to users belonging to different cultures resorted to the creation of multiple, human translated and acculturated mirror sites.
Adaptation of content for users with special needs is a large and looming issue that has not been satisfactorily resolved. It is worth noting that answering the needs of these populations has historically been the path to improvement in general usability or led to the introduction of appliances with global impact (like the telephone, tape recorder, etc.). Technologies like continuous speech recognition, screen readers and haptic-enabled interaction devices (like the iFeelÔ mouse) will play a major role in this area.
The term convergence has several meanings (Merriam-Webster), of which two are especially applicable to the topic of this paper:
The two above definitions illustrate both the problems and the state of affairs in the arena of digital convergence. Wireless telephones, PCs, television and various other Internet appliances are competing with each other in taking each other's function. One can get the weather report on the display of mobile phone, watch TV on a computer screen, and interactively shop over TV. This, in itself, can only benefit the end user if there is true uniformity between the platforms. However, this is not the case. The number of competing standards and architectures is staggering.
There are three major aspects of digital convergence (Forman and Saint John, 2000):
What is evident today is the user-demand-driven convergence of content -- the use of re‑writable CD ROMs is increasing because of the possibility of storing large amounts of digital data that are common household items - like digital photographs, music files in MP3 format, multimedia presentations, etc. That the trend is going in this direction is also evident from the fact that DVD players/recorders (which can store even larger amounts of data) are the most successful electronic product in recent years.
Convergence of delivery platforms is a much thornier issue that has not even been resolved for the shift from analog to digital television. We are witnessing a kind of parallel evolution similar to the second definition of convergence quoted above: different platforms are developing similar characteristics, and the 'selection forces' are user preferences and the characteristics of the medium itself.
The questions of delivery of digital content are also far from being resolved. Whether the content will reach us via fiber-optic cable, broadcast or satellite is not clear. Although it may seem that the questions of the delivery method are irrelevant, for the end user they have enormous implications for the future of power and control over the media.
While it may seem pointless to speak about interface design in light of chaos and unresolved issues in the domain of digital convergence (further complicated by lack of clarity regarding the characteristics of the digital medium), there are some general design guidelines that seem evident. In simple terms, the design has to be device-aware and user-aware. This means that the content has to be scalable (useable with devices with different display characteristics, storage space, processor power) and adaptive (useable by users of different abilities, goals, backgrounds and cultures). Use of intelligent algorithms and adoption of common hardware standards can resolve scalability of the content. Creation of truly adaptive content is still a huge challenge and currently can be resolved only by human effort in creating adequate redundant content representations that suit different needs. On the other hand, adaptive delivery of content (and 'awareness' of user actions) is part of the digital medium potential, and will eventually determine the ways in which the medium is used.
Ultimately, delivery in the digital medium (and use of the term digital medium) will include any end-user delivery method where the content, at some point, was encoded in binary form in order to use some of the advantages of the digital medium (for example, reproducibility). This would include a printed page (from a printer), or an email message retrieved with a mobile phone via text-to-speech software.
Theoretically, the need for re-purposing should disappear if the medium is used to its potential: that is, the delivery of practically limitless accumulated content becomes infinitely adaptive (and user adaptable) to meet any individual's need, knowledge level or cultural background. Of course, we are far from this ideal, and in the meantime, we still have to develop efficient ways of inter- and intra-medium re-purposing of content.
AltaVista search engine, on-line. http://www.altavista.com
Forman, P., Saint John, R.W., (2000) "Creating Convergence", Scientific American, November issue, on-line http://www.sciam.com/2000/1100issue/1100stjohn.html
Logitech iFeelä mouse on-line: http://www.logitech.com/cf/products/productoverview.cfm/79
Ma, W., Bedner, I., Chang, G., Kuchinsky, A., Zhang, H. (2000) "A Framework for Adaptive Content Delivery in Heterogeneous Network Environments", presented at Multimedia Computing and Networking 2000 (MMCN00), January 24-26, 2000, San Jose, California, on-line http://www.cooltown.hp.com/papers/adcon/MMCN2000.htm
Merriam-Webster on-line dictionary http://www.m-w.com/home.htm
Milekic, S. (1999) Emerging tools & techniques of digital media: history log and multiple futures, in Bearman, D., Trant, J. (eds.) Cultural Heritage Informatics: Selected papers from ICHIM'99, Archives and Museum Informatics, Pittsburgh
NASA educational sites on-line: http://www.grc.nasa.gov/WWW/K-12/John/K-12_Teacher_Resources.html
New York Times Daily Lesson Plan on-line: http://www.nytimes.com/learning/teachers/lessons/
ReplayTV on-line: http://www.replaytv.com/home.htm
Shneiderman, B., (1999) Universal Usability: Pushing Human-Computer Interaction Research to Empower Every Citizen, Institute for Systems Research Technical Report, T.R. 99-72, on-line http://www.isr.umd.edu
TIVo on-line: http://www.tivo.com