Or will it? In sharp contrast, Max Anderson hypothesizes another vision:
|"The demand for the original work will increase rather than decrease, following repeated exposure at an institutionally authorized site on the World Wide Web or its successor."8
Many see transformative qualities of the web. For Anderson it will be positive:
|"Although we may bemoan the extent to which we are different in our contributions to society today from what we were 30 years ago, we cannot return to a time when audiences were tolerated rather than listened to, information was sporadically shared, and amenities were an afterthought. And for that we are without question better institutions."10
Besser, though, seems cautious about the losses to be incurred in the transformation of museums on the web as "‘the aura’ is lost," referring to Walter Benjamin, "the viewing of a unique object within the same context and setting as everyone else."11 As the museum becomes more like a library rather than an archive, he predicts that visitors will want an interactive role, the authority of the curator will be diminished and that of high culture in general.12 Actually such a transformation is what many museologists have been waiting for. Douglas Worts and Chris Morrisey13 with their strong backgrounds in issues of museum education and visitor research, point out the potential of technology in museums but only "if we adopt new paradigms of understanding our visitors and their experiences", adding:
|"Technology has the potential to perpetuate (or even inflate) what is wrong about museums and their communication with visitors by reinforcing the authoritative perspective to the exclusion of other voices. Alternatively, at its best, technology can help all of us see new relationships between objects, information, the experiences of others, and our own personal responses to the world in which we live."14
We are led to our second question after we have settled the first question of the web suitability to museum expression: what is it of the museum experience that we want to convey? And in what manner? What evaluation approaches will we take to get us to our goal and to see that we have reached it?
[ RETURN to OUTLINE ]
II. The Missing Discourse:
Strangely, thus far in the evolution of discussions of museums on the web there have been only a few murmurs about museological precepts and in particular about the importance of the public experience of the museum in our discussions of museums and museology on the web.15 So far in my readings about museums and the web, the conceptual museum approaches of collections managers, of information scientists or even research enthusiasts overwhelmingly inform the museum web discussion; at the same time in practice museum efforts reflect a public relations and commercial impulse in web development with some important educational and interactive exceptions. In discussions about the web, we also see the development of the argument that the museum is essentially about information, although sometimes linking this informational view to the social and transforming nature of museums. This viewpoint it would appear has very much been adopted by those who are involved in the development of new technologies in museums. Are there other views of the museum that can expand our view of the web presence?
For one, the field of evaluation of museums and their publics has a century of study behind it and a way of thinking about museums that has fundamentally altered our way of conceptualizing a museum but some of the older notions remain in museum discussion as permanent and repeating themes. The existing approaches to museums and the web, for the most part, ignore yet another idea of the museum which has had some support for decades. This view holds that museums are about people and the meanings that they are trying on the one hand to convey or, on the other the meanings that they are making of the museum experience. People then, not the object or the information, are at the centre of the museum experience. It is people who use and save the objects, it is people who form collections and then it is people who create the power structures and environments of the four-walled museums. It is also people -- whether individually, in a group or a community -- who make up both the visitors and the non-visitors to museums but who are still museum clients. It is these people whose view of the museums give it existence whether in practical terms of tax dollars or legislative validity or in the sense that they come to the museum, accessing it directly or indirectly, for research,enlightenment, education, entertainment or status. So, too, by extension, it is people who are at the centre of the museum experience on the web.
If so, then the work of studies about the public in, as well as general attitudes to, museums become critical to our thinking about the museum in its myriad of forms including that of being on the web. Yet, it is noteworthy that little bridging has occurred between the museological fields of visitor research and web development. Is it just that the web museum displays the same characteristics of all web endeavour that undervalues the user-perspective in design and development?
At the same time I would argue our impulse to go to the web occurs without the context of the whole area of Human-Computer Interface or User-Centered design work, built from fields such as computer science and industrial design, information studies, education and cognitive psychology and Human-Computer Interface and now turning to look at the web.
This paper, then, will attempt to begin to bridge several fields of interest including that of research about the museum learning and experience and new museological perspectives as a context for our museum web vision. It will follow with a discussion of user-centred, or Human-Computer Interface research and some of the nascent museum web evaluation work going on with suggestions for future development.
[ RETURN to OUTLINE ]
III. Musing on the Muses.
Before we begin to lay out the state of research about museums and their publics and the web and its publics, then, we need to set the stage of the museological discussion. Current research about the museum and the museum experience takes us into a museum paradigm based in the user-created experience of museums that goes to the very essence of making museums. This is an idea of museum that could adapt to the web in ways that support, and extend, the museum idea. From this viewpoint, then, what is the essence of the museum experience that we wish to transfer to the web and what can we effectively create in web technology given current developments? More than the object fetishism, more than information and data transfer, and certainly more than public relations and sales opportunities, the museum experience is about meaning and knowledge building that is based in the visitor, or in people’s experience of the museum. Such a perspective, I suggest profoundly alters the way in which we would approach museum web making.
Let me just unpack some of the views of museums that are around us. It is possible to map the concepts of recent, and indeed past, discussions of museology around a number of problematiques of museological models-of-knowing, at the root of all reflective practice and the essence of thinking about museums.16
[ RETURN to OUTLINE ]
Problematique #1. What is a Museum?
The fundamental question that divides our thought about museums on the web is the elementary question of what is a museum? For at the heart of this discussion is the idea of museum that you have, and consequently the idea of museology that goes with it. Let me try to release you from the notion that there is any one, correct, historically accurate definition of a museum. As Francis Henry Taylor wrote in 1945 in Babel’s Tower: The Dilemma of the Modern Museum, "Each generation has been obliged to interpret this vague word ‘museum’ according to the social requirements of the day."17 Too often, we work within stereotypical definitions of museum arguments within the museum profession that are quite ahistorical. Furthermore, we operate without the benefit of an overview of the conceptual frameworks of museology that are available to us and make up the over one hundred years of the study of museums.
Many of the philosophical discussions about the museum center around the questions of definition and terms like "traditional" and "new" museums are part of the historical landscape. It has been an unproved axiom of museums that museums are at their essence unique institutions devoted to concrete objects, physical things, the material remains of the past. From this view another variation on the definition of museums is framed, as Weil and Harrison 18 have pointed out, in terms of the functions and operations of museums as concrete and tangible phenomena, much like the collections on which they are based. First, there are the collections; using a linear view one evolves to what to do to them, to collect, conserve, present and interpret in some combination of terms. It becomes difficult to include museums or museum-related organizations with living collections or even no collections, or those based on an exhibiting model wherein the material shown comes into the place rather than being held there permanently. It is also difficult to include a museum model based on experience or concept as in the case of science, interpretive centres or art centres where the experience of electricity, of nature or of art becomes the component of the museum experience. Within this view, starting as it does with the things on which museums are based, people, those who view the museum, are often thought of last in the line of functions -- the auxilliary function of museums.
All too often this interpretation of museums sees the museum in terms of a building and the functions within it. The praxis of museums would dominate the discussion in terms of the how to, the what rather than the why. Most often, thought about the museum -- and certainly education or training to work in one -- is reduced to expertise about a specific discipline with some add on techniques that support the application of that knowledge within a museum.
Most often the view of museums centres on "things" or "objects" as the essence of the museums role and function, the beginning point of the collecting function at the heart of museums role. Interestingly, Maxwell Anderson with his enthusiasm for new technologies and the web still retains the strong belief in the museum’s special role as a place of "objects". For him, the power of the web is that of bringing the visitor to the museum for the unique confrontation with art and to share in the same experience known to curators and connoisseurs.19 But is it the objects or the ideas they represent that museums hold and nurture? This is a question long argued in the museum world. The classic debate between Benjamin Ives Gilman and George Brown Goode,SAS SAS BlackBrown Bravo Bravo 20 that every museology student is introduced to in the early months of their study, was reenacted in the debate between Cameron and Knetz and Wright in the late 1960s21 and reappears in the web discussions.
MacDonald and Alsford echoed these earlier debates when they raised the view of museums as information within their models of museums' roles,22 claiming that the "advantage of a shift in orientation away from objects towards information is that it should make it easier (when formulating museums’ mission, for instance) to balance the traditional functions of collection, preservation, research and display. A more recent version of the debate has occurred around the Information Highway discussion. Deirdre Stam proposed "the importance of the ‘information base’ underlying museums missions and functions, and its potential for supporting more cohesive and integrated institutions."23 Stam interprets the axioms of The New Museology as invoking the museum’s information base, "The full complex of data supporting institutional activities ranging from the pragmatics of acquisition to the abstractions of interpretive display."24
Although usually separated conceptually by a functional view of museums, the information elements of the museum, although generated by different departments and processes, are conceptually tied, because they are connected to the whole purpose of the museum and its mission. Stam also raises the implications of tying the information base of the museum to the axioms of The New Museology of "value, meaning, power, control, interaction with visitors, interpretation, understanding, authenticity and authority."25 She then continues to take the ideas of information and The New Museology to museum praxis, internal and external operations, to see how they can be reworked to respond to the new conceptualization of museums with information as a resource at the heart of its functions:
|"The approach involves integrating internal information such as coordinating curatorial and conservation files, providing wider access for staff and public to newly coordinated institutional data, drawing more deeply from sources that reveal the context of objects (through more assiduous use of published material and original archival resources), and preparing more sensitively for relating to the community at large (by conducting research on and with visitor constituencies)."26
Others, often with information science, library science or collections management orientations, have picked up on this approach to the new reform of museums in the Information Age and describe the museum and its survival in terms of the Information revolution. But there is a big difference between the terms ‘information’ and ‘concepts’, at least in their implications. While I value the view of museums as information places, I think that this definition of museums can stretch the museum off balance as much as traditional notions of museums as pure object collections generated phenomena and could lead to a philosophy of information for information’s sake. This is an approach to the technological iterations of the museum that forsakes the experience base of the museum as places of meaning making and knowledge and understanding. Even George MacDonald and Stephen Alsford have written of the other equally important modes of museum including the Museum as symbol, vision, showcase, treasure-house, memory, communicator, mentor, celebration, host and resource.27
With the mention of George MacDonald, we cannot escape another related debate, what are the appropriate roles of museums as especially defined as seen between the oppositions of education and entertainment and the move to Disneyfication? Echoes of these arguments are found within the web and the differences of design approaches from the entertainment thrust of a David Siegel to the user orientations of Jakob Nielsen. 28 So Negroponte can actually refer to himself as P.T. Barnum, a great museum man of a type for the nineteenth century. One feels that some of the discussions about museums on the web have to do with old ghosts of the "dime museums" of earlier museum history as recast in the onslaught of the "heritage industry" and the movement of museums to web entertainment forms.
An alternative view to any of the above is one that is essentially phenomenological, moving to the view that museums are steeped in the human psyche. The museum is defined in more abstract terms as an idea, even a human process, that takes a variety of forms from the personal collection to the formality of the British Museum. Flipping the traditional model of museum functions, the visitor is the start of the museum phenomenon. Beyond this it is also possible to go beyond the visitor to people in general. At the heart of this view, lies the idea of the making of museums, possible to all peoples, across all cultures and all classes and a wide variety of forms, that can include material evidence, animate or inanimate from art to artifact to anteater, or ideas and experience, encompassing the phenomenon of living.
If you believe that the museum today is one static form, resembling the large state museums of nations, then you will see the trend to the web as a fundamentally new entity, which resembles that of its forebears only in small part. In contrast, it may actually be possible to posit that the museum, is a dynamic, complex and variable human form of endeavour, an entity which has shown over time adaptive qualities as it transforms to new societal developments at least in some form. Therefore, the museum with the web is yet another adaptation of this human phenomenon of collecting and showing, which has been in evidence in human civilization for many, many centuries, and hence can be analyzed with museological ideas and tools.
At this point, we must acknowledge that we have fundamentally inverted the definition of the museum, to one that is much more about human experience, about people as museum makers and as those who make meaning in museums. This is a person-centred model of the museum idea. This shift in emphasis acknowledges that museums are not set in time, but are still constantly being created and re-created. In North America, I believe, that our notions of the living foundation of museums comes as much from the nearness of living cultures that contest attempts to put them in cases, whether that be of First Nations or other cultural communities.
By this argument, clearly, I am not suggesting the transferring of the museum as traditionally and statically defined over to the web but a definition that is more dynamic and historically set in its complex forms of human act of saving and showing material culture, and its reflection upon the individual and social psyche.29 Thus, it is in the personal experience of museums that the essence of the museum lies. The museum as idea then can rest in the museum that you make in the corner of your house, in the eight-columned imitation of a Greek temple, or indeed on the web in a virtual museum or of a museum virtually represented. The museum then exists in a triad of modes -- object, meaning, and personhood. It is in the interrelationship of these that the museum exists and this multiplicity of functions knits the variety of views about museums together into a dialectical argument that reasserts itself again and again over time. Perhaps it is time to reject false oppositions and accept the ambiguity of the museum definition as human and social form that has its own permanence and longevity, that is inherently about things, ideas and people and their interrelationships.
Consequently, if you believe that the museum today exists in a complex of forms, sizes, types and intents, that is essentially mirroring the idea of museums--a meta-museology, if you like -- rather than one specific form of museum then you will see the museum on the web as yet another form of communication that can be as museological as any other and which shares some of the same tensions of politics, and meaning of the four-walled physical spaces of museums. Perhaps the very essence of museums and their relationship to people and objects can actually be finally fulfilled through use of web and Internet technology to invoke multiple discourses about the meanings.
[ RETURN to OUTLINE ]
Getting to New Museology, the New Museum
and Web Museology
Ideas sympathetic to the interpretation of museums as phenomenological as identified above have been brewing for decades, if not centuries, and have gelled in the last twenty years in the form of what is sometimes entitled "New Museology". It is a view of museums as places related to social, economic and political contexts needing higher relevancy and meaning to their cultures and the significance to community and certainly to individuals and the process of identity. Although Peter Vergo brought the term to book form in 1989,30 the term had been around for over a decade growing out of the ecomuseology movement in France since the late 1960s and paralleling the museum reform movement in various parts of the world. Probably, Julia Harrison is correct in claiming our time as unique for the congruence of these questioning perspectives:
|"The ‘why’ of the existence of the museum is not a new issue, but placed in the context of the new historicism, post-modernism and deconstructionism it takes on a new urgency as these paradigms expose new understandings of museums." 31
Of course progressive theories of museums have existed for some centuries.32 Although not often acknowledged, there is a long tradition of criticism of museums resembling this second view that has existed since the first museums. Rejection of the old and traditional museums, devoted as they are to the whims of an elite, that might be defined as upper-class, sometimes middle-class or inside groups of curators, directors or boards, is followed by a call for a "reform", "new", for some form of change and a movement to community.33 Instead museums should be fundamentally for everyone in society. So T.R. Adam wrote in 1937:
|"To someone outside the world of art criticism, there must seem an element of almost mystic faith in this belief in the power of great paintings to communicate abstract ideas of beauty to the uninformed spectator...When background is lacking -- where there is no knowledge of what the artist is attempting to say in terms of time, place, or social meaning -- the resulting impression is confused and is likely to be painful as pleasurable." 34
Another view of the question of what is a museum is discussed in museology in terms of the appropriate role for museums; should they be centres of research, education or entertainment. In 1971, Duncan Cameron published his classic article "The Museum: A Temple or Forum?" that has influenced so many modern museum discussions. For Cameron, museums were in an "identity crisis," split between their roles as a temple or as a forum. His answer was that museums should be both, but especially they "must institute reform and create an equality of cultural opportunity." For Society would "no longer tolerate institutions that serve a minority audience of the elite."35 Many authors have written in similar vein, so we find echoes of the views of the social and educational mission of museums in the AAM’s Museums for a New Century and reiterated forcefully in the AAM’s Excellence and EquityBlackBrown Bravo SAS SAS Bravo 36 and in a myriad of books and articles in various parts of the world dealing with museological exploration.
The approach was perhaps most clearly articulated by the ecomuseum movement that started in France, crystallized in the work of George Henri Riviere, one of the original and most influential museologists, and Huges de Varine. The word ecomuseum was first introduced in the 1971 ICOM general conference in Grenoble and first depicted at le Creusot, Montceau les Mines. The large established museums, like the Louvre, represented "old or traditional museology". "New Museology " was about the local community’s attempt to retain its identity amidst extreme industrial transformation using the idea of a museum as social action. These ideas were reiterated in the work of René Rivard and Pierre Mayr and in the ecomuseology movement in Quebec and used in the formation of the Musée d’Haute Beauce. 37
According to museological thinkers like René Rivard, 38 we should note that the museum has evolved from the confines of an institution to a territory and located in a community. The objects have now become the community and the staff are now facilitators engaging in empowering the people to curate and show their own things. Aside from judgments of whether any ecomuseum can truly be participatory in power and direction or whether the institutionalizing tendencies soon take over, we can see that the idea of the ecomuseum has deeply captured thinking about museums in different parts of the world. One of the areas of the world that has been at the forefront in evolving ideas of ecomuseology is that of the countries of Scandinavia that influenced such ecomuseologists as René Rivard and in turn have been influenced by them. In a 1992 piece, Ewa Bergdahl, Director of the Ekomuseum Bergslagen (founded in 1984) shared the philosophy:
|"To maintain the living spirit in museum work means conducting a continual historical debate amongst the participants in the museum project in which the attitudes to the past are constantly remoulded and new questions are being asked ... She therefore considers that one important task should be to enhance understanding and knowledge of man’s role in his physical environment [sic] and the connection between energy, raw materials and human activities...The goal of the museum -- to develop an instrument of social analysis, identification and active historical awareness -- is high but not unrealistic. A process-oriented museum is never finished. The pursuit of a balance between the body and the soul of the museum is fundamental and never-ending." 39
The movement for New Museology was occurring in parallel to other questionings of the museum that took the form of analyses of power, ideology and representation and deconstruction combined with questions of the cultural participation in museum by a variety of communities. Museums are being analyzed in terms of their relation of museum to society, economics and politics as part of the discourse of their significance, relevance and meaning. By now many authors have focused on the analysis of museums in terms of the authority they represent and have begun to examine alternatives. Some have approached this critique in terms of Marxian theory as in the work of Duncan and Wallach,40 (1978). Many others have used the precepts of anthropology to develop a framework for the analysis of power of representation particularly of one people by another, often in terms of the problem of museums in holding and showing the "other."41 For some this is through the lens of ideology, rampant in the museum setting.
Still others have adapted theoretical frameworks based on the literary theories of linguistic and semiotic analyses of museums meaning.42 We have seen similar discussions locate on the meaning of objects and material culture particularly.43 In the end, the ideas that museums set a context for meaning and as a consequence that no environment can be neutral have become part of the discourse on museums as we meet the millennium.
Lisa Roberts sums up this rethinking of the role of the museum as a place where meaning is created and negotiated:
|"By omitting any mention about the decisions behind the determination of object’s meaning, museums exclude visitors not only from an awareness that knowledge is something that is produced but also from the possibility that they themselves may participate in its production. ..Thus stated, the work of interpretation becomes an act of empowerment, because it provides with both the knowledge and the consent to engage in critical dialogue the messages museums present."44
Peter Walsh, raises the question of the Web and the "Unassailable Voice" of the museum experience, the tone, the attitude that "pervades museum labels, brochures, exhibitions, catalogues, the guided tour, audio-visual presentations, and now Web sites. For the most part, it is both impersonal and disembodied: ... "45 Walsh believes in the transformative potential of the linking of the museums and the Web. "This collaboration, properly directed, can not only bring the wealth of museums to a far wider audience, it can help replace the traditional "Unassailable Voice" with one that is kinder, gentler, less pompous, more interesting, and, ultimately, far more inspiring."46
If we were at all unsure that museums are places of negotiated meaning that can translate into very real political situations the last twenty years have, particularly in North America, brought us to an overwhelming realization of the politicization of museum work and capacity for controversy. Examples abound from the Mapplethorpe show and or "Enola Gay" or in Canada "Spirit Sings", "Into the Heart of Africa" or the more recent debate over the placement of a Holocaust Gallery at the Canadian War Museum in Ottawa.47 At the root of these contests is the question of the authority of the museum especially of the curation, and sometimes an exhibition team, to control the message and form of the exhibition. Museum anthropology and some museologists have led the way in this discussion in Canada as evidenced in the work of Michael Ames, Bruce Trigger, Julia Harrison and in the work of the Assembly of First Nations and Canadian Museums Association.48
In North America over the last 10 years, some seminal works have set the tone for the discussion about museums, partially in response to these theories and controversies. Two of these resulted from conferences held at the Smithsonian Institution that were followed by publication of texts: Exhibiting Cultures: The Poetics and Politics of Museum Display and Museums and Communities.49 Questions of the museum's role in cultural diversity have become part of the discourse as attempts are made to broaden the base of museum support and involvement.50
No longer is it possible to see museum collections or displays as "neutral" statements, even those that have attempted to contextualize material culture in a culturally ‘sensitive’ way. Further museums are, perhaps awkwardly, exploring fundamentally new ways of working with groups in society through advisory, consultative and cooperative methods and even introducing the concepts of shared ownership. Now we struggle to evolve methodologies of collaboration with community groups that are impossible to define fully and to push our exhibition means to allow for multiple voices and authorities; yet within the framework of a civil society that pursues some degree of fairness for all communities.
I am not yet aware of any museum web pages facing such controversy but we can all imagine it. While we are all wrestling with the thorny issue of copyright, political controversy and censorship are around the corner.
[ RETURN to OUTLINE ]
Problematique #2. The Museum For Whom? From Object to People: Experience, Memory, Meaning and the Ideas of Knowledge-Building
Another premise of newer notions of museology as identified above must involve the public(s), whether named as visitors, cultural groups or community, within the power equations. We may discuss access to the museum in terms of who comes or of the intellectual access when they do come to a museum and in terms of the entire museum as an effective communication system. We have for decades attempted to permeate the four walls of the museums, to extend the physical reach of the museum by means of outreach programs, circulating exhibits, school kits, buses and trains, and satellite museums to open museums and access galleries that bring the communities into the four walls. Further, we have expanded the notion of access to museums beyond the physical to that of the museum experience, thereby inviting questions of personal and social identity, memorization and meaning-making as well as those of politics and negotiation.
Worts and Morrisey invoke the words of Neil Postman who states that museums express a need to answer the question of "What it means to be human?" They continue:
|"Part of our contention is that the museum is not simply a building, a collection and expert information. Rather, it is perhaps more fundamentally "a place of the muses’, which is first and foremost a creative psychic space with the experience of individuals. The physical museum in which we work is better understood as providing a set of conditions that can facilitate an individual’s experience of the muses. If this is true, then the new communication technologies need to be understood and developed with an awareness of their role as facilitating experience- not delivering it."51
So what part will museums on the web have in answering the question of "What it means to be human?"
In recent years, the philosophy of constructivism has had a major impact on thinking about museums particularly in the areas of museum education, learning and the nature of the museum experience. The ideas of constructivism have been brought to the museum by theorists like George Hein, John Falk and Lynn Dierking and last, but not least, to museums on the web by the work of Jamieson MacKenzie among others. Hein offers a good summary:
|"Current education literature is dominated by discussions of constructivism. This new name for a set of old ideas has major implications for how museums address learning. Constructivism is particularly appropriate as a basis for museum education if we consider the wide age range of museum visitors. How can we accommodate this diverse audience and facilitate their learning from our objects on their voluntary, short visits?"52
Accordingly, MacKenzie, who also espouses constructivism, proposes the idea of "Learning Museums" where students in classrooms become the curators of virtual museums. These are museums that are not only virtual but local and yet global, dynamic, multidisciplinary and multi-sensory, and continually changing.53 According to MacKenzie, "Virtual museums offer a different kind of learning-one which is fresh and vibrant."54
It turns out that the philosophy of constructivism has been brought to other places that are moving to the web. Dr. Carl Bereiter and Dr. Marlene Scardemelia’s work on the concept of "Knowledge-Building" has been honoured by a number of web thinkers including Seymour Papert who prefers the words "Knowledge-Nurture". In the last few years these knowledge-building ideas have begun to influence museologists, like the Art Gallery of Ontario’s Doug Worts, some of whom are working on the web and ideas about the museum on the web.
Rather than following the models of "Ask the Expert," where the exchange is person-to-person, one with the goods and the other not, or of "Cross-School Research Project", where students work together or alone to infer knowledge from sources, Bereiter and Scardemelia propose the operation of "The Knowledge Society in Miniature."55 In this model, all kinds of people and organizations are linked, schools, universities, museums, parents, businesses across a variety of localities by means of an interactive database, CSILE. A constructivist premise, these people who are involved in the actual work of society, "engaged in the construction, utilization, and improvement of knowledge," are linked so that cross-sector interaction can occur built on participants learning. The intention of this model is that through involvement students do not work within enclaves but interact with the real world of "knowledge work."56 These same ideas of knowledge-nurturing are at the root of the CSCW (Computer-Supported Cooperative Work) and CSCL (Computer-Supported Cooperative Learning) movements.57
As a result of many of these latest ideas of museums and their publics have brought a shift in museum exhibition and education theory to visitor centred learning and meaning making, as at participatory-learning stations of science museums, or the San Francisco Fine Arts Museum "Game Room", and at the AGO the "Share Your Reaction" cards and "Explore a Painting in Depth" audio guide. Clearly the implications of these museum learning philosophies will have an impact on the making of web pages and their evaluation.
[ RETURN to OUTLINE ]
Memory and Meaning
Along with ideas of the way in which learning takes place, constructivism has shifted the emphasis both to the way in which individuals construct personal meaning given the processes of memory, identity, and the social context in which this occurs. One of the first works to bring our attention to the human relationship to "things" -- i.e., symbolic objects, artifacts, and works of art -- was the 1981 volume The Meaning of Things by Mihalyi Csikszentmihayli and Rochberg-Halton.58 Lois Silverman, following on, has taken for her doctoral dissertation a mass media framework, in which the interpretation of symbolic objects is viewed as a creative, and audience focused process that happens in a context of social and relational meaning of one’s companions. Museums can be a case for study of social construction of meaning particularly through a study of people’s talking with each other.59
In studying visitor pairs Silverman deduces seven frames through which to interpret their interactions based on verbal interactions, like object description, evaluation, relation to special knowledge, relation to personal experience. These seven frames fit into three basic modes, the subjective, the objective, and the combination. Silverman also finds that gender, education and age impacts to an extent on which of these modes one uses. Most importantly, Silverman sees the social context, such as the pair relationship, as the filter for the museum experience; these frame constructs, forms of conversation or talk, impacts on the construction and maintenance of meanings whether in the museum or elsewhere. The museum experience, then, occurs within the social context of meaning of messages and socially constructed through talk with companions and for construction of identity. The result is a complete challenge to the sender-receiver linear one-way communication model transmission of intended meaning or messages. Visitors value reminiscing, associating personal experiences, recognizing their taste, appraising the worth of objects they own, expressing their competence, expressing their identity.60
Silverman concludes that visitors as meaning-makers may be seen as loss of power or as possible democratizing view of museum experience in that visitors and museum practitioners all learn more about way "things" have meaning in society. She sees that there is a challenge to museum practitioners to reflect the "updated" view of a more interactive, democratic, responsive visitor/museum relationship in all their endeavors. This is not simply a matter of listening to visitors to find out if they heard what we wanted them to hear. It is also a matter of listening to visitors, to find out if and why they are visiting museums and/or even listening at all.
By extension, the same process will be occurring on the web and invites a test to examine her notions on museum web experience by users. The importance of Silverman and others’ work, it would seem is in allowing us to realize that there are a number of communication modes in which museums operate for people, sometimes objective, sometimes subjective, and in combinations of the two. While we may more often choose one mode over another it is also probable that this is in flux according to the moment and social circumstance. The importance of user participation in the museum meaning making, a la Silverman and others, though, has not escaped web makers like Peter Samis, of San Francisco’s Museum of Fine Art, who at the MDA 1995 conference Information: The Hidden Resource, Museums and the Internet claimed:
Maria Piacente in her 1996 research also identified several web sites working with the premise of creating dialogue and multiple voices on the web sites. So the Exploratorium’s website "Remembering Nagasaki", from 1995, with images from a Japanese photographer that documented the horrors of nuclear bombing; with "Atomic Memories" the web audience was invited into the discussion, opening a forum of debate concerning the act of bombing Hiroshima and Nagasaki. I agree with Piacente that this web experience not only acknowledges different points of view but looks to the nature of memory and its power not only in museums but on the web.
Still, there are many questions left as to how museums can create constructivist learning opportunities that value the aspects of personal meaning making and how to translate this to the web effectively. We may also need to examine the degree to which constructivism fits the museum model. Clearly at this point the new discourse around museology aligns quite well with constructivist interpretations of the museum and museum experience and possibly for the web.
[ RETURN to OUTLINE ]
Problematique #3. Whose Museum Is it?
Corporatism or Cooperation
At the same time as there have been pressures to open up the museums to multiple voices, interpretations and negotiate meanings in political and personal terms terms, there has been an equal if not stronger tendency to corporatism in the museum and these ideas too have influenced the phenomenon of museums on the web. David Bearman has cautioned that museums on the web are not mere distribution systems for collections nor are they cybermarkets but they should be based on the values of museums especially that of interaction and the potential of this technology. There needs to be careful study of the interrelations of economic structures and museums, that I do not think is carried out often enough.
The web has come to the museum at a time of funding cutbacks and pressures to move to revenue generation, creative fundraising and new management – i.e., retrenchment and downsized – structures. I have previously referred to the process of "the global commercialization of museums and question the loss of our museological souls"63 and the impact on our career paths.
So when Stephanie James, a student in our Museum Studies Program conducted a survey on why museums were going to the web in the Spring of 1997, she found the importance of promotion in the rationale for museum web pages based on returns from 40 people, and 33 museums:
Purpose of Museum Web Page
73.5% also mentioned that the purpose of the site had evolved in the form of design and content and that there had been a shift away from marketing and promotional work to:
|Greater Access to Collections
||Greater Educational Focus
||Increasing Staff Input
|Increasing Public Input
She also found that 42% could not identify any theory behind their site, although others identified marketing, communication, education, information, design, entertainment and common sense.65 54% had no testing of visitors to the site, 14% examined hits, and 9% in-depth survey. Most web endeavours originated in technical services or marketing departments.66
This pattern may not be so unusual; as Bearman has reminded us that technology tends to be introduced in traditional ways, such as in treating the web as a brochure. Nor would it seem are museums much different than any other group on the web. As David Bearman holds out the vision of the RICH future (Repository of the International Culture of Humanity), he also suggests the model of a consortium of museums where the collaboration of numbers of institutions enhances the perspectives of single institutions, where the pool of resources of members can be strategically directed to a usable and desirable product for the public, that can handle the problems of legal challenges of info design and delivery.
"One of the clearest lessons of the Internet and GII is that structural changes in museum economies and institutional mandates must be anticipated and planned if the opportunity it presents is to be truly transformative. Otherwise the Internet will prove to be just another publication vehicle and the museum community may see little benefit from it -- indeed it could be a threat as other purveyors of cultural heritage squeeze the museums out of the position of being primary providers of cultural experience.67
Essentially Bearman is asking how can museums in a digitized world use their social position and content to find ways to pay for a future that is universally accessible? The possibility of Web commerce holds out some hope for many museums. and has been engaged by many. As a client of the Shopping Channel and of web software and book buying, I can see the possibilities of museum web commerce but the discussion of our non-profit limits should also be engaged.
Further, there are also a number of new management models affecting the approach to museum operations. Whether we think in terms of organizational learning or cooperative work models and wait for the transformation of the museum working space, we can see the importance of engaging the whole museum in the planning, design and evaluation of the museum web presence. Furthermore making new technology, according to the literature of Human-Computer Interface, is complex requiring teamwork: and a multidisciplinary process.68
The Internet and the Intranet are possible web worlds awaiting the transformed management of museums. What does the movement of Computer Supported Cooperative Work (CSCW) have to contribute to our web working methods in museums? Or how many museums web sites show a directory to staff or begin to make transparent the work that they do and in what manner so as to involve the public via the web? Further, are we even using the web to build cooperative work models using the software equivalents of Lotus Notes or CSILE?69 These are issues that museums are only now entering and that will absorb much work in mission discussion, user study and development in upcoming years.
[ RETURN to OUTLINE ]
IV. Meta Data Meets the Meta Visitor: Musing on the Public Experience of the Museums and the Web
Given all of the ideas about museums and the museum experience indicated above, what are the implications for thought about museums on the web? Our Webmeisters have already raised questions about the user and the importance of interactivity and participation in the web experience. Are we yet reflecting the rich discussion of museological discourse identified above? So David Bearman asks:
|"How can we make the facts of these objects sing to the virtual visitor? How can we enable them to have an experience? The first requirement for museums is to recognise that the networked environment is interactive, and therefore can be user driven. It enables us to respond to the visitor rather than pump information at him . If used to its best purposes, the networked environment enables a user to construct an experience with personal meaning."70
For Bearman and others, the answers lie in the provision of structural metadata that allows for user understanding of context, method and structure, bits in context, to allowing for usability. So thesauri, schema, organic constructs, the terms of information architecture, have to be blended into the user interface in a manner most useful to the user and woven into the museological discourse. There is more to this though: the rich texture of discussion about the museums’ publics that has been building for over 100 years. Lynn Dierking will in this session, and her parallel paper, look to the work of Museum Visitor Studies and its implications for the web. Suffice it for me to say that we must not assume the arrogance of the modern that only we have invented thinking about users either in museums or on the web. Perhaps, as important, work in the visitor experience of museums may also have relevance for the field of Human-Computer Interface research.
[ RETURN to OUTLINE ]
Gertrude Nappa Sam Suede Luva Kid Edelman Black Metallic LeatherSilver 1w0qa5x & The Web
What do we know about museums and the Web? Gordon has argued that, "The embryonic nature of the Web and its rapidly changing state make evaluation of audience meaningless."71 Several recent studies – those by Downs and Quinn -- have shown that little assessment of Web site effects is being done by museums. Findings from the later, the British Columbia Museums Association study to measure the effectiveness of the involvement of cultural heritage institutions with the Internet, showed that very few institutions could assess Internet results in any objective manner. Although museums were highly interested in the Web, their ability and "Professionalism" was "abysmal."72
Nevertheless, there is some research occurring on museums' efforts on the web and interest is growing. They are of three general types, borrowing terminology from the Museums Visitor Research field: 1. Front-End; 2. Formative; 3. Summative. In this section of this paper, I will look at the valuations of the web and compare it to models from the field of Human-computer Interface or Usability study in order to explore newer forms of usability study for museum web study.
First though let us take a brief look at the field of Human-Computer Interface or User-Centered Design and see what is of significance to putting museums on the web. This is not so unusual a notion. There has been an historical connection between museum presentation and ergonomics and design, the forerunner of Human-Computer Interface, that originated in the study of work practices earlier in century eventually named Human Factors- a branch of applied psychology devoted to guiding and enhancing the design of artifacts. Although Human Factors gained attention during W.W.II when it was applied to complex new weaponry it has been expanded to apply to everything from industrial design to exhibit and multi-media exhibits in museums.
In the field ofSAS Bravo BlackBrown SAS Bravo Human-Computer Interface (HCI), also called User Interface, we see the extension of Human Factors to the computer world. The premise is that computer systems do not work on their own but are interactive, engaging human users in computer-assisted tasks. It is this play between the user and the machine that is the most significant factor in the success or failure of technology, new or old.
|"Narrowly defined, this interface comprises the input and output devices and the software that services them; broadly defined, the interface includes everything that shapes users’ experiences with computers, including documentation, training, and human support."73
Such user testing is now a large part of software development. For example in a 1992 survey by Myers and Rosson that brought 74 responses from software developers, 48% of the application code and a comparable development time was assigned to the interface.74 Still, according to many authors, this is an area of evaluation that is still poorly understood and user study is seldom taken up either because it is seen as too expensive or as beyond the abilities. So Wigand et.al. have argued that most web sites are designed "from the organization’s rather than the user’s perspectives" and refer to one report that noted that 90% of companies studied constructed their sites without any consultation with users.75
Nevertheless the academics and texts recommend user study. Now, similar arguments are occurring for user study and involvement for web product and can be found on the web in the personalities of Jakob Nielsen and David Spiegel and whose guidelines are being consulted by some museum web designers.76 Surely the statistics would be similar for museum web pages.
According to Ron Baeker et. al. there is a core list of issues for user-centred design that we can take to heart in our approach to museums on the web:
- Make it easy to determine what actions are possible at any moment (make use of constraints).
- Make things visible, including the conceptual model of the system, the alternative actions, and results of actions.
- Make it easy to evaluate the current state of the system.
- Follow natural mappings between intentions and required actions, actions and required effects, and between the information. that is visible and the interpretation of the
- Or in other words,
1) make sure the user can figure out what to do
2) the user can tell what is going on.77
There are many authors in HCI who have provided lists of design principles for designing interactives. As in Museum Visitor Studies, there are two methodological schools, the quantitative and qualitative. For some authors evaluations must be based on behavioral analysis and empirical measures. For example, Jakob Nielsen’s 1989, "discount usability engineering" work promoted a rigorous quantitative approach with a test of small subsets of the system with relatively few users who think aloud as they work. Nielsen admits that in spite of a process of iterative design, the interface was not perfect. On reflection, developers had used intuition to generate initial user requirements that could indeed be in error. We also see that there is a spectrum of philosophies behind many of these user study enthusiasts from those like Nielsen who evolve from the education and computer school to David Siegel who evolves from the entertainment and design fields. Both though urge extensive user study and involvement.
Here for example are Jakob Nielsen’s eleven steps involving user-centered study in developing new technologies:
|I. Know the User
||5. Participatory Design
a. Individual user characteristics
|6. Coordinated design of the total interface
b. The user’s current and desired tasks
|7. Apply guidelines and heuristic analysis
c. Functional analysis
SAS Bravo Bravo BlackBrown SAS d. The evolution of the user and the job
|9. Empirical testing
|2. Competitive analysis
||10. Iterative design
|3. Setting usability goals
a. Capture design rationale
a. Financial impact analysis
|11. Collect feedback from field use78
|4. Parallel design
According to Gould, Boies and Lewis,79 on usability, the user-focus is central to the entire process of production.
Early Focus on Users
Designers should have direct contact with intended or actual users- via interviews, observations, surveys, and participatory design. The aim is to understand users’ cognitive, behavioural, attendant, anthropometric characteristics -- and the characteristics of the jobs they will be doing.Bravo BlackBrown SAS Bravo SAS
Early-- and Continual -- User Testing
The only presently feasible approach to successful design is an empirical one, requiring objective and measurement of user behaviour, careful articulation of feedback, insightful solutions to ensuing problems, and strong motivation to make design changes.
A system under development must be modified based upon results of behavioural tests of functions, user interface, help systems, documentation, training approach. The process of implementation, testing, feedback, evaluation, and changes must be repeated to iteratively improve the system.
All aspects of usability (e.g., user interface, help systems, a plan, documentation) should evoke in rather than defined s and should be under one management.
Testable behavioural target goals
What would happen if we took a similar approach to the development of web presence of museums?
Given this brief excursion into the world of Human-Computer Interface and Usability Study, what are museums doing in the nature of web page evaluation and what could be available to us.
[ RETURN to OUTLINE ]
Museum Web Evaluation Types
I will use the three types of evaluation used in educational evaluation.
1. Summative- Post-mortem Evaluation
One of the first sorts of museum web evaluation came from web-based search engines like Lycos that take criteria used to evaluate all web sites and extend to the museum web page. Sometimes these are called competitive analyses. For example the ROM is in the top 5 % as rated by Lycos. Another example of this is Musee which grades museum sites from 1 to 5 on the criteria of 1)Travel Visitor Information, 2) Education Materials, 3) Research, 4) Fun/Entertainment, 5) Shopping and 6) Visual Content. /www.musee-online.org/reviews.htm>. I am not sure at this point to what degree such intermediary evaluations affect the museums web page design and re-design. The winning of a high rating is often advertised at a site’s home page but do museums actually aim to be highly rated on them. Operators of these services do indicate that webmasters do ask for advice on how to get a higher rating in their pages.80 Museums & The Web’s ‘Best of the Web’ contest provides us with a peer review method of evaluation in which web pages are divided by type and voted on the web about a month before the conference is held each year. www.archimuse.com/mw98/ frame_best.html>.
The most frequent type of summative evaluation is of one site by its owner. Hits to sites are mentioned as one measure although it is not very accurate or necessarily a record of any meaning beyond a very general sense of use. Use comment pages and FAQs are the most frequent methods for feedback from audience. Again, to what degree this information is used in reworking web pages remains unknown. Some institutional summative study is also occurring in terms of hits and navigational patterns of visitors to the web. The user log analyses, with the right expertise, can reveal some things about your web site and its effectiveness. Software available for analysis records the length of visit to the site and page, gives demographic information like location of visitor.81 This is the type of study that James Jensen has performed for the National Museum of Science and Technology in Ottawa. Some sites like the Montreal’s Museum of Fine Art, and Ottawa/Hull’s Canadian Museum of Civilization have also created a questionnaire that is filled out as part of the web visit that gives the basics of information on demographics and perhaps even a hint of motivational information. These methods resemble many visitor studies in museums though such studies are biased to the web visitor who comes to the site and not all potential web site visitors. They require a higher level of technical knowledge of web site construction and conceptual linkage.
Some museums have actually engaged in a visitor study of their web site. The National Museum of American History held a meeting with the Teacher Advisory Committee to discuss educational activities on the web as part of an informal summative evaluation. Soft Kid Tan Ralph LeatherDeep Midnight Soft SuedeBlack Black Saddle Lauren Suede LeatherDark Soft Kid Aminah Kid Kid Super SuedeSaffron Super Super LeatherEggplant LAUREN SuedeLight Taupe ad6qUwFnw> At present, Debra Luneau of the ROM has evolved a questionnaire that she is giving to selected ROM library users who are cued to visit the web site terminals in the Library and make comment. Her work is very much based on the criteria of Jakob Nielsen one of the leading web page evaluators.
There have also been a number of academic or independent comparative studies of web pages that are based on what the HC Interface folks called heuristic study. To put this in plain language, the researcher creates categories for measurement of effectiveness based on a personal study of existing criteria, of one’s own experience of the phenomenon and the existing norms of design standards.82
One example of such a study is that of Maria Piacente in her Masters paper for Museum Studies at the University of Toronto who identified three types of museum web pages and examined them. The first was the "Electronic Brochure", essentially an advertising sheet format like the brochure or handout used at sites or to get visitors to come to sites. The second was "The Museum in the Virtual World" whereby the actual museum was projected onto the web by means of maps, floorplans, images, online collections or exhibits, both real and virtual. Here, the real-life museum is recreated83 online. Maria Piacente found this to be the case particularly with some art galleries and used the example of the Philadelphia Museum of Arts Cezanne pages (http://libertynet.org/). There are several variations of this type. Sometimes the world-wide web is used within an exhibition as in the AGO’s "Oh Canada Project". Some museums are actually using the web site as an archive for former exhibits, that extends the life of special exhibits and gives rich material to the web presence; Piacente's example is the Peabody Museum of Archaeology and Ethnology. < http://www.peabody.harvard.edu/maria/default.html> Some museums have designed their web presence as an extended collections access. Examples of the collections based web are the Hooper Virtual Palaeontology Museum, the University of California’s Museum of Palaeontology, the Canadian Warplane Heritage Museum < http://www.warplane.com/fighters.html/>, and Regina’s the Mackenzie Art Gallery material on Sask. artists < http://www.uregina.ca/~macken/>. And some have gone to the establishment of searchable databases for collections information access, i.e Schools, National Museum of American Art. Some of these sites represent deep sites, in the sense of hierarchical connections and hyperlinks.
Maria Piacente identifies a third of approach to museum web pages, "The True Interactives". Here, the pages may have some relation to real museum but they also add or reinvent the museum and even invite the audience to do so. Often these sites differentiate the web from museum by its name, especially those of the science centres. So the Exploratorium has the "Learning Studio". A few art galleries have also taken this approach such as the Dia Centre for the Arts http://www.diacenter.org/. Examples of these engaging sites are the Saskatchewan Science Centre < http:// www.sciencecentre. sk.ca/>; the Natural History Museum, London < http:// www.nhm.ac.uk/>; Minneapolis Institute of Arts www.artsmia.org/>; the Children’s Museum of Indianapolis www.a1.com/children/home.html>; and the Franklin Institute Science Museum’s Science Learning Network sln.fi.edu/>. Although Piacente conducted her analysis nearly two years ago, many of her sites remain the leading competitors in such events as the Best of Museums on the Web.
Piacente’s typology of web pages is very useful particularly because it parallels the typologies we have developed for exhibition analyses.84 It also admits that there are multiple purposes not only to museums but to their web pages. Perhaps we should be developing some sense of the complex of museum web page types and that there may be differing purposes and evaluation criteria for them.
Some other summative comparative studies have been conducted by Katherine Futers and by Dr. David Barr. Futers prepared a study for the Museum Documentation Association in 1996 by surfing for several hours each day to museum web pages and making notes about her experience. Futers, herself a person new to the Internet, concentrated on what was available and how accessible the pages were. She concluded, "There are museums out there that are leading the way in providing strong, useful, productive and entertaining web pages", noting in particular the work of the Exploratorium.85
Dr. David Barr’s study, although an heuristic one, tried to place the examination within the critical needs of adult and life-long learning. Barr distilled six measurements key to adult learning resources: information content, communication properties, accessibility interactivity, feedback/community and visitor services and three qualities of websites in general: structure, complexity and dynamics. Each of these were then sub-divided until he was testing for 70 indicators, to include such items as total file size, links to related sites, cyberambiance, image to page ration and pattern of hyperlinks. He used the program Web Analyzer 2.0 from In Context Software to quantify the qualitative variables and then performed the analysis.BlackThunder Vince Vince Camuto Vince Camuto Waldan BlackThunder Waldan Camuto 7I4HTqW Barr found that four of the most effective sites were highly valued for adult learning; only one, however, was a museum site and the other three commercial adult learning sites.
|"For museums, such sites may represent one of the few effective vehicles for extending the museum’s lifelong learning mandate beyond the museum walls and beyond the duration of the individual museum visit."SAS BlackBrown SAS Bravo Bravo 87
Two further types of visitor evaluation occur during the planning and implementation of a projects, 2. Front-end and 3. Formative. There are a few examples of each in the museum web world but not very many as yet.
[ RETURN to OUTLINE ]
2. Front-end- User Needs
Several sites have attempted to use the methods of focus groups to look at web possibilities. The McMichael Canadian Art Galleries Artifice project, originally planned as a CD-ROM and now also for the web, began with focus groups with teachers and students from the North York School Board. The Florida Natural History Museum Science has also used front-end evaluation in developing its web pages. No doubt there are some other examples; still for the most part museums have not yet used the front-end methods of user study to any great extent.
There have been other projects that have more generally attempted to get at the underlying premises of user needs for information from museums. At the National Museums of Scotland Catechism project was a survey of visitor questions by Helen McCorry and Ian Morrison. Catchism was created to test the assumption that "the present approach to museum information is too highly structured and too theoretical and therefore discourages access instead of improving it." Many of the information databases architectures, according to McCorry have been created based on assumptions based on 1970s and 1980s systems, including manual systems, and their demands, and bibliographic databases where terminology and authority references are controlled and accepted.88 These approaches do not readily think about the user in ways appropriate to this new media and usage.
Another project that must be mentioned is that of the Getty’s AHIP’s Points of View Project, from January of 1995 to Spring of 1997, under the direction of Jane Sledge. It focussed on questions of user’s demands for data as a premise for understanding the users use and need for museum information. Eventually through three focus groups of museum staff and audiences discussions were held regarding the nature of information uses. Then, a number of humanities scholars were studied by Dr. Marcia Bates for their searching behaviours of Getty databases89 with some surprising results.
Jane Sledge has suggested that:
To what degree, then, should the web presence contain a mediated assistance with human intervention versus information architecture? One suggestion has been that information design be set up on a "novice-to expert continuum" depending on existing expertise and interest. This approach has been criticized as too hierarchical and creates a mistaken notion that scholars require "different entry points" than other users.91 We can see that the novice-expert approach is contrary to constructivist learning ideas. Further there are issues of design and presentation to be investigated.92
[ RETURN to OUTLINE ]
3. Formative- User Test While In Development
We have now a few examples. Over the last months, the McMichael, with the assistance of Digital Renaissance’s usability architect Diane Howie, has brought the technique of "discount usability engineering" promoted by Jakob Nielsen into use through the development phase of the prototype. Here five interviews were conducted in a one-to-one relationship engaging the user with the product, Artifice, and recording in detail the responses and behaviours.
In another example, the Indianapolis Children’s Museum has taken a slightly different and more participatory procedure for developing the web site. Inviting web visitors to contribute their ideas to the web making.
Some museums now are cross-over thinking from exhibits or multi-media to web development, and often have used visitor research93 and evaluation in these settings and are anxious to bring it to the web work. At the Australian War Memorial in Canberra, Linda Ferguson, the ‘Audience Advocate and Evaluator’, has been conducting a formative evaluation on the new website for logic and user-friendliness.The Natural History Museum of Los Angeles County engaged noted museum evaluator Randy Korn to evaluate both a kiosk and the web page for the traveling exhibit "Cats! Wild to Mild." These efforts no doubt reveal the extent to which the web efforts are integrated into the other functions, especially that of exhibiting and visitor study, in the museum.
Again, the field of Human-Computer Interface gives us a sense of all the possibilities of user-study that are open to us. Gould lists the many methods available to us in user-study:
Methods to Carry Out Early Focus on Users
Talk with users
Visit customer locations
Observe users working
Videotape users working
Learn about the Work organization
Try it yourself
Expert on design team
Surveys & Questionnaires
Methods to Carry Out Early -- and Continual-- User Testing
Printed or video scenarios
Early user manuals
Hallway and Storefront methodology
Computer bulletin boards...and conferencing
Methods to Carry Out Iterative Design
System development work organization
Methods to Carry Out Integrated Design
Consider all aspects of usability in the initial design.
One person or group has responsibility for all aspects.94
Clearly museums have only begun to explore the possibilities of user-study for museum web page development. Of course from the preceding argument you will see that I would also like such evaluation to occur within the museological frame of reference. So if we were able to take the premises of New Museology and the newer learning paradigms for museums and if we were able to take the work of visitor research and extend it to the web, we could clarify the premises of web museum evaluation and profitably meld them to the User-centered Design approaches appropriately.
[ RETURN to OUTLINE ]
V. Moving to Participatory or Collaborative Approaches to the Design of Web: Implications for Museums and the Web
In this section I will attempt to meld the areas of museology, Human-Computer Interface and web page evaluation around an emerging type of museological web page experience.
One of the most interesting developments mentioned above is that of the evolution of concepts of participatory design. Work in the last two decades in Scandinavia and in some other locations in participatory design for information technology offers some important opportunities that parallel much of the thinking of the New Museology or what I like to call Participatory Museology. Given that some of the thinking of the New Museology (as discussed above) has also occured in Scandinavia, there is an interesting parallel in thought. Such approaches in user-centred design try to move from the lab to the workplace and to adjust to the variety of users involved moving to User-involved design. This approach rejects the catchall imprecision of user study concepts as useless and fundamentally inefficient as, despite user study through all the steps of development, the final test of a project often shows that the results do not fit the users and so begins the remedial action of design.
Liam Bannon has argued that we switch from the concepts of users to actors to acknowledge the active and agency role of the participants or clients should play in the process of the design. Bannon prefers the word actors to humanize the study, and moves from a concept of human as passive to participatory.95 Human Factors in his view has often reduced the human element, the user, to "another system component" that underplays issues of individual motivation, membership in a community of workers and the complexities of human action. "People are more than the sum of parts; be they information-processing subsystems or physiological systems, they have a set of values, goals and beliefs about life and work." For Bannon, they are "active agents" in the design itself. Bannon trained at the University of Toronto where some of the ideas of Scandinavian participatory design and evaluation have made some inroads.
In Scandinavia, 1970s legislation brought workers into the development of computers design on the principle of codetermination. By the 1980s projects were developed to focus on design of new kinds of computer support using skill and product quality to bring in the users’ perspective, thus blending issues of quality management with participatory design. In this practice, emphasis moved from system descriptions to mock-ups and game-like design sessions, to invoke fantasy and imagined futures and involved worker participants in the process.
Bödker, Grönboek and Kyng (1993) are participatory design researchers from Aarhus University in Denmark who have developed applications to enhance workplace skills through increased flexibility , interplay between technology and the organization of work in a manner that requires the full cooperation users and developers. So the Project Manager becomes Project Facilitator, and cooperative design the goal:
|"Cooperative design which in our perspective means empowering users to fuller participation and cooperation, changes the rules of the game...The cooperative design approach begins by creating and environments in which users and designers can actively consider the future use situation. It is a process where users and designers don’t have to wait until the final act to know if the application will fit the practice of the users."96
Similarly work in computer-supported collaborative learning (CSCL) and collaborative work (CSCW) offer some important conceptual shifts extending our picture of the user experience beyond the solitary viewer or web user to address more social forms of involvement in the museum experience. These working methods have profound implications for the way we develop new media in our museums that will require re-thinking management structures and philosophies.
[ RETURN to OUTLINE ]
Moving To Participatory/Collaborative Museum Web Work:
We need to point out that museums are already working in ways implied by participatory design and in the spirit of cooperative museology as represented in the New Museology. The examples I would point to are the L.A. Culture Net, a three year project that runs to June 1999. home.lacn.org/LACN/> Here, the point is " people understanding and participating in their community through culture" and through the Internet. So they can hold a "Webraising" where the webmasters and others act as facilitators to the web communities, virtual and other. Here the style of interaction is participatory, and collaborative among cultural and educational organizations and the private sector. Another example of community approach to web making is that of work at the AGO, first with the "Oh! Canada Project" and now with the web cooperative working method of developing the 60’s project.
Finally, I must admit that I am enamoured of another experiment with the museum as a phenomenon, very much in the spirit of the "New Museology" and the new person-centred museum discussed above, that has been evolving for at least four years. Two artists, Michele Kolnicker and Michael Kiselinger, created a virtual museum, the Heimat Museum, for Castle Toller in Upper Austria. The intention was to gather various ideas and represenations of "heimat", German for connection to home or homeland, through the WWW. Topics were childhood, community, food, language, living-spaces, surrounding, things, crane-school. Then material would be put on display in multi-media installations and sculptures of all the participants on the occastion of a Regional festival. The project is still going on now, three years after, as each day someone from around the world adds another story, memory and often an image. fgidec1.tuwien.ac.at/1002situations/> This museological exercise symbolizes very well the idea of museums as places of meaning-making and memory as well as that of community, now defined over the Web.
The possibilities of the web expression of some of the premises of New Museology and of the newer ideas of museums and meaning-making it seems to me are endless and exciting. In the next few years, I believe we will engage the groupware technologies and software to explore Computer-Supported Cooperative Museology and in ways that respond both to the New Museology and the new politics of community and identity in museums.
[ RETURN to OUTLINE ]
VI. The Muses On the Web- Conclusion
With this discussion we have covered a lot of ground in an attempt to tie together several areas of study necessary to conceptualizing and evaluating museums and the web. From the questions of museology, to the understandings of user /usability study, the context for looking and evaluating museums on the web has been explored. What this discussion does invite is the possibility of much more user study in the development phases of museums web activities. It also raises the question of how museums may in fact reconceptualize their relationships with people, groups, and communities using the technology of the Internet and in the spirit of the current discourse in museology. In a sense we have come full circle from the participatory notions of museology to the participatory ideas of user involvement in creating new technologies like the web and both utilizing the study of users and publics to greater benefit of all.
By now, I also hope that you are convinced that the discussions of museology on the web should occur both within the larger discussion of museums and museology as well as that of Human/Computer Interface study in a manner informed by the nature of museums in all their complexity, including theirBravo Bravo SAS SAS BlackBrown "Bodies and Souls" as evoked by ecomuseologist Ewa Bergdahl.
[ RETURN to OUTLINE ]
1. Heather Menzies, Whose Brave New World? (Toronto: Between The Lines, 1996), p 19. [ RETURN to TEXT ]
SAS Strippy Patent Black Patent Black PatentBone SAS PatentBone SAS Strippy 6wqtXf has been influenced by the work of Masters of Museum Studies students’ work; that of Stephenie James and Maria Piacente. The treatment of issues of new technologies and user study follows from my exposure to the work of various members of University of Toronto’s Knowledge Media Design Institute including Ron Baecker and Joan Cherry. In addition, Dr. David Barr, Debra Luneau of the ROM, and Richard Gerarrd assisted in the ideas and the production of the paper. [ RETURN to TEXT ]
3. Max Anderson, "Introduction", The Wired Museum, Washington, AAM, 1997, p. 15. [ RETURN to TEXT ]
4. Doron Swade, "The Digital Superhighway and the Curator," Museum Collections and the Information Superhighway Conference, Science Museum 10 May 1995. < http://www.nmsi.ac.uk/researchers/confabs.html#swade> [ RETURN to TEXT ]
5. Tom Wright, " Museum Collections and the Information Superhighway Conference, Science Museum 10 May 1995. < http://www.nmsi.ac.uk/researchers/confabs.html#wright> [ RETURN to TEXT ]
6. Tomislav Sola, "Museum Professionals- the endangered species," in P. Boylan (ed.), Museums 2000. Politics, People, Professionals and Profit (London: Routledge, 1992), pp. 101-113. For more of this discussion, see Peter Van Mensch, "What Museums, What Success? Some museums Are More Equal Than Others," Conference ‘Art Museums and Price of Success," Boekmanstichting, Amsterdam, December 1992, pp. 1-5, www.xs4all.nl/~rwa/boekman.htm >. [ RETURN to TEXT ]
7. Howard BesserMerrell Merrell BlackGunsmoke Moc Ice BlackGunsmoke Jungle Merrell Jungle Moc Jungle Moc Ice rXFqrwH, "The Changing Role of Photographic Collections with the Advent of Digitization," The Wired Museum, Washington, AAM, 1997, p.120 [ RETURN to TEXT ]
8. Max Anderson, "Introduction", The Wired Museum, Washington, AAM, 1997, p. 19. [ RETURN to TEXT ]
9. Ibid., p. 19. [ RETURN to TEXT ]
10. Ibid., p. 32. [ RETURN to TEXT ]
11. Besser, The Wired Museum, p. 120. [ RETURN to TEXT ]
12. Besser, The Wired Museum, p. 121. [ RETURN to TEXT ]
13. Doug Worts and Kris Morrisey, "Technology, Communication and Public Programming: Going Where Museums Have Rarely Gone," AAM Sourcebook p. 175-179. For more Worts, see ‘Extending the Frame: Forging a New Partnership with the Public", New Research in Museum Studies, vol.5 (1994) and "Museums’ Technology Frontier: How Treacherous is it?", in press for 1998, Washington,D.C.: AAM. [ RETURN to TEXT ]
14. Ibid., p. 177. [ RETURN to TEXT ]
15. Exceptions are Suzette Worden, "Thinking Critically About Virtual Museums," Andrea Witcomb, "The End of the Mausoleum: Museums in the Age of Electronic Communication," and Peter Walsh, "The Web and Unassailable Voice," in Museums and the Web 97: Selected Papers , Pittsburgh: Archives and Museum Informatics, 1997. [ RETURN to TEXT ]
16. See Lynne Teather, "Museum Studies: Reflecting on Reflective Practice," Journal of Museum Management and Curatorship 10 (1991), pp. 403-417 and Peter Van Mensch, Ph.D. Thesis, University of Zagreb,1992 at http://www.xs4all.nl/~rwa/boekman.htm. [ RETURN to TEXT ]
17. Francis Henry Taylor, Babel’s Tower: The Dilemma of the Modern Museum (New York, 1945), p. 39. [ RETURN to TEXT ]
18. Stephen Weil, Rethinking the Museum, (Washington: Smithsonian Institution Press, 1990), and Julia Harrison, "Ideas of Museums in the 1990s", Museum Management and Curatorship 13 (1993), 160. [ RETURN to TEXT ]
19. Presentation to Museum Studies Program, University of Toronto, Fall, 1996. [ RETURN to TEXT ]
20. Benjamin Ives Gilman, Museum Ideals of Purpose and Method, (Cambridge, 1923) and George Brown Good, "Museum-History and Museums of History,. [ RETURN to TEXT ]
21. Duncan Cameron, "A Viewpoint: The Museum as a Communication System and Implications for Museum Education," Curator XI, ( March 1968), pp. 33-40, and E.I. Knetz and A.G. Wright, "The Museum as a Communication System: An Assessment of Cameron’s Viewpoint" Curator XIII, (3 December 1970), pp. 204-212. 1Brown Pelotas Camper Black 1Medium Ariel 1 16002 Brown wqOIdBO
22. George F. MacDonald and Stephen Alsford, "The Museum as Information Utility", Museum Management and Curatorship, 10 ( 1991), pp. 306-7. [ RETURN to TEXT ]
23. Deirdre Stam, "The Informed Muse: The Implications of "New Museology’ for Museum Practice," Museum Management and Curatorship 12(1993), pp. 267-283. [ RETURN to TEXT ]
24. Ibid., p. 271. [ RETURN to TEXT ]
25. Ibid., p. 267. [ RETURN to TEXT ]
26. Ibid., p. 280. [ RETURN to TEXT ]
27. George MacDonald and Stephen Alsford, George Macondald and Stephen Alsford, Museum for the Global Village (Ottawa: Canadian Museum of Civlization, 1989). [ RETURN to TEXT ]
28. David Siegel is at www.cea.edu/> and Jakob Nielsen is at www.useit.com>. [ RETURN to TEXT ]
29. Lynne Teather, "Reflecting on Museum Practice", Journal of Museum Management and Curatorship 10 (1991), pp. 403-417. [ RETURN to TEXT ]
30. Peter Vergo, (ed.) The New Museology, (London: Reaktion Books, 1990) [ RETURN to TEXT ]
31. Julia Harrison, "Ideas of Museums in the 1990's". Journal of Museum Management and Curatorship 13(2)(1994), p. 162. [ RETURN to TEXT ]
32. To pursue the history of museum thinking, see Eileen Hooper-Greenhill, The Shape of Knowledge (London: Routledge, 1992); Lynne Teather, Museology and Its Traditions: The British museum Experience, 1845-1945, Ph.D. Dissertation, University of Leicester, 1984; Lisa Roberts, From Knowledge to Narrative: Educators and the Changing Museum, Washington: Smithsonian Institution Press, 1997, and for a European perspective, see Peter Van Mensch’s web site where he has mounted chapters of his doctoral thesis Towards a Methodology of Museology Ph.D. Thesis, University of Zagreb, 1992, < http://www.xs4all.nl/~rwa/boek01.htm> [ RETURN to TEXT ]
33. Paul Marshall Rea, The Museum and the Community: A Study of Social Laws and Consequences, ( Lancaster, Pa.: Science Press) 1932. [ RETURN to TEXT ]
34. T.R. Adam, The Civic Value of Museums (New York:: American Association for Adult Education, 1937), p.26. [ RETURN to TEXT ]
35. Duncan Cameron, "Museums, Temple or Forum," Curator 14(1) (March 1971), p. 23. [ RETURN to TEXT ]
36. Bloom et al, Museums for a New Century, (Washington: AAM, 1984) and Excellence and Equity (Washington: AAM, 1992.) [ RETURN to TEXT ]
37. Stevenson, Sheila (1987). "Balancing the Scales: Old Views and a New Muse", MUSE, 5,1: 30-33. [ RETURN to TEXT ]
38. For more discussion on ecomuseums, see René Rivard, Opening Up The Museum or Towards a New Museology: Ecomuseums and "Open Museums" (Quebec City, 1984) and Andrea Hauenschild, Claims and Reality of the New Museology: Case Studies in Canada, the United States and Mexico, Ph.D. Dissertation, University of Hamburg, 1988. SAS Bravo BlackBrown SAS Bravo [ RETURN to TEXT ]
39. Ewa Bergdahl, "Summary," Paper in Nordisk useologi 2 (1996), pp. 71-86. www.umu.se/nordic.museology/NM/962Summaries.html> [ RETURN to TEXT ]
40. Carol Duncan and Allan Wallach, "The Museum of Modern Art as Late Capitalist Ritual: an Iconographic Analysis," Marxist Perspectives, (Winter 1978): 28- 51 and Duncan, Carol and Wallach, Alan, "The Universal Survey Museum", Art History 4: 1980, 448-469 and Robert Lumley (ed.), The Museum Time Machine: Putting Culture on Display, (London: Routledge, 1988). [ RETURN to TEXT ]
41. See for example George Stocking, (ed.) Object and Other: Essays on Museums and Material Cutlure,, (Londong: University of Wisconsin,Press, 1984); Michael Ames, Museums, the Public and Anthropology, (New Delhi: Concept Press, and Vancouver: Univrsity of British Columbia Press,1986) and Cannibal Tours and Glass Boxes: The Anthropology of Museums ( Vancouver: UBC Press, 1992) and Assembly of First Nations and the Canadian Museums Association (1992) Turning the Page. Forging New Partnerships Between Museums and First Peoples, Ottawa: CMA, 1992. [ RETURN to TEXT ]
42. Edwina Taborsky, "The Discursive Object," in Susan Pearce (ed.) New Research in Museum Studies 1 (London: Athlone Press, 1990), p. 52. [ RETURN to TEXT ]
43. Susan Pearce, Objects of Knowledge, (London: Athlone Press, 1992). [ RETURN to TEXT ]
44. Lisa Roberts, From Knowledge to Narrative. Educators and the Changing Museum (Washington, D.C.: Smithsonian Institution, 1997) p. 79. [ RETURN to TEXT ]
45. Peter Walsh, "The Web and the Unassailable Voice," Museums and the Web, 1997, Los Angeles, Pittsburgh: Archives and Museum Informatics, 1997, p. 69. WhiteWhite Anthracite BlackWolf BlackBlack Black Air Nike Grey White White Max Motion x7TIaYq
46. Ibid., p. 75. [ RETURN to TEXT ]
47. "The Hidden Peoples of the Amazon exhibition" at Museum of Mankind in London (1986), "The Spirit Sings: Artistic Traditions of Canda’s First Peoples" in Calgary and Ottawa (1988), "Into the Heart of Africa" in Toronto (1990), and Te Maori and the San Francisco Fine Art Museum. For a discussion of "Hispanic Art in the United States: Thirty Contemporary Painters and Sculptors "(1984-7) in Houston a travelling exhibition in the US triggered controversies Livingston, Jane and Beardsley, John. "The Poetics and Politics of Hispanic Art: A New Perspective", in Ivan Karp and Steven Lavine, ()eds.) Exhibiting Cultures: The Poetics and Politics of Museum Display. Washington: Smithsonian Institution Press, pp. 104-120. [ RETURN to TEXT ]
48. In Canada, Michael Ames, Bruce Trigger and Julia Harrison. Porcelain Gull Green Fastpack Dark BlueDune RedUrban GTX Face The Deep Navy Grey North Beige 1 Fortuna Hedgehog Garnet UFwqZxnAaY
49. Ivan Karp, Christine Kreamer and Steven Lavine, (eds.) , Exhibiting Cultures: Poetics and Politics of Museum Display ( Washington: Smithsonian Institution Press, 1992) and Karp, I., Kreamer, Christine and Lavine, Steven (eds.), Museums and Communities: The Politics of Public Culture (Washington: Smithsonian Institutional Press, 1991). [ RETURN to TEXT ]
50. John Kuo Wei Then, "Creating a Dialogic Museum: The Chinatown History Muaseum Experiment", in (Ivan Karp, Christine Kreamer and Steven Lavine, eds.) Museums and Communities: The Politics of Public Culture. Washington: Smithsonian Institution Press, 1992. pp. 286-326, and Edmund Barry Gaither, "Hey, That’s Mine": Thoughts on Pluralism and American Museums", in Ivan Karp, Christine Kreamer and Steven Lavine, (eds.) Exhibiting Cultures: Poetics and Politics of Museum Display. Washington: Smithsonian Institution Press, 1992, pp. 42-56. [ RETURN to TEXT ]
51. Worts and Morrisey, p. 177. Harmony Mephisto Black WaxyCubaMulti QueenieTan Grain Snzqvw
52. George Hein, "The Constructivist Museum," Paper for Journal for Education in Museums 16 (1995), pp. 21-23, p.1. http://www.gem.org.uk/hein.html For an excellent discussion of Constructivism see the Miami Museum of Science web page . < http://www.miamisci.org/ph/lpintro5e.html > [ RETURN to TEXT ]
53. Jamieson MacKenzie, " Museums and the Web ’97, (Pittsburgh, Pa.: Archives and Museum Informatics, 1997), p.79. [ RETURN to TEXT ]
Millierge Millierge Star Stars StarPure DeGeneres Star Gold Stars White Star ED Millierge Black Ellen Chapastar Gold Millierge OwFOqBgZ, p. 80. [ RETURN ]
55. Marlene Scardemelia and Carl Bereiter, "The Challenge of a Knowledge Society", p.2. For more information about CSILE see csile.oise.utoronto.ca/CSILE_biblio.html > [ RETURN to TEXT ]
56. Ibid., p.2. [ RETURN to TEXT ]
57. Ibid. [ RETURN to TEXT ]
58. Mihalyi Csikszentmihayli and Rochberg-Halton, The Meaning of Things:Domestic Symbols and the Self, (Cambridge: Cambridge University Press, 1981). [ RETURN to TEXT ]
59. Lois Silverman, " Of Us and Other "Things": The Content and Functions of Talk By Adult Visitor Pairs in An Art and A History Museum", Ph.D. Thesis, University of Pennsylvania, 1990. [ RETURN to TEXT ]
60. Ibid., p. 269. [ RETURN to TEXT ]
61. Anne Fahy and Wendy Sudbury (eds.), Information: The Hidden Resource, Museums and the Internet. Proceedings of the Seventh International Conference of the MDA (Cambridge: MDA, 1995), p.25, as cited by Futers, p.1. [ RETURN to TEXT ]
62. Maria Piacente,Surfs Up: Museums and the world Wide Web, MA Research Paper, Museum Studies Program, University of Toronto, 1996, p. 12. [ RETURN to TEXT ]
63. Lynne Teather, Museology and Its Traditions, p. 414. White Nike Black White TintIgloo Dark Free PulseParticle Black White BlackBlack GreyCollege Flyknit Black Particle Beige Navy PhantomWhite Blue College 3 1 TR Crimson Navy Beige BBgqr
64. Stephanie James, "Survey Results," March 5, 1997, p.1-9. www.chass.utoronto.ca/~sjames/museum/survey.htm> [ RETURN to TEXT ]
65. Ibid., p.3. [ RETURN to TEXT ]
66. Ibid., p.5. [ RETURN to TEXT ]
67. David Bearman, "Museum Strategies for Sucess on the Internet," Paper for Museum collections and the Information Superhighway conference, at the Science Museum 10 May, 1995. www.nmsci.ac.uk/researchers/confabs.html#bearman>. [ RETURN to TEXT ]
68. Ron Baecker, et.al., p. 74. [ RETURN to TEXT ]
69. See Proceedings of CSC: ‘97, The Second International Conference on Computer Support for Collaborative Learning, Dec. 10-14, University of Toronto, Dec. 1997. [ RETURN to TEXT ]
70. David Bearman, "Museum Strategice for Success on the Internet," Paper presented at Museum collections and the Information Superhighway Conference, Science Museum, London, (May 10, 1995), p.4. <www.nmsi.ac.uk/infosh/bearman.htm> [ RETURN to TEXT ]
71. Sue Gordon, "Is Anybody Out There?", Hands on: Hypermedia and Interactivity in Museums: Selected papers from the Third International Conference on Hypermedia and Interactivity. Pittsburgh: Archives and Informatics, 1995. [ RETURN to TEXT ]
72. Clifford Quinn, BCMuseums Association WWW, 1996. [ RETURN to TEXT ]
73. Preface, XI,Human-Computer:Readings in Human Computer Interaction:Toward the Year 2000, Written and Edited by Ronald M. Baecker, et al.D, San Francisco, Morgan Kaufman, 1995. [ RETURN to TEXT ]
74. See John Falk and Lynn Dierking, The Museum Experience (Washington, D.C.: Whalesback Press, 1992). [ RETURN to TEXT ]
75. David Kline, "Memo to the Boss: Your Web Site is Useless," /www.hotwired.com/market/96/15/index1a.html> (April 1996), as cited in Rolf T. Wigand et.al, "Electronic commerce and User-Based Design of a Web site: Targeting the Technology Transfer Audience," Journal of Technology Transfer vol.22 (1), pp. 19. Red WhiteSpice Native Dublin Bone Grey Treklite Fitzsimmons Bone Black Bone Brown WhiteJiffy Shoes Dublin Howler Pigeon White Grey Grey x66pqwF
76. Personal Communication with 4 Museum Web page designers from Museum-L listserve. [ RETURN to TEXT ]
77. Ron Baeker et. al., Human-Computer:Readings in Human Computer Interaction:Toward the Year 2000, p. Xi. [ RETURN to TEXT ]
78. Nielsen, as cited in Baeker et.al. p.72. New Balance Black 1165v1 SilverGrey Black xRz0rwzqd
79. Gould, Boies, and Lewis, 1991, as cited in Baecker, et. al., 1995, p. 76. [ RETURN to TEXT ]
80. Personal Communication, March 5, 1998. [ RETURN to TEXT ]SAS SAS Bravo BlackBrown Bravo
81. Patricia Downs in her study of Web usage by museums points to the University of Cambridge www.statslab.cam.ac.uk/~sret1/stats/stats.html> [ RETURN to TEXT ]
Suede SuedeDusty Minnetonka Brown Double Boot Fringe SuedeTaupe SuedeStone Zip Black Side SuedeGrey SuedeBrown nBx07UBqwS, "Heuristic Evaluation." < http://www.useit.com/papers/heuristic/> [ RETURN to TEXT ]
83. Maria Piacente, Surfs Up_: Museums and the world Wide Web, MA Research Paper, Museum Studies Program, University of Toronto, 1996. [ RETURN to TEXT ]
84. Ibid. [ RETURN to TEXT ]
85. Katherine Futers, "Tell Me What You Want, What You Really, Really Want: a look at Internret User Needs." www.open.gov.uk/mdocassn/eva_kf.htm> [ RETURN to TEXT ]
86. Dr. David Barr,"Website Learning," Paper presented at the OMA Education Colloquium, Fall of 1997. < http://www.digiserve.com/adl_ss/research/> [ RETURN to TEXT ]
87. Ibid.NavyNavy Sepia Mid Lowa Renegade BlackDark Grey BeigeBlack GTX HoneySepia qq68Y p.6. [ RETURN to TEXT ]
88. Jane Sledge, "Points of View", Paper presented to ICHIM/MCN ‘95, p.2. [ RETURN to TEXT ]
89. Marcia Bates, Document Familiarity, Relevance, and Bradford’s Law: The Getty Online Searching Project Report No.5. Graduate School of Education and Information Studies, University of California, L.A., 1997. [ RETURN to TEXT ]
90. BlackBrown Bravo Bravo SAS SAS Ibid., p.4. [ RETURN to TEXT ]
91. Ibid., p.5. [ RETURN to TEXT ]
92. Kody, Janney and Jane Sledge," User Access Needs for Project Chio," Unpublished, August 1995.Culltural Heritage Online project of CIMi Consortium. Testing SGML and Z39.50. [ RETURN to TEXT ]
93. As discussed in Museum-L listserve June/July 1997. The website is www.nhm.org/cats>. [ RETURN to TEXT ]
Coral PatentHeartsLeaf PatentWhite PatentBotanical Work by MarbleGrid LeatherBlack PatentTwilight PatentVine Dansko Black PatentBlue Leather Printed Wonders PatentGrey t6PqU the Design Principles: (Gould, 1988) as cited in Baeker et. al. p. 76. [ RETURN to TEXT ]
95. In "From Human Factors to Human Actors: The Role of Psychology and Human-Computer Interaction Studies in System Design," in Ron Baecker et. al., p. 205. [ RETURN to TEXT ]
96. Susanne Bödker, Kaj Grönboek, and Morten Kyng, "Cooperative Design: Techniques and Experiences from the Scandinavian Scene, " in Ron Baecker et. al. p. 215. [ RETURN to TEXT ]
Last modified: March 20, 1998. This file can be found below http://www.archimuse.com/mw98/A 2 Eye Jute Sperry IvoryNude Braided O Welt 8xzpWwd
Send questions and comments to email@example.com