Paul Dourish
Apple Computer, Inc.
1 Infinite Loop MS:301-4UE
Cupertino
CA 95014 USA
dourish@acm.org
(Current address: Xerox Palo Alto Research Center, dourish@parc.xerox.com)
For all their differences, all computer system design approaches share at least one form of representation -- computational representation, embodied and embedded in the software. Software design and programming are, fundamentally, representational processes, concerned with the definition, creation and manipulation of computational representations. This paper considers the nature of computational representations in interactive software, with a particular concern for how they come about and how they are used. I consider a number of problems in interactive systems and relate them to this problem, and, based on a number of recent developments, outline a new computational approach to representation which offers a solution to these problems.
However, one form of representational practice is common to all approaches to interactive software design, and that is the computational representation in the software itself. Representation is not only a tool in the design process but it is also, critically, its output.
Software design and development is fundamentally a representational process. Computer programs are formalised representations of complex behaviour, and they achieve this behaviour through the manipulation of data structures which are themselves representations of entities beyond the scope of the software itself. Looking "under the hood" provides us with no escape from the representational. Programming languages are mapped onto processor instruction sets (more representations) which are implemented using digital (representational) logic.
The question which will concern us here is that of the ontological status of these representations. Programming is an intentional phenomenon, and so the question of the status of the representations developed there is one of critical importance. Furthermore, while real-time process control systems concentrate on representing plants and mechanisms, interactive systems spend much of their time representing people and their work. HCI's concern with the role of systems in supporting people and work must, then, be based on some understanding of the representations through which computer systems operate.
The fact that software is inherently representation is not at issue here. Instead, my concern is with the forms that those representations take and the uses to which they are put in interactive systems.
Clearly, as in any other form of software, a significant number of the representational artifacts embedded in interactive software are those concerned with the system itself and with its operation. Cursors, screen positions, printers, windows, dialogue boxes and responses are all elements common to much modern interactive software.
What is particularly interesting about the representational aspect of any software system, though, is that set of representations which refer to entities outside the domain of the computer system itself. Representations of cars, valves, stock prices, product orders and pressures all reflect the software's intensional stance on the world [Smith, 1996].
In interactive software, we also direct our representational energies towards the other partners in interaction -- users and the artifacts of their work. However, this introduces a paradox. Computers are closed systems in which operation depends mechanistically on only and exactly those factors about which the system has "knowledge" (or a representation). People and work, however, are subject to an uncountable number of factors. Work is organised as much around politics and earthquakes as it is around roles and procedures.
The representations of users, work and work artifacts in computer systems, then, can be no more than that -- representations rather than objects, and representations which reflect only limited aspects of the objects themselves.
One particular and significant set of interface problems which have been the focus of my recent work concern the mismatch between the rigidity of computational representations and the fluidity of human activity. This concern lies at the heart of a number of areas of research investigation in Human Computer Interaction, from the concern with customisation and tailorability to investigations of work practice. For example, the participatory design and user-centered design approaches can be seen a response to poor "fit" between technological systems and human ones, which is then irreversible since computational representations are fixed and unresponsive.
Rather than taking this problem to be one focused on design practice, I have instead taken it to be focused on the artifacts of design -- on software. My concern has been with how the conceptual and practical structures of software development are grounded in precisely this rigidity.
The critical problem with many representation uses in interactive systems design is that they can become too seductive. Designers can be enticed into an ontological shift in which the representations come to be used as if they were the things for which they stand.
This problem (or at least some aspect of it) is at the heart of debates such as that between Suchman and Winograd (CSCW Journal, 2(3), 1994) over the politics of representation in collaborative systems, and in particular in workflow technologies. While Suchman is concerned about the social and political import of representation and of what it leaves out, Winograd argues that technologies necessarily represent, and that, as far as technology is concerned, a non-representational stance is no stance at all. However, I would claim that the issue is about how the representations are used and in what relation they stand to their referents. Suchman's arguments are based on an assumption that this ontological shift takes place (which is all too often true).
The perspective I take here, then, is one which starts by asking not what is represented, but why, and one which takes the correspondence between representation and object as one of ongoing achievement and work. Representation is a matter of use -- representations are as representations do. We naturally wish our representations to be stable, or else they cannot support the functions of coordination across time as we would require them to. On the other hand, "stable" is not the same as "static". The perspective on representation which will be illustrated in the rest of this paper is one which requires the representations to be worked rather than used.
The earlier sections have introduced the two key elements of the work described briefly in this paper. The first is a concern with how the stable correspondence between representations and their referents is managed and maintained, rather than simply positing that it exists. The second is a concern with representation not only in the design process, but in the artifacts and outcomes of design. However respectful we might be of the fluid nature of work and work settings in our design process, our efforts will be for nothing if cannot be equally (or appropriately) fluid in the systems which we develop. The representational question is, at heart, a technical question and requires a solution which is technically adequate.
As a basis for this technical requirement over the past few years, I have been exploring an approach to software architecture based on "computational reflection".
Computational Reflection [Smith, 1984] is an architectural principle which was developed and explored first in the domain of programming languages but which, lately, in a variety of revised forms, has been applied to a wide range of computational problems. The reflection principle states that a computer system can have a representation of its own activity which is causally connected to that activity. In other words, there is a bidirectional relationship between the representation and the activity. Changes in the system's behaviour are reflected in changes in the representation; and similarly, explicit changes to the representation will cause changes in the behaviour. More recently, this principle has been exploited in the development of the metaobject protocol [Kiczales et al., 1991], an object-oriented encoding of internal system behaviour, and in Open Implementations [Kiczales, 1996], an approach to system architecture which separates the use of system abstractions from their configuration and implementation.
This approach to system architecture constitutes a radical revision of the representational status of abstraction in software engineering and design [Kiczales, 1992]. Abstractions serve as a fundamental representational tool in the development of software, but they also serve to meet the goals of modularity, separation, management and reuse. As such, we have developed the "black box" abstraction which offers a representation of component functionality while hiding the implementation details. The paradox here is that it is only through the details of the implementation that functionality can be achieved, and the "information hiding" aspects of abstraction can obscure the details which are significant for the use of the abstraction in any particular circumstances. The Open Implementation approach reconfigures this notion and creates two levels for talking about abstractions. The first, the base interface, is the traditional model in which the abstraction is used to achieve some higher-level functionality. The second, the metalevel interface, is used to talk about the base level interface, and to control its behaviour according to a revealed model of the internal behaviour of the implementation (the reflective representation).
We have been using this approach in a variety of interactive system projects to investigate how this reconceptualisation of abstraction can be used to make the representations upon which interactive systems are based more fluid and responsive to the varying patterns of use which exist in real world settings. To give a flavour of this approach, I will briefly present three research systems which take different directions based on this one approach.
Prospero [Dourish, 1996] is a toolkit for creating CSCW applications. Prospero was designed to offer the programmer considerably more flexibility than traditional toolkits, and therefore to provide a more effective bridge between the capabilities offered by any particular system and the flexible needs of a variety of applications, settings and groups.
A traditional toolkit (or programming language) offers a language for representing the behaviour of applications. It provides a set of facilities, constructs and mechanisms for connecting them together in order to generate a particular application. The range of ways in which they can be combined and used determines the range of specific applications which can be generated. However, the mechanisms offered by any particular implementation of a language or toolkit contain commitments to particular styles of end-product. For example, in a collaborative toolkit, we might offer a "shared object" which will be viewable and editable by all users in a collaborative session. However, the details of precisely how this object is shared affect how it can be used in any particular circumstances and what sorts of applications can be developed with it. The choices of replication strategy, locking, conflict management, etc., carry implications for, for example, interactive response and the extent to which users can be aware of each other's activity -- critical aspects enabling collaboration to proceed [Dourish and Bellotti, 1992].
Like any other CSCW toolkit, Prospero provides a set of mechanisms and behaviours which can be used to create collaborative applications. However, it also provides a metalevel interface so that the details of precisely how the system goes about achieving its functionality can be revealed to the programmer and controlled for particular applications. For instance, the programmer can control the synchronisation mechanism so that, within the same programming framework, the programmer can control the synchronicity of the toolkit and create applications which are "synchronous", "asynchronous", or fall at points in between [Dourish, 1995b]. Similarly, the programmer can use the metalevel interface to configure the synchronisation mechanism in terms of meaningful semantic actions in the application domain, rather than lower level data storage semantics [Dourish, 1996b].
In other word, Prospero provides not only a language for representing collaborative application behaviour, but also a means by which those representations can be controlled. The programmer can become involved in what the representations mean in any given circumstance, and so tailor what lies behind those representations to accommodate the needs of different applications and settings.
Freeflow is a prototype workflow system developed at the Rank Xerox Research Centre [Dourish et al., 1996]. Based on studies of workflow systems in real use, such as that of Bowers et al. [1995], Freeflow's primary research goal was to investigate the ways in which a workflow system could more fluidly mediate between process representation and process execution.
Like any other workflow system, Freeflow embodies a representation of working procedures, which reflect a set of dependencies between a procedure's component tasks. However, while most workflow systems conflate physical, organisational and temporal dependencies, Freeflow allows these to be specified and acted upon independently. A Freeflow user can engage in the different activities making up a procedure without regard to a fixed temporal order. Freeflow does not impose such an order; instead, it maintains a set of constraints between tasks and maps the deviations between the abstract process and any particular concrete instance of it. Freeflow manages the physical and organisational dependencies as constraints between activity instances, and provides support for users to break those dependencies temporarily as required by particular circumstances.
The process representation, then, plays a different role in Freeflow than it does in traditional workflow systems. In the traditional approach, the representation serves to constrain and control user action, and the ways in which users engage in tasks. In Freeflow, in contrast, it acts a resource for relating individual actions, as they arise and occur, to a larger process. it describes relationships and provides ways to see the emergence of procedure activation as a whole without imposing unnecessary restrictions on the actual execution.
The lesson of a range of studies of procedure management in real world settings is that the execution of these procedures is a creative, improvised process in which users flexibly manage the requirements of the procedure within infinitely varied circumstances and particular settings. In Freeflow, we provide explicit support for this fluidity by reconstituting the role of the process representation, as resource rather than restriction.
Most recently, we have been investigating a new model of interactive system design which is aimed at supporting the styles of improvised, situated activity which have emerged from sociological investigations of interaction work [Suchman, 1987]. Drawing on the same ethnomethodological foundations on which Suchman based her work, we have been investigating the use of reflective representations as accounts which systems offer of their own action [Dourish, 1995a].
This work forms part of a larger programme of research which aims to forge foundational (rather than practical) relationships between computer systems design and sociological investigation [Button and Dourish, 1996]. In the accounts work, we have been looking in particular at the relationship between abstraction (as a technical concept) and accountability (as drawn from ethnomethodology). For ethnomethodology, a fundamental property of human social action is that it is accountable, by which they mean that humans organise their activity to make it recognisable as precisely the sort of activity it is. It is in this organisation that people find the means to make the social world meaningful for practical purposes. Accountability is the way in which action is (and is made to be) observable and reportable in the course of its production. By providing the means for action to be made and seen as meaningful, the property of accountability supports the concerted character of human social world. In contrast, however, the abstractions by which computational action is structured are manipulated without reference to a larger structure; and the information hiding properties which we employ in computational abstractions stand in stark contrast to precisely this observable-reportable nature which ethnomethodology finds in human action.
We employ the reflective approach in order to provide people with the means to find computational activity (more) meaningful, through the same notions of observability and predictability. Reflective representations can be used to provide a context for the behaviour of interactive systems, by placing the instantaneous phenomena within a frame of the system's mechanism. What the reflective representation reveals about the structure of the implementation becomes a means to find the particular behaviour in which the system engages meaningful.
There are three components to this investigation. First, it is an architectural proposal, arguing for a new way of constructing interactive applications based on an appreciation of the relationship between interaction and system structure. Second, it is an interface proposal, requiring investigation of new ways to structure and present information about system state and action [Curbow and Dourish, 1997]. Third, it is a conceptual proposal, reorienting our perspectives on the nature of interaction with computers, moving from an "I-say, you-say" dialogue model to a richer one of the co-production of work.
The three projects described above were conducted separately (or at least with as much separation as is possible given that one person was involved in all of them). However, they form the basis of a larger research agenda.
The agenda is to understand the relationship between computational representation and representation as conducted in the world as an everyday practical matter. This tackles Suchman's question head-on, but places it squarely within the frame of technological development. The sensitivity which we display to representations of people and work in many of our emerging design approaches reaches, it seems, only so far. When the voices have been heard, and when the fluidity and richness of the working environment have been noted and appreciated, we set down, in ones and zeros, the rigid computational structures through which we will interact.
So a richer notion of representation in interactive systems, and an understanding from which its implications can be more thoughtfully examined, must be one which encompasses that one form of representation which all interactive system designs have in common -- the computational representation which the software embodies.
I have discussed issues of representation in interactive system design from the perspective of software development. All system design activities share at least this one form of representation, and yet we consider the implications more in terms of the design process than of the artifacts which are its outcomes.
In recent work, I have been exploiting a revised notion of abstraction in software development to tackle these problems in a variety of interactive system domains. Computational reflection provides the technical background not simply to take representations as we find them, but to look beyond to the question of how they come about and how they are sustained, as the outcome of an ongoing effort.
These projects point to a range of ways in which we can reconsider the nature, role and use of software representations, and provide a more comfortable fit between users and technology.
Acknowledgments
Most recently, I have been developing these ideas along with colleagues in the Discourse Architecture Laboratory at Apple Computer, including Dave Curbow, Tom Erickson, Jed Harris, Austin Henderson, and Brian Smith. Their contributions are gratefully acknowledged.
References