awareness: what is it and how to study it?

Awareness systems leverage sensor data by reprocessing, distributing, and representing them for users. Already in the mid-1970’s, the combination of finger and talk commands enabled Unix users to find out who is online and to chat. Awareness is an important contemporary problem and becoming more and more so. Computer-mediated indicators of remote people feature so commonly in contemporary information and communication technologies that the question of how they are interacted with on lies at the heart of one the most significant development in the history of personal computer: the transformation of personal computer from a tool for an individual to a tool for social intercourse.

This page summarizes my take on awareness, what it is and how to think about it. I have been working in the area of mobile awareness since 2003 when we started to sketch an application that later was named ContextContacts (Figure 1). Frustrated with the feebleness of awareness research in CSCW, I started to develop with my collaborators an approach that borrowed heavily from Social Cognition. I have written this page with the idea of making my research more accessible.

1. What is "awareness"?

I have been arguing that awareness is best understood as as social inference of computer-produced awareness cues. First-order awareness is the the immediate change in a user's belief state that is a results of an inference. Second-order awareness is the more persistent state of knowing that another person is doing something that lasts beyond an individual act of inference. The point of this distinction is that computers may make a user feel being aware of a remote person in the absence of an awareness interface. [2]

Awareness is therefore essentially a mental representation, or a belief state, of somebody else’s current situation. Awareness exists, ontologically speaking, primarily as a mind’s construction rather than as a practice or activity. This does not mean that the activities where inferences take place are not important, as they evidently are. However, it does imply that it would be a fallacy to hold action and inference (or the resulting state of awareness) as somehow analytically inseparable. [4]

2. How does user's awareness this differ from computer awareness?

A user's state of awareness should not be confused with computer-provided awareness information. The basic unit of is an awareness cues. I have defined awareness cues as: 1) computer-produced, as opposed to user-controlled cues such as presence lines, 2) ''live'', i.e. automatically updated in real-time, and 3) symbolic or iconic by type, not analogous like video or audio. [5]

3. What is social inference, then?

Generally, an inference has three components: a) a set of premises, b) a conclusion, c) rules, principles, templates, or procedures which connect the premises to the conclusion in a reasonable manner. This is Reid Hastie's 1983 definition. Social inference is a special case of inference where the conclusion concerns another person or a group of people. Findings from Social Cognition point out that people are unlikely to use all information or evaluate them thoroughly, but rather apply shortcuts and develop simple rules. The whole process is heavily driven by pre-knowledge. [4]

4. Where does this approach come from?

My view is basically a "translation" of Social Cognition research —traditionally focused on face-to-face situations—to a setting where the cues are not “direct” or “natural” but essentially mediated by technology. Social Cognition has its roots in classic European thinkers like Wundt, Le Bon, Durkheim, and later Gestalt psychologists such as Lewin and Koffka, who theorized about the relationship between the psychological and the social. The birth of Social Cognition as a scientific field coincides with the cognitive revolution that put forward an antithesis to Behaviorism that did not accept the inclusion of mental or cultural elements in scientific explanations of human behavior. The foundational question of Social Cognition is the (mental) processing and storage of social information. Under this umbrella, the modern Social Cognition has studied attitudes, attributions, categories, prototypes and other representations, as well as group dynamics, social identity and many other themes; and quite a few of the concepts have gained status within folk psychology. Generally speaking, Social Cognition subscribes to methodological individualism, which treats “the social” as factors in the analysis of an individual. [4]

My work is by no means the first application of Social Cognitive research to the study awareness systems. One influential framework has been that of Carl Gutwin and Saul Greenberg who borrowed Neisser’s notion of schema and Endsley’s notion of situation awareness to explain workspace awareness. Another influential piece of work, although carried out in a different domain, has been the analysis of social cues in email by Lee Sproull and Sara Kiesler. My view is that despite such endeavors, social cognitive approaches have failed to secure a position in mainstream awareness research. I feel that this is part because have not reflected the depth and breadth of work carried out in the mother science.

5. How does it differ from previous ones?

There are roughly speaking three ways to explain how awareness cues influence human action. First, one can argue that an awareness cue, say "Antti is away from the keyboard," is associated with one dominant interpretation, for example an availability inference, such as "Antti is not available for chat." In other words, the content of a cue determines the interpretations which it can reliably serve, just as the buttons on a dialogue box determines the possible uses. Designers are thus the arbiters of the use of their system and there is little flexibility on users' part.

Second, according to one reading of the theory of Lucy Suchman, human action is best understood in terms of how it makes use of contextual circumstances, not in terms of fixed mental capacities or preprogrammed action plans. Awareness cues are part of these circumstances and they feature in human action as situational resources, one among many. As “the situation” is the arbiter of how humans act, any given cue is in principle infinitely “flexible”. Therefore, designers have little or no control over how the system will be used.

Between these two extremes is a third view that places the locus not to the cue or to human action, or its context, but to human mind. There must be some logic in and limitations to making sense of a cue, and this logic must somehow be sensitive to the situation at hand. We know from our everyday experience, and from research, that people sometimes succeed in interpretation and can go beyond the literal meaning of given information, and we also know that sometimes interpretations go wrong. What explains the different outcomes of interpretation of a cue?

6. What do we know about the inference process, how does that work?

According to the findings of Social Cognition, people do not thoroughly evaluate all available information, as implied by normative theories of rationality, but apply shortcuts and simple rules to overcome cognitive limitations. These rules draw heavily from pre-knowledge about the other person and are sensitive to frequencies of events in the world. Pre-knowledge and inferential shortucts together enable “jumping into conclusions”, quickly, and they enable arriving at interpretations that go beyond the literal meaning of the cue. The downside is that biases and errors are bound to occur, and effort is needed to turn or override routinely produced interpretations if they are deemed negative.

I have argued a practical and useful approach to understanding inference process distinguishes among four elements [2, 4]:

• Awareness cues.
• Processing goals. Processing of awareness cues is directed by goals of the user. Depending on the goal, users apply different inference frames (pre-knowledge), inference rules (heuristics), and selectively pick one or more awareness cues to process.
• Pre-knowledge. Effects of pre-knowledge are among the most robust findings in Social Cognition. I have utilized a simple distinction between general knowledge (e.g., knowing that one ought to turn off audio alarms while in a movie theater) and specific knowledge (e.g., knowing that John does not do that).
• Inference rules. In [2], we charted some of the heuristics users use to make sense in the presence of multiple cues. We found for example that users often anchor their processing of multiple cues to a hypothesis formed on the basis of the first cue. In our data, the most frequently used anchors were location cue and online status cue.

7. What is known about how mobile users process awareness cues outside laboratory?

A) Typical processing goals.
In a set of field studies of ContextContacts [2], two active processing goals were prominent: coordination, of mobility or communication, and self-expression and discussion. Active perceivers in general concentrate primarily on how the question their behavior affects that of the responder's of their target and they therefore tend to ignore other important sources of information. The third processing goal identified in that set of trials, 3) companionship, is in turn associated with more passive and holistic processing of cues. Many users mentioned feeling of companionship as the main benefit of using the system; a rather different benefit from utilitarian ones. Users expressed feelings of presence, closeness, affection, communality, and connectedness as being mediated by awareness cues. Parenthetically, emotional effects of awareness cues would not be covered by the original definition of (user) awareness given by Paul Dourish and Sara Bly in 1992, who defined awareness as understanding of activities of others that contextualizes your own activities.

B) The breadth of inferences from a single cue.
We have been very surprised by the ability of users to go beyond the literal meanings of awareness cues. Consider, for example a group of schoolmates who, during a three week trial, learned to utilize ContextContacts' district cue to make inferences including place (home, school), location (floor in a building, particular room), movement (being on the move, next probable destination), co-presence of others, potential for action (availability for communication, availability for face-to-face meetings) [2]. Upon learning interpretational skills, even very mundane awareness cues can give raise to remarkably subtle inferences about another person. This finding has significant implications to the design of cues [6].

C) Central inferential frames.
In our examination of which kinds of pre-knowledge is utilized in inference through awareness cues [2], we identified four dominant categories: 1) knowledge of another person's current activities, 2) time of day, 3) known regularities in behavior (esp. movement), and 4) expected behavior in social situations were the most referred to inferential frames. Based on general knowledge, one can make some inferences even of a total stranger [4]. However, to reliably infer another's activities and intentions, a deeper relationship is necessary.

D) Speed of inference.
In [2] we also logged everything that happened in the application, all the way down to movement of cursor in the contact list. From that data, we measured what we dubbed "the pre-call delay": the time that a user keeps a contact highlighted in the contact list before pressing the button to place the call. This is indicative of the time consumed in processing the cues, most probably spent assessing whether the person is going to be available or not. When we compared cue-augmented contacts to unaugmented contacts, a large difference was observed: When there are no cues for a contact, 60% of calls are placed within only 0 to 1 second after moving the selector on top of the contact. When the cues are present, 60% of time users stop for 1, 2, or 3 seconds before placing the call:

E) Processing of multiple cues.
The problem of multi-cue awareness interfaces is that they in a sense provide too much information. In [2] we observed that users learned almost habitual ways to pick up one or two cues and make a rapid inference based on that, e.g. "she's busy because moving". Another heuristic we observed was anchoring, which is best illustrated by the following quote: "The hand was mostly white, but it did give more hope when it was red. At that point when you call, I do not often look if it's red or not. The only thing is that when there's no audio or tactile alarm, there's no hope of reaching the other person if the hand is white. But if it is a red hand, you usually thought that he/she might notice your call. (Entrepreneur)"

8. Why are some cues more difficult to interpret than others?

Looking at any multi-cue system (like ContextContacts [3]), one finds some cues easier to understand than others.One explanation for apparent fluency of processing is facilitatation by transfer of interpretation skills. For example, most of us already know the significance of time of day as an implicit cue. Online status cues in mobile awareness resemble cues in IM and location cues resemble location-disclosure carried out at the beginning of phone calls. This observation has an important ramification: As developers and designers, we should not justify some ideas for awareness cues just because we cannot easily entertain any uses for those. In studies of ContextContacts [2], we have seen how difficult for example the Bluetooth-based cues are at first, but how users become more and more observant in their practices of using those cues.

9. Why are there individual differences?

Despite the fact that the description of social cognitive mechanisms is framed in terms of judgmental errors, there are many virtues, but they are achievable only through experience and practice. By developing inferential skills with an awareness cue, users can improve

• Familiarity: the ability to transform an inference problem to a more familiar form where previous knowledge can be applied.
• Selectivity: the ability to sift the relevant from the irrelevant.
• Anticipation: the ability to take into account experienced frequencies of occur-rences to anticipate future events.
• Situational sensitivity: the ability to take into account situation-specific (e.g., schemas) and person-specific knowledge (e.g., person memories) in one's inter-pretation. On the one hand, one should balance coherence of action over time, so that any new situation does not arbitrarily change the course of action. On the other, one must be sensitive to particular situational features. If preconceptions dictate the inference, the cues at hand will have no informative value and they would be useless.
• Rapidity: the ability to process information rapidly enough to accommodate the moment-to-moment demands of social interactions.

Note that all of these are dependent on practice and experience that can only develop with time. A direct implication is that user studies of awareness systems should provide enough time for users to learn interpretational skills -- and develop conventions within the user group.

10. What makes mobile awareness special, if anything?

The answer to this question depends on what is meant by the term mobile. If it refers to movement of the user, mobile awareness is special in the sense it has to capture the meaningful relationship between the represented user and her context. If it refers to mobile devices, mobile awareness is unique in how it is riddled by the limitations of interfaces [5]. If it refers to the negation of desktop-bound interaction, mobile awareness is special in the sense it has to deal with the whole repertoire of human needs and roles, from work to leisure, from the spouse to the hobbyist, from the happy to the sad, and so on and so forth.

I see that mobile awareness is going to introduce a reneissance of awareness research. Just think how interesting cues are enabled by mobile and ubiquitous systems and compare them tho those that were researcheed in groupware!

11. Are there implications to design?

Absolutely! In a recent paper [5], I go through the design issues more systematically, covering questions like:

1. Design of an individual cue: Which modality to use? How abstract the level of description needs to be and what to do with outlier data? Should one use imperative or neutral cues? Are predictive cues better than descriptive?
2. Cue layout: Are there degrees of freedom in how cues can be laid out on a screen and across many screens? One cue or many? How to design access to a cue? How to support mobile use?
3. Integration: Should cues be integrated into other functionality and content and, if so, how?

An excerpt from the paper [5], a table summarizing considerations for various design choices:

12. Are awareness cues useful also in non-awareness applications?

Yes! An awareness cue is a powerful design concept. Awareness cues are small, integratable, and generic. In CoMedia [1] (Figure 2), we explored integration of awareness cues to a mobile group media application. We ended up using a set of icons consistently throughout the interface:

All other cues were expressed in textual format. Across user studies of prototypes where cues have been integrated to media or communication functionality [1,2], we have found a couple of re-occurring benefits of integration: 1) they increase communication success and reduce the rate of failed call attempts, 2) they are useful in coordination of mobility and turntaking, 3) they afford ad hoc encounters, 4) they promote feeling of companionship within the group, 5) they help making sense of other's actions (e.g., "why did X reject my call?"), 6) they are topicalized, i.e. raised as topics in conversation, 7) they help streamlining communication once users learn to anticipate their presence (e.g., you don't have to write "I'm in Berkeley" if the location cue already tells that.)

13. How can we study inference in user trials with real prototypes?

This approach may appear as too heavy to be adopted for real prototype trials. Furthermore, social psychologists tend to study social inference in controlled laboratory conditions; How valid can field studies be anyway? The key method in our studies has been cue-based narrative elicitation. We have combined this with logging [2] and observations [1].

Cue-based narrative elicitation is an interview-based method. An awareness cue is pointed to the interviewee, and the aim is to get her recount real, actual episodes of using that cue for something. We typically ask for the latest event that brings into mind. The corpus of episodes that is gathered this way can be analyzed either qualitatively (e.g., what kinds of awareness or presence experiences were mentioned) or quantitatively (e.g., using a coding scheme, one can count how often do the users refer to experiencing remote others’ presence). [2]

14. Where is awareness research heading?

Awareness research is advancing rapidly on one of its fronts: the awareness cues. Upcoming applications tap emotions, moods, social situations, social networks, and technological environments. New applications have been presented that are not just work-related, task-based but activity-based and situation-based. In the near future, I believe to see cues that represent not just individuals but dyads, groups and teams, and perhaps whole communities. On the brink of the ubicomp era, I also expect cues that leverage interoperability by by being integrated across whole complexes of services. These developments, which I think are inevitable and already taking place, only make it more relevant to develop the theoretical and methodological bases of awareness in HCI. More is not necessarily better.

acknowledgements

I want to acknowledge and thank the contributions of close collaborators Mika Raento, Giulio Jacucci, Sakari Tamminen, Antti Salovaara, Esko Kurvinen, Esko Lehtonen, John Evans, and Sauli Tiitta.

references

1. Jacucci, G., Oulasvirta, A. et al. (2007). CoMedia: Mobile group media for active spectatorship. Proceedings of CHI 2007, ACM Press,  New York, pp. 1273-1282. pdf
2. Oulasvirta, A., Petit, R., Raento, M., & Tiitta, S. (2007). Interpreting and acting on mobile awareness cues. Human-Computer Interaction, 22 (1&2), 97-135. (pdf)
3. Oulasvirta, A., Raento, M., & Tiitta, S. (2005). ContextContacts: Re-designing smartphone's contact book to support mobile awareness and collaboration. Proceedings of Mobile HCI 2005, ACM Press, New York, pp. 167-174. pdf
4. Oulasvirta, A. (in press). Social inference through technology. To appear In P. Markopoulos, W. MacKay, & B. de Ruyter (Eds.), Awareness Systems: Advances in Theory, Methodology and Design, Springer, Series on HCI.
5. Oulasvirta, A. (in press). Designing mobile awareness cues. Proceedings of Mobile HCI 2008, ACM Press, New York.
6. Raento, M., Oulasvirta., A., Petit, R., & Toivonen, H. (2005). ContextPhone: A prototyping platform for context-aware mobile applications. IEEE Pervasive Computing Special Issue on Smartphone, 4 (2), 51-59. (pdf)
7. Raento, M., & Oulasvirta, A. (in press). Designing for privacy and self-presentation in social awareness. Personal and Ubiquitous Computing. (send email to receive a copy)