The Role of Interaction Histories in Mental Model Building and Knowledge Sharing in the Legal Domain

Abstract: This paper reports on a study examining attorneys' and law librarians' use of their memory and information they record externally in searching for, using, and sharing legal information. The paper suggests automatically and manually recording search histories and basing user interface tools on this information to support mental model building and knowledge sharing in the legal information domain. The research described is part of the author's dissertation research [1] that examined the use of search histories in legal information seeking and use, and proposed interface design recommendations for information systems. While searching for and using information, attorneys learn about legal topics and use this knowledge in their work. They create mental models and share their new knowledge with colleagues. Computers can automatically record human-computer interaction events. This information can help searchers represent and share new knowledge. The recorded information can be provided back to the user through the user interface to support searching for and using information, learning about the subject matter and sharing this knowledge with others. In this study, attorneys and law librarians were interviewed and observed to assess their use of their memory and external memory aids while searching for and using legal information. The results reported here focus on the role of interaction histories and history-based interface tools in supporting mental model development of legal information seekers of a topical area and sharing this information with other users.

Keywords: Search Histories, Information Retrieval, Legal Informatics

Categories: SD D2.2.2, H.3.3

1 Introduction

The research reported in this paper examined the information-seeking behavior of attorneys and law librarians in the American legal system with special attention to their internal memory use and external history-keeping practices. Attorneys and law librarians in the U.S. legal system search for and use legal information stored in online databases on a day-to-day basis. Precedent-based legal practice requires legal professionals to refer to court opinions to establish the current interpretation of the law. The precedent-based legal domain is highly information and knowledge-intensive. Legal litigation work is highly dependent on the proper identification, retrieval, and management of up-to-date information. Legal information is historically based, it develops over time and builds on previous information: in the American legal system, the law is interpreted through court opinions written by judges in courtrooms. These court opinions often cite other previous court opinions and legal cases, strongly building on earlier information. Keeping up-to-date with new developments and locating relevant information is a professional and ethical responsibility of attorneys. Helping attorneys with searching for, finding, managing, and using this information is a challenge for information and knowledge management systems.

Search histories and search-history-based user interface tools are proposed to answer this challenge. Search histories are computer-user interaction histories automatically recorded while attorneys and law librarians search for legal information in computer-based databases. Search histories are sometimes called interaction histories in this paper. They contain a record of user steps, systems responses, documents, and other information in a temporal order.

The attorneys and law librarians in the study used the Westlaw system (http://www.westlaw.com) to search for information. The Westlaw system currently has nearly 15,000 databases containing legal, business, and general information. Attorneys search for information in these databases through formulating queries, examining result sets, selecting and reading documents. Attorneys' use of the information usually coincides with their information search; the two activities are often carried out in parallel. Using information involves reading and interpreting information returned to users in the form of court opinions and other documents. Relating the new information to the attorney's task at hand, deciding how and what to use the information for, and finally using the information in new documents are all defined as information use in this paper. While searching for, interpreting, and using this information, the user's interaction history can be recorded in computerized environments and fed back to him through the user interface to support individual learning and knowledge sharing. This interaction history contains user steps, system responses, documents, and other information. In addition to providing a direct display of search histories, other information management tools can be developed, such as functions to support the collection, processing, and sharing of information.

Attorneys and law librarians were observed while searching for information and interviewed afterwards to study user behavior and investigate user needs. The data collected informed the design of prototype user interfaces to present search history information, attorneys and law librarians were also involved in the interface design process. Research results revealed that recording users' steps automatically and allowing them to record their interpretations and thoughts manually through notes and annotations are useful features of information systems that can help users learn and share their knowledge with others. The paper will first review related research, then describe the methodology used, and finally discuss the results related to learning and knowledge sharing. For a more detailed review of the results please see [1].

2 Related Research

Sutton [2] described attorneys' mental model building while searching for information. He criticizes earlier studies for defining relevance as pure topicality. He steps back and defines relevance for legal research as a first step in evaluating legal information. He characterizes relevance as a function of the mental models or conceptual map of the law constructed and maintained by attorneys:

AA relevant case is one that plays some cognitive role in the structuring of a legal argument.@

Sutton's definition of relevance in the legal information field builds on the event space of the case, placing other similar cases in this space.

Sutton describes legal practitioners' cognitive maps of the law as having three levels:

1. base-level modeling of the contours of the event space;
2. context sensitive exploration of the space and the populating of the relevant subsector;
3. disambiguation of the subsequent model.

He also describes these stages in terms of information-seeking activities, sources, and tools used. The first level is often accomplished through training, learning about the general issues of an area of the law. The second level, context-sensitive exploration@, focuses on a particular issue, how the legal principle has been applied to the facts of the reported case. The third level describes the process of disambiguation among the results retrieved by the attorney's searches. In this process, cases in the event space are evaluated based on their juristic status and treated according to their status. If their jurisdiction does not require their use in the attorney's litigation, they are removed or their influence diminished. The cases are Shepardized (their subsequent history checked for negative and positive changes in interpreting the law and the decision) in order to examine their current status and the results of this are also taken into consideration when deciding the impact of the cases. At the end of this process the mental model is finalized for the time being. Sutton remarks that the three processes are going on in parallel in real-life information-seeking situations. The mental model building described in the current paper focuses on the second and third levels, in which the attorney starts out with a picture of the legal area, explores the conceptual space with the help of this model, updates the model and then uses it to judge new results and updates it with new information found. Recorded search histories can help the development of these mental models through various tools and can represent them in external forms. A list of documents and text sections looked at, organized, and annotated by an attorney can later remind him of his thinking, support learning, and can represent new knowledge to other collaborators. The results section describes user's second- and third-level mental model building that can be supported by search histories. The next study reviewed also studied legal information seeking and use, although focused more on the physical actions of users.

Marshall et al. [3] described a study of law students preparing for a Moot Court competition where they practice case litigation against other student teams. Their findings mirror the findings of this study in many respects. Especially interesting here are the functions needed to support learning and information use by law students based on recording students' actions, and allowing students to annotate and organize information. They examined the students' information-seeking, reading, writing, and learning activities, including annotation techniques, in order to test and design an e-book technology, XLibris.

Marshall et al. [3] found that the ability to annotate and organize information is very important in supporting legal work, pointing to the need for interface functions supporting these. Captured annotations and organization schemes can be used for learning and sharing information with others. Marshall et al. point out that annotation techniques are taught to law students, and often different annotations are used to prepare a document for different purposes. Re-reading and re-annotations are frequent in the legal field; often annotations are overwritten or selected annotations are marked for a second time.

Annotations vary in importance and usefulness. Students also often created reminders and plans through annotations on document printouts that they later used to guide their further research. These annotations can serve as potential mental model representations of the students' interpretation of the information and they also serve as drafts for documents, along with the user's organizational scheme.

Documents collected for the Moot Court trial are organized according to the tasks and purposes they will be used for. At the same time, as students get closer to writing documents of their own, their organization schemes become closer and closer to their writing objectives. Organization schemes were activity-based and changed through the process of working with the documents, reorganizing information was a way to conduct work. The authors suggested flexible organization tools that allow reorganization easily.

The redesign suggestions for XLibris focused on the following areas: navigation, retrieval, annotation, and organization, the last two being of most interest here. Improvements to the annotation tools included the ability to easily re-annotate previously marked documents and also the ability to annotate previous annotations without having to go back to the original document, thus supporting thinking. The authors also added a notebook feature where users could collect clippings annotations. More flexible organization tools were also added to the original design: workspace labeling and divider pages were introduced. Both the annotation and the organization tools and other information processing tools can be helpful to users to learn about a topical area and build mental models.

3 Methodology

Initial exploration of the topic was carried out through a literature review, interviews with reference librarians in a special library, and an analysis of usability testing videos of the Westlaw legal information system. The conclusion of these preliminary data collection effort was that search histories can enhance information seeking and current tools need improvements to satisfy user needs. Based on these findings, an iterative methodology was designed including user behavior data collection, search history framework development, and interface design. Attorneys and law librarians were observed while searching and interviewed afterwards to learn about their memory use while searching for and using information. Eight participants were involved in the observations and interviews conducted by the author. Additionally, eight interviews from [4] were generously made available for this study by T. R. Halvorson. Participants in the first group were asked to search the Westlaw databases for a topic of their choice. The problem had to be subject-oriented and involve several linked questions. While searching, participants were asked to think aloud. After the searching, they were interviewed about the search session and about their memory and history use in general. In the Halvorson interviews, participants were interviewed about their information-seeking practice, many of which involved the use of history mechanisms. Transcripts of think-aloud sessions, interviews and observation notes were analyzed and the results led to theoretical framework development and the design of interfaces. These were used in the participatory design sessions, and further interfaces were also designed by participants. The transcripts from these sessions were also analyzed and they informed the theoretical framework.

Prototype interfaces were developed through several iterations with intensive input from participants. These were critiqued by several groups of legal professionals, however they were not formally tested. Formal evaluations of the usefulness and impact of these interfaces are planned for the next phase of the research.

4 Results and Discussion

The results indicated that legal information users use their memory and automatically and manually recorded interaction history information to support their information seeking and learning. In specific, the use of interaction histories and history-based tools in mental model building and knowledge sharing are described here. The involved user needs assessment and user behavior study resulted in the design of user interface tools. As mentioned above, although these tools were demonstrated to study participants and their reactions were collected and applied, formal evaluation studies are part of future research plans.

The first section (4.1) describes individual user's mental model development through external representations of his mental model, information organization, and annotation tools and actions. The second section (4.2) builds on these external representations to discuss sharing information and knowledge with others.

4.1 Mental model building

4.1.1 Introduction

A user's mental model is a cognitive model of the topical area the searcher builds while looking for information, it is in a sense the final outcome of the searching phase, the result of interpretation and the first step of using the information. Building a mental model of a legal topical area is tightly related to the interpretation of information found while searching. By integrating new information into the knowledge structures of the user, it becomes available for reuse in future work. This section describes how automatically and manually recorded interaction histories can help externally represent the user's new knowledge, experience, and mental model. It then discusses the user's mental model building activities that can be supported by user interface tools building on search histories, especially information organization and annotation. These activities are integral part of an attorney's work and they create external knowledge representations that can be stored and shared with others.

Mental models are one way to think about knowledge representation in memory. Mental models are cognitive representations of legal topical areas that constantly change in light of new information encountered by the legal professional. Searching stems from an information problem, a gap or other discrepancy in the user's knowledge (and knowledge representation). Interpretation during searching aims at applying new information found to the original state of the user's knowledge and repair the discrepancy. Interpretation and mental model building involve reading the documents found, often rephrasing them in some form, assigning meaning to them through linkages to current knowledge, and recording the process and the results in order to build them into the knowledge network and mental model of the searcher. Interpretation builds the user's mental model and is often reflected in the interaction history of the user.

4.1.2 Interaction histories as mental model representations

A record of what information the user encountered and how she reacted to it can help in externalizing users' knowledge models of an area. Recording search histories and allowing users to manipulate them can help with:

1. recording and preserving results of interpretation,
2. the heavy processing of information helps transfer it from working memory to LTM, and integrate it with pre-existing knowledge structures, thus building a mental model.

Allowing users to view and manage their interaction histories makes them aware of their progress and the information they found. It can help them learn new information better by being exposed to it longer, and can also help with planning and monitoring the search process. Explicitly presenting the history display as it is being built makes users aware of the recording and can lower privacy concerns.

The process of interpretation and learning about a topical area is cumulative, past knowledge forms the basis of handling new information. Interpreting and integrating search results with old knowledge will eventually lead to the answer; however without recording the results of integrating new knowledge into old models, the answer may be lost by the end of the process. Searchers can try to remember all the information found, but this may be difficult with only one exposure to it and lack of manipulation. Keeping track of this process in electronic environments through history-supported tools is an obvious application area of search histories. Recording results is a good foundation for recording their interpretation by the user and their linking to current knowledge.

4.1.3 Support for user mental model building

Interpretation leads to the development of mental models through the integration of new information with the previous knowledge of users. Attorneys read documents, took notes, printed documents, annotated them, organized and reorganized them while searching for and interpreting information in the study. These actions helped them make sense of the information, learn it, and prepare it for future use. Providing tools for these actions and recording their results over time can help searchers build mental models and learn, as discussed below. External representations of mental models can be helpful this process, participants often described processes to record these.

Tracking the interpretation carried out by searchers is important but challenging. Recording the thinking of searchers is a more complex task than recording their action, it can only be recorded if the user explicitly enters written/typed notes, annotations or voice recordings. Systems should provide tools for entering notes and annotations linked to interaction histories recorded automatically; these can help record interpretation and the user's thinking. They help interpretation by allowing users to reformulate what they found and link it to their current knowledge. Functions to create knowledge models and link documents found to the models are also necessary. Tools to organize documents found in the search were found to be very important, the organization structure can also often represent these knowledge models.

Users often represented their knowledge structures in simplified physical formats, such as an outline of topical areas and issues in a document to be written or an organizational structure for storing documents.

As mentioned above, attorneys in the study created many of these systems, from paper files to word processing documents with appropriate headings. Attorneys interviewed in the study developed elaborate paper-based research filing systems that they meticulously maintained and kept up-to-date. The organization structure of these files represented the structure of how they thought about legal areas in light of their practice area. The clusters changed time to time based on the tasks of the attorney and changes in the practice of the legal issue. This is a good indication of the usefulness of physically representing internal knowledge structures for document management purposes. However, this external representation can also be helpful in learning about an area through visual representation.

Assigning categories from the user's scheme to documents or document sections also serves as a kind of interpretation activity. Categories describe the user's knowledge of an area, usually the topical area of the information seeking. Selecting and assigning categories to results act as a kind of rephrasing of the information in the documents in terms of the pre-existing categories of the user describing the topical area or the problem/task.

These external representations can be used as starting point in searching, they can be built into pre-search notes or shopping cart organizations and then later applied to searching, and refined during the search. Often search results and knowledge gained from reading them lead information seekers to rearrange these representations to reflect their new knowledge. Providing a tool based on earlier activities that are updated as the search progresses can help users refine their knowledge structures about an area. Structure should be complemented with notes, annotations, verbal explanations, and links to search results in order to better represent the knowledge of the user in a reusable format.

Another tool used by attorneys was annotation. Often typed annotations and user notes serve as a behavioral counterpart to interpretation and mental model building. The notes can have a temporal role in helping the user form a correct mental model, by the end of which process they lose their significance and can be made inactive. In this sense flexible model building tools can support the interpretation steps of searching.

Participant 1: Typed annotations, I forget things all the time.
Interviewer: But you wouldn't keep it, you would keep it for a session but not...
Participant 1: Yes, I wouldn't keep it, just the way I work is, by the end of the session, I'll cement it into my thinking hopefully.

Representing a previous version of the mental model through organization schemes or annotations allows the user to reflect on the change to it, thus reinforcing learning. Sharing mental models is important, as it provides a process of sharing information among team members.

4.2 Knowledge sharing

4.2.1 Introduction

Sharing search histories emerged as an important need from the interviews and observations. Often legal practitioners mentioned recording information with the goal of sharing it with others.

Various reasons were mentioned for sharing, some of these were related to team coordination, task delegation and accountability, while others to shared learning, shared decision making, and knowledge transfer. This second group is of interest in this paper. The representations created while searching for and working with information were communicated to others to explain why certain decisions were made, to update others about progress made or information found, or to prompt potential new interpretations of information by others.

4.2.2 Current knowledge sharing practices

Currently attorneys and law librarians share search histories through email, printing and notes. The technology available to searchers influences whether and how searchers share information, as in the following example where a law librarian describes her method of delivery of search results:

"Halvorson: How do you capture and save information from the Internet? Botluk: It depends on whom I am doing it for. Often I just copy and paste into an email message, or just send them the uniform resource locator (URL) to look at. I usually do not save the research onto my hard drive."

(Halvorson, 1999)

Another way to share search history information in an intermediated setting is to publish complex queries for patrons to use. A variation of this occurs in intermediated search environments, where librarians or professional searchers carry out searches for end users; in this context it is important to record searches as patrons may return to retrieve the results again or with related information problems. This tool is used more and more in organizations in the form of request-tracking databases in libraries or information resources available on Intranets.

"Halvorson: Has the Internet had any effect on that initial stage? Best: No, but it will, once we get a significant number of research memoranda and other documents into an in-house electronic database. We are currently designing a research database that we can use through our firm's intranet. Once that is operational, one of the first things I would do is look there to see what we already have. Hopefully it will be user-friendly enough that the lawyers in the office will use it, too, and will look there before they even come to see me."

New and improved tools are needed to help users with this process. Collecting and preserving individual team members' knowledge and information records can help support organizational memory representations. A database of previous searches and results can serve as the organization's memory in relation to information gathering. The form of this organizational memory can vary based on the tasks. Interface tools to support information and knowledge sharing will be described in more detail.

4.2.3 Collaborative learning, decision making, and training

Search histories can be shared in order to facilitate collaborative learning and decision making. Often it is necessary for the whole team to be aware of new information discovered by one team member. Sharing search histories can help the responsible team member to explain and share ideas and findings.

Using a record of previous searches is a good way to teach searching. In a team situation a senior member can share a query or a series of queries with junior colleagues in order to inform them about sources and search strategies. Librarians may use the same tactic to train attorneys. Search histories are also used to help diagnose existing problems in searches, and thus train through a history-supported help system.

Interviewer: So here you can go back to Lexis, when you go back, would you like something there that's personalized?
Participant 5: [ ... If] you want to show a search that this is something you can do or this is a search I have done to help others learn, then yeah, I can see it would be helpful.

One of the responsibilities of law firm librarians is to keep attorneys informed about sources and legal information seeking in general. Creating checklists and pathfinders based on previous searches is an indirect way to use a record of previous searches in training and leading people in future searches. Publishing complex queries related to important areas of the law that the firm deals with is another way. Attorneys can run these queries periodically to keep themselves up-to-date on an area. A law librarian in the next quote describes the creation of checklists for specific types of information requests that librarians can use:

"Halvorson: Do you use a checklist to remind yourself of places to look? Chick: I've started to make checklists for company information and expert witness information. We get asked that kind of thing a lot and it's easy to forget a good source. I have a paraprofessional on staff and it probably would help him, too."

Senior attorneys checking junior attorneys' work also has a training effect.

A database of previous searches and results can serve as the organization's memory in relation to information gathering. This is very important in large organizations or teams, where members may change over time, but their knowledge needs to be captured for future use on the project.

5 Summary and Conclusion

This study examined current legal information-seeking and use behavior in order to learn about the role of interaction histories in mental model building and sharing knowledge. Based on the results of the user behavior study, prototype user interfaces were designed through an iterative participatory design process, involving legal professionals. The results showed that automatically and manually recorded interaction histories are beneficial in helping users to find, learn, and use information.

Further investigation is needed into the impact of these tools on legal information users' information seeking and use activities.

References

[1] Komlodi, Anita. 2002. Search history for user support in information-seeking interfaces. PhD. dissertation, University of Maryland.

[2] Sutton, S. A. (1994). "The role of attorney mental models of law in case relevance determinations: An exploratory analysis." Journal of the American Society for Information Science 45(3): 186-200.

[3] Marshall, C. C. P., Morgan N.; Golovchinsky, Gene; Schilit, Bill. N. (2001). Designing e-books for legal research. Joint Conference on Digital Libraries (ACM and IEEE-CS), Roanoke, VA, ACM.

[4] Halvorson, T. R. (1999). Law of the super searchers: The online secrets of top legal researchers. Cyberage Books, 1999. 360 pp.