The Role of Interaction Histories in Mental Model Building
and Knowledge Sharing in the Legal Domain
Anita Komlodi
(UMBC, Baltimore, MD, USA
komlodi@umbc.edu)
Abstract: This paper reports on a study examining attorneys'
and law librarians' use of their memory and information they record externally
in searching for, using, and sharing legal information. The paper suggests
automatically and manually recording search histories and basing user interface
tools on this information to support mental model building and knowledge
sharing in the legal information domain. The research described is part
of the author's dissertation research [1] that examined
the use of search histories in legal information seeking and use, and proposed
interface design recommendations for information systems. While searching
for and using information, attorneys learn about legal topics and use this
knowledge in their work. They create mental models and share their new
knowledge with colleagues. Computers can automatically record human-computer
interaction events. This information can help searchers represent and share
new knowledge. The recorded information can be provided back to the user
through the user interface to support searching for and using information,
learning about the subject matter and sharing this knowledge with others.
In this study, attorneys and law librarians were interviewed and observed
to assess their use of their memory and external memory aids while searching
for and using legal information. The results reported here focus on the
role of interaction histories and history-based interface tools in supporting
mental model development of legal information seekers of a topical area
and sharing this information with other users.
Keywords: Search Histories, Information Retrieval, Legal Informatics
Categories: SD D2.2.2, H.3.3
1 Introduction
The research reported in this paper examined the information-seeking
behavior of attorneys and law librarians in the American legal system with
special attention to their internal memory use and external history-keeping
practices. Attorneys and law librarians in the U.S. legal system search
for and use legal information stored in online databases on a day-to-day
basis. Precedent-based legal practice requires legal professionals to refer
to court opinions to establish the current interpretation of the law. The
precedent-based legal domain is highly information and knowledge-intensive.
Legal litigation work is highly dependent on the proper identification,
retrieval, and management of up-to-date information. Legal information
is historically based, it develops over time and builds on previous information:
in the American legal system, the law is interpreted through court opinions
written by judges in courtrooms. These court opinions often cite other
previous court opinions and legal cases, strongly building on earlier information.
Keeping up-to-date with new developments and locating relevant information
is a professional and ethical responsibility of attorneys. Helping attorneys
with searching for, finding, managing, and using this information is a
challenge for information and knowledge management systems.
Search histories and search-history-based user interface tools are proposed
to answer this challenge. Search histories are computer-user interaction
histories automatically recorded while attorneys and law librarians search
for legal information in computer-based databases. Search histories are
sometimes called interaction histories in this paper. They contain a record
of user steps, systems responses, documents, and other information in a
temporal order.
The attorneys and law librarians in the study used the Westlaw system
(http://www.westlaw.com) to search
for information. The Westlaw system currently has nearly 15,000 databases
containing legal, business, and general information. Attorneys search for
information in these databases through formulating queries, examining result
sets, selecting and reading documents. Attorneys' use of the information
usually coincides with their information search; the two activities are
often carried out in parallel. Using information involves reading and interpreting
information returned to users in the form of court opinions and other documents.
Relating the new information to the attorney's task at hand, deciding how
and what to use the information for, and finally using the information
in new documents are all defined as information use in this paper. While
searching for, interpreting, and using this information, the user's interaction
history can be recorded in computerized environments and fed back to him
through the user interface to support individual learning and knowledge
sharing. This interaction history contains user steps, system responses,
documents, and other information. In addition to providing a direct display
of search histories, other information management tools can be developed,
such as functions to support the collection, processing, and sharing of
information.
Attorneys and law librarians were observed while searching for information
and interviewed afterwards to study user behavior and investigate user
needs. The data collected informed the design of prototype user interfaces
to present search history information, attorneys and law librarians were
also involved in the interface design process. Research results revealed
that recording users' steps automatically and allowing them to record their
interpretations and thoughts manually through notes and annotations are
useful features of information systems that can help users learn and share
their knowledge with others. The paper will first review related research,
then describe the methodology used, and finally discuss the results related
to learning and knowledge sharing. For a more detailed review of the results
please see [1].
2 Related Research
Sutton [2] described attorneys' mental model building
while searching for information. He criticizes earlier studies for defining
relevance as pure topicality. He steps back and defines relevance for legal
research as a first step in evaluating legal information. He characterizes
relevance as a function of the mental models or conceptual map of the law
constructed and maintained by attorneys:
Sutton's definition of relevance in the legal information field builds
on the event space of the case, placing other similar cases in this space.
Sutton describes legal practitioners' cognitive maps of the law as having
three levels:
- base-level modeling of the contours of the event space;
- context sensitive exploration of the space and the populating of the
relevant subsector;
- disambiguation of the subsequent model.
He also describes these stages in terms of information-seeking activities,
sources, and tools used. The first level is often accomplished through
training, learning about the general issues of an area of the law. The
second level, context-sensitive exploration@, focuses on a particular issue,
how the legal principle has been applied to the facts of the reported case.
The third level describes the process of disambiguation among the results
retrieved by the attorney's searches. In this process, cases in the event
space are evaluated based on their juristic status and treated according
to their status. If their jurisdiction does not require their use in the
attorney's litigation, they are removed or their influence diminished.
The cases are Shepardized (their subsequent history checked for negative
and positive changes in interpreting the law and the decision) in order
to examine their current status and the results of this are also taken
into consideration when deciding the impact of the cases. At the end of
this process the mental model is finalized for the time being. Sutton remarks
that the three processes are going on in parallel in real-life information-seeking
situations. The mental model building described in the current paper focuses
on the second and third levels, in which the attorney starts out with a
picture of the legal area, explores the conceptual space with the help
of this model, updates the model and then uses it to judge new results
and updates it with new information found. Recorded search histories can
help the development of these mental models through various tools and can
represent them in external forms. A list of documents and text sections
looked at, organized, and annotated by an attorney can later remind him
of his thinking, support learning, and can represent new knowledge to other
collaborators. The results section describes user's second- and third-level
mental model building that can be supported by search histories. The next
study reviewed also studied legal information seeking and use, although
focused more on the physical actions of users.
Marshall et al. [3] described a study of law students
preparing for a Moot Court competition where they practice case litigation
against other student teams. Their findings mirror the findings of this
study in many respects. Especially interesting here are the functions needed
to support learning and information use by law students based on recording
students' actions, and allowing students to annotate and organize information.
They examined the students' information-seeking, reading, writing, and
learning activities, including annotation techniques, in order to test
and design an e-book technology, XLibris.
Marshall et al. [3] found that the ability to annotate
and organize information is very important in supporting legal work, pointing
to the need for interface functions supporting these. Captured annotations
and organization schemes can be used for learning and sharing information
with others. Marshall et al. point out that annotation techniques are taught
to law students, and often different annotations are used to prepare a
document for different purposes. Re-reading and re-annotations are frequent
in the legal field; often annotations are overwritten or selected annotations
are marked for a second time.
Annotations vary in importance and usefulness. Students also often created
reminders and plans through annotations on document printouts that they
later used to guide their further research. These annotations can serve
as potential mental model representations of the students' interpretation
of the information and they also serve as drafts for documents, along with
the user's organizational scheme.
Documents collected for the Moot Court trial are organized according
to the tasks and purposes they will be used for. At the same time, as students
get closer to writing documents of their own, their organization schemes
become closer and closer to their writing objectives. Organization schemes
were activity-based and changed through the process of working with the
documents, reorganizing information was a way to conduct work. The authors
suggested flexible organization tools that allow reorganization easily.
The redesign suggestions for XLibris focused on the following areas:
navigation, retrieval, annotation, and organization, the last two being
of most interest here. Improvements to the annotation tools included the
ability to easily re-annotate previously marked documents and also the
ability to annotate previous annotations without having to go back to the
original document, thus supporting thinking. The authors also added a notebook
feature where users could collect clippings annotations. More flexible
organization tools were also added to the original design: workspace labeling
and divider pages were introduced. Both the annotation and the organization
tools and other information processing tools can be helpful to users to
learn about a topical area and build mental models.
3 Methodology
Initial exploration of the topic was carried out through a literature
review, interviews with reference librarians in a special library, and
an analysis of usability testing videos of the Westlaw legal information
system. The conclusion of these preliminary data collection effort was
that search histories can enhance information seeking and current tools
need improvements to satisfy user needs. Based on these findings, an iterative
methodology was designed including user behavior data collection, search
history framework development, and interface design. Attorneys and law
librarians were observed while searching and interviewed afterwards to
learn about their memory use while searching for and using information.
Eight participants were involved in the observations and interviews conducted
by the author. Additionally, eight interviews from [4]
were generously made available for this study by T. R. Halvorson. Participants
in the first group were asked to search the Westlaw databases for a topic
of their choice. The problem had to be subject-oriented and involve several
linked questions. While searching, participants were asked to think aloud.
After the searching, they were interviewed about the search session and
about their memory and history use in general. In the Halvorson interviews,
participants were interviewed about their information-seeking practice,
many of which involved the use of history mechanisms. Transcripts of think-aloud
sessions, interviews and observation notes were analyzed and the results
led to theoretical framework development and the design of interfaces.
These were used in the participatory design sessions, and further interfaces
were also designed by participants. The transcripts from these sessions
were also analyzed and they informed the theoretical framework.
Prototype interfaces were developed through several iterations with
intensive input from participants. These were critiqued by several groups
of legal professionals, however they were not formally tested. Formal evaluations
of the usefulness and impact of these interfaces are planned for the next
phase of the research.
4 Results and Discussion
The results indicated that legal information users use their memory
and automatically and manually recorded interaction history information
to support their information seeking and learning. In specific, the use
of interaction histories and history-based tools in mental model building
and knowledge sharing are described here. The involved user needs assessment
and user behavior study resulted in the design of user interface tools.
As mentioned above, although these tools were demonstrated to study participants
and their reactions were collected and applied, formal evaluation studies
are part of future research plans.
The first section (4.1) describes individual user's
mental model development through external representations of his mental
model, information organization, and annotation tools and actions. The
second section (4.2) builds on these external representations
to discuss sharing information and knowledge with others.
4.1 Mental model building
4.1.1 Introduction
A user's mental model is a cognitive model of the topical area the searcher
builds while looking for information, it is in a sense the final outcome
of the searching phase, the result of interpretation and the first step
of using the information. Building a mental model of a legal topical area
is tightly related to the interpretation of information found while searching.
By integrating new information into the knowledge structures of the user,
it becomes available for reuse in future work. This section describes how
automatically and manually recorded interaction histories can help externally
represent the user's new knowledge, experience, and mental model. It then
discusses the user's mental model building activities that can be supported
by user interface tools building on search histories, especially information
organization and annotation. These activities are integral part of an attorney's
work and they create external knowledge representations that can be stored
and shared with others.
Mental models are one way to think about knowledge representation in
memory. Mental models are cognitive representations of legal topical areas
that constantly change in light of new information encountered by the legal
professional. Searching stems from an information problem, a gap or other
discrepancy in the user's knowledge (and knowledge representation). Interpretation
during searching aims at applying new information found to the original
state of the user's knowledge and repair the discrepancy. Interpretation
and mental model building involve reading the documents found, often rephrasing
them in some form, assigning meaning to them through linkages to current
knowledge, and recording the process and the results in order to build
them into the knowledge network and mental model of the searcher. Interpretation
builds the user's mental model and is often reflected in the interaction
history of the user.
4.1.2 Interaction histories as mental model representations
A record of what information the user encountered and how she reacted
to it can help in externalizing users' knowledge models of an area. Recording
search histories and allowing users to manipulate them can help with:
- recording and preserving results of interpretation,
- the heavy processing of information helps transfer it from working
memory to LTM, and integrate it with pre-existing knowledge structures,
thus building a mental model.
Allowing users to view and manage their interaction histories makes
them aware of their progress and the information they found. It can help
them learn new information better by being exposed to it longer, and can
also help with planning and monitoring the search process. Explicitly presenting
the history display as it is being built makes users aware of the recording
and can lower privacy concerns.
The process of interpretation and learning about a topical area is cumulative,
past knowledge forms the basis of handling new information. Interpreting
and integrating search results with old knowledge will eventually lead
to the answer; however without recording the results of integrating new
knowledge into old models, the answer may be lost by the end of the process.
Searchers can try to remember all the information found, but this may be
difficult with only one exposure to it and lack of manipulation. Keeping
track of this process in electronic environments through history-supported
tools is an obvious application area of search histories. Recording results
is a good foundation for recording their interpretation by the user and
their linking to current knowledge.
4.1.3 Support for user mental model building
Interpretation leads to the development of mental models through the
integration of new information with the previous knowledge of users. Attorneys
read documents, took notes, printed documents, annotated them, organized
and reorganized them while searching for and interpreting information in
the study. These actions helped them make sense of the information, learn
it, and prepare it for future use. Providing tools for these actions and
recording their results over time can help searchers build mental models
and learn, as discussed below. External representations of mental models
can be helpful this process, participants often described processes to
record these.
Tracking the interpretation carried out by searchers is important but
challenging. Recording the thinking of searchers is a more complex task
than recording their action, it can only be recorded if the user explicitly
enters written/typed notes, annotations or voice recordings. Systems should
provide tools for entering notes and annotations linked to interaction
histories recorded automatically; these can help record interpretation
and the user's thinking. They help interpretation by allowing users to
reformulate what they found and link it to their current knowledge. Functions
to create knowledge models and link documents found to the models are also
necessary. Tools to organize documents found in the search were found to
be very important, the organization structure can also often represent
these knowledge models.
Users often represented their knowledge structures in simplified physical
formats, such as an outline of topical areas and issues in a document to
be written or an organizational structure for storing documents.
As mentioned above, attorneys in the study created many of these systems,
from paper files to word processing documents with appropriate headings.
Attorneys interviewed in the study developed elaborate paper-based research
filing systems that they meticulously maintained and kept up-to-date. The
organization structure of these files represented the structure of how
they thought about legal areas in light of their practice area. The clusters
changed time to time based on the tasks of the attorney and changes in
the practice of the legal issue. This is a good indication of the usefulness
of physically representing internal knowledge structures for document management
purposes. However, this external representation can also be helpful in
learning about an area through visual representation.
Assigning categories from the user's scheme to documents or document
sections also serves as a kind of interpretation activity. Categories describe
the user's knowledge of an area, usually the topical area of the information
seeking. Selecting and assigning categories to results act as a kind of
rephrasing of the information in the documents in terms of the pre-existing
categories of the user describing the topical area or the problem/task.
These external representations can be used as starting point in searching,
they can be built into pre-search notes or shopping cart organizations
and then later applied to searching, and refined during the search. Often
search results and knowledge gained from reading them lead information
seekers to rearrange these representations to reflect their new knowledge.
Providing a tool based on earlier activities that are updated as the search
progresses can help users refine their knowledge structures about an area.
Structure should be complemented with notes, annotations, verbal explanations,
and links to search results in order to better represent the knowledge
of the user in a reusable format.
Another tool used by attorneys was annotation. Often typed annotations
and user notes serve as a behavioral counterpart to interpretation and
mental model building. The notes can have a temporal role in helping the
user form a correct mental model, by the end of which process they lose
their significance and can be made inactive. In this sense flexible model
building tools can support the interpretation steps of searching.
Participant 1: Typed annotations, I forget things all the time.
Interviewer: But you wouldn't keep it, you would keep it for a session
but not...
Participant 1: Yes, I wouldn't keep it, just the way I work is, by the
end of the session, I'll cement it into my thinking hopefully.
Representing a previous version of the mental model through organization
schemes or annotations allows the user to reflect on the change to it,
thus reinforcing learning. Sharing mental models is important, as it provides
a process of sharing information among team members.
4.2 Knowledge sharing
4.2.1 Introduction
Sharing search histories emerged as an important need from the interviews
and observations. Often legal practitioners mentioned recording information
with the goal of sharing it with others.
Various reasons were mentioned for sharing, some of these were related
to team coordination, task delegation and accountability, while others
to shared learning, shared decision making, and knowledge transfer. This
second group is of interest in this paper. The representations created
while searching for and working with information were communicated to others
to explain why certain decisions were made, to update others about progress
made or information found, or to prompt potential new interpretations of
information by others.
4.2.2 Current knowledge sharing practices
Currently attorneys and law librarians share search histories through
email, printing and notes. The technology available to searchers influences
whether and how searchers share information, as in the following example
where a law librarian describes her method of delivery of search results:
"Halvorson: How do you capture and save information from the Internet?
Botluk: It depends on whom I am doing it for. Often I just copy and paste
into an email message, or just send them the uniform resource locator (URL)
to look at. I usually do not save the research onto my hard drive."
Another way to share search history information in an intermediated
setting is to publish complex queries for patrons to use. A variation of
this occurs in intermediated search environments, where librarians or professional
searchers carry out searches for end users; in this context it is important
to record searches as patrons may return to retrieve the results again
or with related information problems. This tool is used more and more in
organizations in the form of request-tracking databases in libraries or
information resources available on Intranets.
"Halvorson: Has the Internet had any effect on that initial stage?
Best: No, but it will, once we get a significant number of research memoranda
and other documents into an in-house electronic database. We are currently
designing a research database that we can use through our firm's intranet.
Once that is operational, one of the first things I would do is look there
to see what we already have. Hopefully it will be user-friendly enough
that the lawyers in the office will use it, too, and will look there before
they even come to see me."
New and improved tools are needed to help users with this process. Collecting
and preserving individual team members' knowledge and information records
can help support organizational memory representations. A database of previous
searches and results can serve as the organization's memory in relation
to information gathering. The form of this organizational memory can vary
based on the tasks. Interface tools to support information and knowledge
sharing will be described in more detail.
4.2.3 Collaborative learning, decision making, and training
Search histories can be shared in order to facilitate collaborative
learning and decision making. Often it is necessary for the whole team
to be aware of new information discovered by one team member. Sharing search
histories can help the responsible team member to explain and share ideas
and findings.
Using a record of previous searches is a good way to teach searching.
In a team situation a senior member can share a query or a series of queries
with junior colleagues in order to inform them about sources and search
strategies. Librarians may use the same tactic to train attorneys. Search
histories are also used to help diagnose existing problems in searches,
and thus train through a history-supported help system.
Interviewer: So here you can go back to Lexis, when you go back, would
you like something there that's personalized?
Participant 5: [ ... If] you want to show a search that this is something
you can do or this is a search I have done to help others learn, then yeah,
I can see it would be helpful.
One of the responsibilities of law firm librarians is to keep attorneys
informed about sources and legal information seeking in general. Creating
checklists and pathfinders based on previous searches is an indirect way
to use a record of previous searches in training and leading people in
future searches. Publishing complex queries related to important areas
of the law that the firm deals with is another way. Attorneys can run these
queries periodically to keep themselves up-to-date on an area. A law librarian
in the next quote describes the creation of checklists for specific types
of information requests that librarians can use:
"Halvorson: Do you use a checklist to remind yourself of places
to look? Chick: I've started to make checklists for company information
and expert witness information. We get asked that kind of thing a lot and
it's easy to forget a good source. I have a paraprofessional on staff and
it probably would help him, too."
Senior attorneys checking junior attorneys' work also has a training
effect.
A database of previous searches and results can serve as the organization's
memory in relation to information gathering. This is very important in
large organizations or teams, where members may change over time, but their
knowledge needs to be captured for future use on the project.
5 Summary and Conclusion
This study examined current legal information-seeking and use behavior
in order to learn about the role of interaction histories in mental model
building and sharing knowledge. Based on the results of the user behavior
study, prototype user interfaces were designed through an iterative participatory
design process, involving legal professionals. The results showed that
automatically and manually recorded interaction histories are beneficial
in helping users to find, learn, and use information.
Further investigation is needed into the impact of these tools on
legal information users' information seeking and use activities.
References
[1] Komlodi, Anita. 2002. Search history for user
support in information-seeking interfaces. PhD. dissertation, University
of Maryland.
[2] Sutton, S. A. (1994). "The role of attorney
mental models of law in case relevance determinations: An exploratory analysis."
Journal of the American Society for Information Science 45(3): 186-200.
[3] Marshall, C. C. P., Morgan N.; Golovchinsky,
Gene; Schilit, Bill. N. (2001). Designing e-books for legal research. Joint
Conference on Digital Libraries (ACM and IEEE-CS), Roanoke, VA, ACM.
[4] Halvorson, T. R. (1999). Law of the super searchers:
The online secrets of top legal researchers. Cyberage Books, 1999. 360
pp.
|