Digital Libraries/Online info seeking behavior and search strategy

• Older versions of the draft developed by UNC/VT Project Team (02-08-2007 PDF WORD)

Module name

Online Information Seeking Behaviors & Search Strategies

Scope

This module covers the theories, models, and practices related to online information seeking behaviors in different settings of digital libraries.

Learning objectives:

By the end of this module, the student will be able to:

a. Identify the fundamental concepts, definitions, and search strategies in online information seeking, as they apply to digital libraries.

b. Apply the models of online information seeking behaviors in explaining the user behaviors that have been identified in empirical studies of digital libraries.

c. Utilize practical methods (e.g., transaction log analysis, interviews, think-aloud protocols) for characterizing information seeking behaviors in digital libraries

5S characteristics of the module:

a. Stream, Structure, Space, & Society: N/A

b. Scenarios: People look for information in digital libraries with their own histories or scenarios of information seeking.

Level of effort required:

a. Class time: 3 hours

i. Session 1 (1 1/2 hrs): Introduction to information seeking behavior models

• Lecture on theories of online information seeking behaviors
• In-class activity (See, 12. Exercises/Learning Activities, a. In class Exercise, below)

ii. Session 2 (1 1/2 hrs) : Empirical studies of the development of search strategies in the context of digital libraries

b. Course preparation for students: 2 hours / session

i. Mostly associated with readings (See the reading list for students in 11. Resources, below)

c. Course assignment completion: 2 hours

i. Conduct either a user data collection practice or a think-aloud protocol practice (See 12. Exercises/Learning Activities, b. Homework Assignment, below)

Relationships with other modules:

a. 6-a: Info needs, relevance

6-a covers the inner state of people's minds and information needs when they are about to start information seeking, and how that state evolves during their browsing and searching for information. 6-b is different from this module in that it deals with the behavioral patterns and the characteristics of users when searching for information in digital libraries.

b. 6-d: Interaction Design, Info Summarization and Visualization, Usability Assessment

6-d identifies theoretical and practical knowledge related to the design and development of the system interfaces of digital libraries in a general context, while 6-a provides strategies and directions for how to search for information in digital libraries.

Prerequisite knowledge required:

a. No prerequisite courses are required.

b. Taking a research method course prior to this course may help students to deal with the interviews in their homework assignments, but the instructor will provide basic guideline for the interviews for the students who don't have experiences with the method.

c. No technical programming or any other skills are necessary.

Introductory remedial instruction:

a. None

Body of knowledge:

1. Basic Concepts & Definitions (Case, 2002)

a. Information need:

i. A motivational state in people's minds that causes thoughts and actions through which they obtain something they desire.

ii. A state in which a person recognizes that his/her knowledge is inadequate to satisfy a goal that he/she has.

b. Information seeking / Information seeking behaviors

i. Actions that people do in response to their information needs

ii. Intentional activities to satisfy information needs

iii. What people do in response to information needs/tasks/problems

• "The purposive seeking for information as a consequence of a need to satisfy some goal" (Wilson, 2000)

iv. A process of discovering patterns or filling in gaps in patterns

• "A process in which humans purposefully engage in order to change their state of knowledge" (Marchionini, 1995)

c. Information Browsing, Searching & Seeking

i. Terms used interchangeably in literature

ii. Need to understand the appropriate meaning in the context of literature

iii. General meanings

• Browsing: Informal and opportunistic behaviors moving across documents, examining them closely to see whether they satisfy the information needs or not
• Searching: The actions taken when trying to find something

The scope of meaning, sometimes, can be narrowed as being tied to specific searching techniques (e.g. Keyword-searching which enables people to retrieve information by inputting keywords in search engines)

• Seeking: While browsing and searching are somewhat focused on behaviors or actions of people, information seeking covers more general categories and the broad nature of cognitive, perceptual, behavioral, and the environmental perspectives of people who are seeking information.

2. Online Information Seeking Behavior Models

a. Information Seeking in Online Environments

i. Allen's Information Tasks (1996)

• Scanning: Scanning tasks, resources, systems and acquiring general information
• Reviewing and Evaluating: Filtering information based on personal criteria (e.g. relevance judgments)
• Learning: Solving problems based on collected information and creating new knowledge
• Planning: Constructing search plans, evolving and developing them until the searches end

ii. Marchionini's Browsing Strategies (1999)

• Scanning: Linear and/or selective scanning
• Observation: Capturing scenes, and making sense of what people see
• Navigation: Having specific goals or objectives when browsing, and developing decision-making progress while organizing and structuring new knowledge
• Monitoring: Casual scanning of environments in order to sense what's in the user's area of interest

iii. Choo, Detlor & Turnbull's Information Seeking on the Web (2000)

• Starting: Identifying sources of information and initiating the search
• Changing: Modifying and re-directing the search process in order to locate information
• Browsing: Scanning resources to understand the contents
• Differentiating: Evaluating and selecting appropriate resources
• Monitoring: Reviewing core sources to be acknowledged by news or changing conditions
• Extracting: Working on a selected source to identify whether it is the most appropriate source and to find connections to other related sources

Model Summary

Summary	Allen (1996 )	Marchionini (1999)	Choo, et al. (2000)
Initiating with information needs to solve problems, or make decisions	Information Needs	Information Needs	Information Needs
(1) Scanning, learning about resources (2) Building general to specific ideas of what resources fit with information needs	Scanning	Scanning, Observation	Starting, Changing, Browsing
Narrowing down the scope of resources to what is really appropriate	Reviewing & Evaluating	Navigation	Differentiating Extracting
Filtering out too broad, less related resources to information needs which evolve during the browsing.	Reviewing & Evaluating	Monitoring	Monitoring
Knowledge building	Learning	-	-
Continues development of information seeking	Planning	-	-

• Allen (1996)

1. Describing the overall process of information seeking from information seeking to knowledge development

2. Addressing the continuous nature of information seeking

• Marchionini (1999)

1. Specifying processes of what people see and capture in their minds

• Choo, et al (2000)

1. Focusing on how people deal with, manage and evaluate resources

iv. Common points

• Models illustrate linear sequences of information seeking, although there are loops and repetitive actions.
• It is assumed that people develop their search from broader topics to narrower topics.
• Information needs evolve during the search.
• People continually evaluate information based on the judgments they make during their search.
• People's knowledge structures evolve during the search.
• Although it is hard to identify the definitive end of a search which indicates that the search is completed, people have strategies when closing their current search session.

b. Search Strategy Development in Online Environments

i. Chowdhury & Chowdhury's the Four-Phase Framework for Information Search in digital libraries (2003)

1. Phase 1: Formulation:

• Triggered by an information need
• Early decision-making stages regarding what to search for and the search variants

2. Phase 2: Action

• Executing information searching with queries
• Associated with search queries and strategies development

3. Phase 3: Review of Results

• Utilizing various tools to display the search results (e.g. sorting, highlighting)

4. Phase 4: Refinement

• Modifying and reformulating search queries, conducting new searches

ii. Marchinionini's Analytical Search Strategies (1999)

1. Building blocks

• Starting searching with basic concepts and then expanding the scope of the search results by combining additional concepts from the search results obtained during the searches

2. Successive fractions

• Using a large subset of the entire database and breaking it down into pieces of concepts relevant to a problem while conducting the searches

3. Pearl growing

• Selecting a base document (a pearl) and using its characteristics such as index terms, title, text, citations, to extract and formulate search terms in order to locate documents relevant to the original information needs

4. Interactive scanning

• Beginning the search with a large set of documents generally relevant to the problem and scanning them to extract their key common features and using them to formulate queries

3. Case Studies of the Search Strategy Development in Digital Libraries

a. Case Study 1: Information seeking behaviors in databases in digital libraries

Park, S. (2000) Usability, User Preferences, Effectiveness, and User Behaviors When Searching Individual and Integrated Full-Text Databases: Implications for Digital Libraries. Journal of the American Society for Information Science, 51(5), 456-68.

i. Research Purpose

To compare the user characteristics in information seeking, such as usability, user preference, effectiveness and searching behaviors, towards the use of different types (integrated levels) of interfaces of a digital library with multiple databases, focusing on how people interact with multiple heterogeneous information resources.

ii. Research Questions

• User preferences: Which system did you like better, find more useful, easier to use, and easier to learn to use?
• Usability & User satisfaction: How easy is it to do the search on the topic? How satisfied are you with your search results? Did you have enough time to do an effective search?
• Effectiveness: Is there a statistically significant difference in performance (aspectual recall) between systems?
• Search behaviors: Is there a statistically significant difference in individual's searching behavior (e.g. the usage of frequency of various system features) between systems?

iii. Methods

• Text Retrieval Conference (TREC) resource use for searches

(Topics of resources & search tasks: Federal Register, Wall Street Journal, Financial Times, & Congressional Record)

• Experimental system interfaces

HERA: An integrated interface with multiple databases

HERMIS: Common interface, individual interface access to multiple databases

• Participants: 28 graduate students
• Conducting searching experiments

(Each participant conduct 3 searches with given tasks on each system)

• Post system and post search questionnaire completion
• Interviews

iv. Findings

• Searcher responses to each system regarding advantages and disadvantages to use
• User preference to interacting with different databases through a common interface than interacting with databases through an integrated interface.
• User desires to control database selection, system features, relevance feedback

b. Case Study 2: Scholars and professionals' information use in digital libraries I

Jones, S., Cunningham, S. J., McNab, R, & Boddie, S. (2000). A Transaction Log Analysis of a Digital Library. International Journal on Digital Libraries, 3, 152-169.

i. Research Purpose

To investigate "the information on user behavior" in the Computer Science Technical Reports collection of the New Zealand Digital library, from analyzing the user log data "both automatically (through calculation of summary statistics) and manually (by examining query strings for semantic clues on search motivations and search strategies)"

ii. Methods

1. Quantitative and qualitative analysis of transaction logs

2. Target digital library: The New Zealand Digital Library (NZDL)

3. Target collection: The Computer Science Technical Reports collection (CSTR)

• 46,000 publicly available computing-related technical reports from 300 research institutes around the world.

3. Data Collection

• User log data with 30,000 queries in NZDL, automatically collected during 61 weeks from April 1996-July 1997.

iii. Findings

1. Quantitative statistics about query use.

• Usage in default query settings
• Query complexity
• Query terms
• Term specificity
• "Failed" and "erroneous" queries
• User sessions
• Query refinement
• Result viewing

2. Qualitative analysis about the search strategies

• Spelling issues
• Sub-collection choice
• Additional search strategies
• "Graffiti" queries
• Boolean queries

Resources

Reading List for Students

1. Session 1: Required

i. Allen, B. L. (1996). Chapter 7, Information Tasks: Interacting with Information Systems, In Information Tasks: Toward a User-Centered Approach to Information Systems (pp.188-200). San Diego: Academic Press

ii. Choo, W. C., Deltor, B., & Turnbull, D. (2000). Information Seeking on the Web: An Integrated Model of Browsing and Searching. First Monday, 5 (2).

iii. Chowdhury, G. G. & Chowdhury, S. (2003). Chapter 8, Information Access and User Interfaces and Chapter 9, Information Retrieval in Digital Libraries. In Introduction to Digital Libraries (pp.152-213), London: Facet Publishing.

iv. Marchionini, G. (1999). Chapter 6, Browsing Strategies. In Information Seeking in Electronic Environments (pp.100-138). NY: Cambridge University Press.

2. Recommended

i. Borgman, C. L. (2000). Chapter 5, Why are Digital Libraries Hard to Use. In From Gutenberg to the global information infrastructure (pp. 117-141). Cambridge, MA: MIT Press Buttenfield

ii. Case, D. O. (2002). Looking for Information: A Survey of Research on Information Seeking, Needs, and Behavior. New York: Academic Press.

iii. Chang, S-J & Rice, R. E. (1993). Browsing: A Multidimensional Framework. In M.E. Williams (Ed.), Annual Review of Information Science and Technology (pp. 231-271), 28. Medford, NJ: Learned Information.

3. Session 2: Required (Case Studies)

i. Park, S. (2000) Usability, User Preferences, Effectiveness, and User Behaviors When Searching Individual and Integrated Full-Text Databases: Implications for Digital Libraries. Journal of the American Society for Information Science, 51(5), 456-68.

ii. Jones, S., Cunningham, S. J., McNab, R, & Boddie, S. (2000). A Transaction Log Analysis of a Digital Library. International Journal on Digital Libraries, 3, 152-169.

Readings for Instructors

1. General Theories and Models of Information Seeking Behaviors in Online Environments

i. Allen, B. L. (1996). Chapter 7, Information Tasks: Interacting with Information Systems. In Information Tasks: Toward a User-Centered Approach to Information Systems (pp.188-200). San Diego: Academic Press

ii. Case, D. O. (2006). Looking for Information: A Survey of Research on Information Seeking, Needs, and Behavior. New York: Academic Press.

iii. Case, D. O. (2007). Information Behavior. In B. Cronin (Ed), Annual Review of Information Science and Technology (pp. 293-327).

iv. Chang, S-J & Rice, R. E. (1993). Browsing: A Multidimensional Framework. In M.E. Williams (Ed.), Annual Review of Information Science and Technology (pp. 231-271), 28. Medford, NJ: Learned Information.

v. Choo, W. C., Deltor, B., & Turnbull, D. (2000). Information Seeking on the Web: An Integrated Model of Browsing and Searching. First Monday, 5 (2).

vi. Marchionini, G. (1995) Chapter 1, Information and Information Seeking. In Information Seeking in Electronic Environment (pp. 1-9). Cambridge, UK: Cambridge University Press.

vii. Marchionini, G. (1999). Chapter 6, Browsing Strategies. In Information Seeking in Electronic Environments (pp.100-138). NY: Cambridge University Press.

viii. Wilson, T. D. (2000). Human Information Behavior. Informing Science, 3(2), 49-56. Retrieved September 6, 2007 from: http://inform.nu/Articles/Vol3/v3n2p49-56.pdf

2. Research Methods for Information Seeking Behavior Studies

i. Wang, P. (1999). Methodologies and Methods for User Behavioral Research. In M.E. Williams (Ed.), Annual Review of Information Science and Technology (pp. 53-99), 34, Medford, NJ: Information Today

ii. Berg, B. L. (2004). Qualitative Research Methods for the Social Sciences. (5th ed). Boston: Pearson.

iii. Burgess, R. G. (Ed.) (1986). Chapter 11, Elements of Sampling in Field Research. In Field Research: A Sourcebook and Field Manual (pp. 75-78). UK: Routledge.

iv. Crutcher, R. (1994). Telling what we know: The use of verbal report methodologies in psychological research. Psychological Science. 5(5), 241-244.

v. Honigmann, J. J. (1986). Chapter 12, Sampling in Ethnographic Fieldwork. In Burgess, R. G. (Ed.), Field Research: A Sourcebook and Field Manual (pp. 79-90). UK: Routledge.

vi. Jansen, B. J. (2006) Search log analysis: What is it; what's been done; how to do it, Library and Information Science Research, 28(3), 407-432.

vii. Jones, S., Cunningham, S. J., McNab, R., & Boddie, S. (2000). A Transaction Log Analysis of a Digital Library. International Journal on Digital Libraries, 3, 152-169.

viii. Wang, P. (1999). Methodologies and methods for user behavior research. In Annual Review of Information Science and Technology (pp. 53-99).

ix. Van Someren, M. W., Barnard, Y., and Sandberg, J. (1994). The think aloud method - a practical approach to modeling cognitive processes, London: Academic Press.

3. Search Strategy Development in Online Environments

i. Aula, A., N. Jhaveri, and M. Käki. (2005) Information Search and Reaccess Strategies of Experienced Web Users. In Proceedings of the 14th International Conference on the World Wide Web (pp. 583-592), ACM Press.

ii. Debowski, S. (2002) Wrong Way: Go Back! An Exploration of Novice Search Behaviours While Conducting an Information Search. The Electronic Library, 19(6), 371-82.

iii. Holscher, C. & Strube, G. (2000). Web Search behavior of Internet Experts and Newbies. Computer Networks, 33(1-6), 337-346

iv. Kuhlthau, C. C. (1993). Chapter 3, The Information Search Process and Chapter 4, Verification of the Model of the Information Search Process. In Seeking Meaning: A Process Approach to Library and Information Services (pp.33-63). Norwood, New Jersey: Ablex Publishing Company.

v. Hsieh-Yee, I. (1993). Effects of Search Experience and Subject Knowledge on the Search Tactics of Novice and Experienced Searchers. Journal of the American Society for Information Science 44(3), 161-174.

vi. Marchionini, G., Dwiggins, S., Katz, A, and Lin, X. (1993). Information Seeking in Full-Text End-User-Oriented Search Systems: The Roles of Domain and Search Expertise. Library & Information Science Research, 15(1), 35-69.

4. Developing Searching and Browsing Strategies in the Context of Digital Libraries

i. Bates, M. J. (2002). The Cascade of Interactions in the Digital Library Interface. Information Processing & Management 38(3), 381-400.

ii. Bates, M. J. (1998). Indexing and Access for Digital Libraries and the Internet: Human, Database, and Domain Factors. Journal of the American Society for Information Science, 49(13), 1185-205.

iii. Bishop, A. P. & Star, S. L. (1996). Social informatics of digital library use and intrastrucutre. In M. Williams (Eds.), Annual Review of Information Science and Technology 31 (pp.301-401). Medford, NJ: Information Today, Inc.

iv. Bishop, A. P, Neumann, L. J., Star, S. L., Merkel, C., Ignacio, E. & Sandusky, R. J. (2000). Digital Libraries: Situating Use in Changing Information Infrastructure. Journal of the American Society for Information Science, 51(4), 394-413

v. Borgman, C. L. (2000). Chapter 5, Why are Digital Libraries Hard to Use In From Gutenberg to the Global Information Infrastructure (pp. 117-141). Cambridge, MA: MIT Press Buttenfield.

vi. Chowdhury, G. G. & Chowdhury, S. (2003). Chapter 8, Information Access and User Interfaces and Chapter 9, Information Retrieval in Digital Libraries. In Introduction to Digital Libraries (pp.152-213), London: Facet Publishing.

vii. Das Neves, F.A. & Fox, E. (2000). A Study of User Behavior in an Immersive Virtual Environment for Digital Libraries. In Proceedings of the Fifth ACM Conference on Digital Libraries, 103-111.

viii. Park, S. (2000) Usability, User Preferences, Effectiveness, and User Behaviors When Searching Individual and Integrated Full-Text Databases: Implications for Digital Libraries. Journal of the American Society for Information Science, 51(5), 456-68.

ix. Paepcke, A. et al. (2000). Search Middleware and the Simple Digital Library Interoperability Protocol. D-Lib Magazine, 6 (10).

x. Schatz, B. R. (1997). Information Retrieval in Digital Libraries: Bringing Search to the Net, Science, 275(5298), 327-334.

xi. Smith, A. G. (2000) Search Features of Digital Libraries. Information Research, 5(3).

xii. Spink, A., Wilson, T., Ellis, D., and Ford, N. (1998). Modeling Users' Successive Searches in Digital Environments: A National Science Foundation/British Library Funded Study. D-Lib Magazine, April.

xiii. Borgman, C. L., et. al. (2005). Comparing Faculty Information Seeking in Teaching and Research: Implications for the Design of Digital Libraries. Journal of the American Society for Information Science and Technology, 56(6), 636-657.

xiv. Payette, S. & Rieger, O. (1998). Supporting Scholarly Inquiry: Incorporating Users in the Design of the Digital Library. Journal of Academic Librarianship, 4(2), 121-129.

xv. Nicholas D, Huntington P, and Watkinson A. (2005). Scholarly Journal Usage: The Results of a Deep Log Analysis. Journal of Documentation. 61(2), 248 -280

Concept map

None

Exercises / Learning activities

A total of three exercises are introduced here. Instructors can choose either one or two of them and use them for the class according to the time allocated for the exercises.

a. In-class Exercise I: A Small Group Activity of the Transaction Log Analysis (See, Appendix A: Transaction Log Data Analysis Practice, as an exercise example)

i. The purpose of this exercise is to provide opportunities in which students look at a set of transaction log data from an information searching system and practice how to interpret and analyze the raw data and make them meaningful in understanding the user behaviors in searching.

ii. The students will be assigned to groups of 3 or 4. The instructor will assign a sample transaction log data set to each group. The sample data is collected at the Davis Library at the University of North Carolina at Chapel Hill on December 14th, 2002. 65 subjects participated in the study and log data from one subject is included here (Subject ID: x68). (Wildemuth, B. M, & Oh, S. (2006) Congruence between Information Known and Information Used in Conducting Known-Item Searches, Presentation at the Annual Research Forum of Librarian's Association of UNC-CH (LAUNC-CH), May 23.)

iv. All groups will have the same data set for the class discussion later. The instructor provides the background information about the original information system, at least about the types of information systems, how and when the transaction log was captured, and if necessary, how the interfaces of the information system look. No user information needs to be released by the instructor, since the students would go through an inference process of analyzing the user characteristics from the exercise.

v. Each group will have 20 minutes to discuss the data. While reviewing the logs, the students will be asked to list the entries that they could identify, to extract any information they analyzed, and any descriptions about user behaviors in using the information system in as much detail as they can.

vi. When the group discussion ends, the class will have a 10 to 20 minute session for a class discussion with the findings from the groups (See the Appendix A for the exercise example), and about the benefits and limitations in using the method.

vi. Here are examples of items that the student can capture from the transaction log data (Jones, 2000):

• User demographic information, if users are asked to provide their demographic information before starting searches
• Use of operators
• Search options of the queries
• Patterns in query construction and refinement
• Common mistakes in searching, like using incorrect spellings of the queries
• Distribution of query terms (e.g., popular terms, newly appearing terms, terms no longer in use, etc.)
• Frequency of query terms (e.g., ranking of the terms, the terms that the recent frequencies increase or decrease, etc.)

b. Homework Assignment: User Data Collection Practice

i. This exercise gives the students the opportunity of being able to meet users and investigate their information seeking behaviors while using a digital library.

ii. This is a group exercise. The students will be grouped into 3 or 4. First, each group will pick a real or hypothetical digital library in any context. Since they need to find practice users (at least 1 per each group members) who may be interested in using the digital library, it would be easier if they chose an existing digital library and have interviews with people who have experiences using it. On the other hand, they could imagine that they plan to build a digital library for a specific target group (e.g., children, students, health professionals, film makers, movie goers, etc) and interview the target group's information seeking behavior patterns.

iii. Interviews will be conducted with at least 1 people per each group members on how they search or would search in the selected digital libraries. The interview questionnaires will be designed by the students, but they will include lists of questions that collect data about the measurement criteria that was discussed in class.

iv. After the interviews, the students will analyze the interview results and evaluate whether the results would explain how people search for information.

v. The students will bring the interview results to the group in the next class session, discuss the issues that they found among group, share the results in the class discussion, and discuss about the benefits and limitations in using the method.

c. Homework Assignment II: Think-Aloud Protocol Practice

i. Students would have a chance to practice the method of think aloud protocols in investigating the use of a digital library.

ii. Two students will be grouped as exercise partners for each other: one as a user and one as a protocol facilitator. The instructor selects three or four types of digital libraries and prepares two information searching tasks for each digital library. For the assignment, the instructor lets each group choose one digital library and provides the prepared questions related to the chosen digital library to each group.

iii. Each group will find a quiet place with a computer, where they can launch the protocol exercise, and prepare a voice recorder (The recorder can be borrowed from the university media center or libraries) The protocol facilitator will ask the user to conduct the search tasks and think aloud describing what they are doing and thinking while they perform the tasks. During the searches, if the user stops talking, the facilitator will prompt the subject to continue. The conversation between the facilitator and the user during the protocol exercise needs to be recorded (The protocol needs to be conducted for less than 15 minutes)

iv. After the exercise, both students transcribe the records together, and analyze what they found about the digital library. They need to write a report including their experience with the method, benefits and drawbacks, difficulties, and limitations of the method as well as findings about digital library use.

Evaluation of learning outcomes

a. Do students know well the fundamental concepts, definitions and theoretical models of online information seeking behaviors?

b. Did students build their own knowledge of the search strategies and the challenges of information seeking in digital libraries?

c. Did students successfully conduct the user interviews through the homework exercise and collect appropriate user data?

d. Do students describe well how the theoretical models of information seeking behaviors are applicable in digital library settings based on the analysis of the user data they collected?

Glossary

No glossary terms needed.

Additional useful links

None

Contributors

a. Developers:

i. Sanghee Oh

b. Evaluators:

i. Jeff Pomerantz (UNC)

ii. Barbara Wildemuth (UNC)

iii. Ashwini Athavale (Indiana University)

iv. Stephanie Hass (UNC-CH)

v. Maureen Henninger (UTS, Australia)

vi. Lindley Shedd (Indiana University)