Evidence Summary
Facet Use in
Search Tools is Influenced by the Interface but Remains Difficult to Predict
A Review of:
Dahlen,
S. P. C., Haeger, H., Hanson, K., & Montellano, M. (2020). Almost in the wild: Student search
behaviors when librarians aren’t looking. Journal
of Academic Librarianship, 46(1),
102096. https://doi.org/10.1016/j.acalib.2019.102096
Reviewed by:
Scott
Goldstein
Coordinator,
Web Services & Library Technology
McGill
University Library
Montréal,
Québec, Canada
Email:
[email protected]
Received: 1 June 2020 Accepted: 22 July 2020
2020 Goldstein.
This is an Open Access article distributed under the terms of the Creative
Commons‐Attribution‐Noncommercial‐Share Alike License 4.0
International (http://creativecommons.org/licenses/by-nc-sa/4.0/),
which permits unrestricted use, distribution, and reproduction in any medium,
provided the original work is properly attributed, not used for commercial
purposes, and, if transformed, the resulting work is redistributed under the
same or similar license to this one.
DOI: 10.18438/eblip29790
Abstract
Objective –
To examine the relationship between student search behaviours and the quality
of scholarly sources chosen from among library search tools.
Design –
Unmonitored search sessions in a facilitated library setting.
Setting –
A mid-sized public university in the United States of America.
Subjects –
50 upper-level undergraduate students in the social and behavioural sciences.
Methods –
Recruited participants were given one of two search prompts and asked to use
EBSCO’s Social Science Abstracts and two configurations of ProQuest’s Summon,
with one being pre-scoped to exclude newspapers and include subject areas
within the social sciences. The search tools were assigned in random order. In
each case, the participant was asked to find two of the “best quality” articles
(p. 3). A librarian was present in the room but did not observe participants;
instead, all sessions were recorded using Camtasia Relay. Afterwards,
participants were interviewed about the process they used and their impressions
of the search tools. They also completed a survey collecting information on
their GPA and whether they had previously had library instruction.
Main Results –
Facet use differed significantly between the EBSCO database and Summon, though
not between the two different configurations of Summon. There was a significant
relationship between high use of facets in one platform being connected to high
use in the other platforms. In contrast to some previous studies, a non-trivial
proportion of participants went beyond the first page of search results. In
support of most previous studies, participants infrequently searched on the
subject field or changed the default sort order. Summon’s
article suggestion feature was noted as being especially helpful, and clicking
on suggested articles was significantly correlated with the number of article
records viewed.
Conclusion –
The choice of search tool has a large influence on students’ subsequent search
behaviour. Many of the advanced features are still missed by students, although
in this study the majority of sources picked were of high quality. The authors
note the importance of configuring the interface so that facets and other
features deemed worthwhile by librarians are higher up on the page. The researchers reason that the prominent display of facets
leads to greater uptake. Despite finding no association between library
instruction and facet use, teaching students how to use facets remains an
advisable strategy.
Commentary
This
article, a continuation of research previously conducted by two of the authors,
examines a few related questions that all revolve around the use of discovery
systems and similar search tools in academic libraries. Foremost among them is
the extent to which users take advantage of facets (or limiters). Previous
research has shown that many students do not use—and perhaps do not fully
understand—facets (Bloom & Deyrup, 2015). The
authors add to this literature by using a within-subjects design testing three
interfaces (two configurations of Summon and the Social Science Abstracts
database) to determine how this might affect facet use. However, the study
measures several other variables including the amount of time taken during the
searches, which links were clicked, and ratings of authority and relevance of
the articles that were selected.
This
commentary relies on the CAT critical appraisal tool (Perryman &
Rathbun-Grubb, 2014). The study is a well-motivated project with an extensive
literature review. The research question is somewhat broad and does not
explicitly mention facets, although this is a major aspect of the study. The
methodology is well described and appropriate to the analyses. For the most
part, the analyses are clear, but the article might have benefited from some screenshots
of the search platforms, especially for readers who are less familiar with
them. Some of the statistical results were presented without clear explanation.
For example, it seems like Pearson correlations were used which requires
interval or ratio level data, but it is difficult to interpret variables like
“use of the scholarly facet” or “clicking on a suggested article” as meeting
that criterion without more details (pp. 5–6). Other limitations of the study,
such as the convenience sampling and recruiting of a narrow subset of students,
are acknowledged and discussed.
This
study is laudable for laying out concrete and refreshingly clear advice on how
librarians should customize search tools to increase the use of facets and
other advanced features: the higher up and more visible, the better. Salience
in interface design is usually taken as common sense, but it can sometimes
(rather ironically) get buried in the practitioner literature. The careful
planning, execution, and analysis of this study is to be admired, but it also
raises the question of whether future studies could achieve a similar level of
thoroughness using more automated means. Could some of this data be captured
programmatically using browser plugins or server logs rather than screencast
software, from which data extraction is extremely labour-intensive? If so, this
might encourage more librarians to do this much needed work.
References
Bloom,
B., & Deyrup, M. M. (2015). The SHU research
logs: Student online search behaviors trans-scripted. Journal of Academic Librarianship, 41(5), 593–601. https://doi.org/10.1016/j.acalib.2015.07.002
Perryman,
C., & Rathbun-Grubb, S. (2014). The CAT: A generic critical appraisal tool.
In JotForm – Form Builder. Retrieved
from https://www.jotform.us/cp1757/TheCat