Original Research

Investigating the cognitive demand levels in probability and counting principles learning tasks from an online mathematics textbook

George Ekol, Simphiwe Mlotshwa

Received: 22 Jan. 2022; Accepted: 05 July 2022; Published: 16 Sept. 2022

Copyright: © 2022. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

This case study carried out during the 2020 coronavirus disease of 2019 (COVID-19) lockdown used online data collection means to investigate the distribution of cognitive demand levels of probability and counting principles (PCP) learning tasks in a popular online Grade 12 mathematics textbook, based on the PCP teachers’ rating. The teachers’ cognitive demand ratings were categorised following Stein’s mathematical task framework. Five mathematics teachers from four secondary schools in two provinces in South Africa participated in the study by filling in an online questionnaire. We developed a rating framework named the mean cognitive demand rating (MCDR) to help us interpret the teachers’ perception of the tasks in terms of cognitive demand to the learners. Data from the teachers’ ratings revealed nearly 65% of the PCP learning tasks in the online textbook were rated as high. Analysis of secondary data from Department of Basic Education diagnostic reports from 2014 to 2020, however, suggests no association between teachers’ rating of learning tasks and learner performance.

Contribution: This study draws attention to a long-standing underperformance in the topic of probability and suggests classroom-based study that focuses on the learners’ rating of the learning tasks themselves to understand clearly how best to support them.

Keywords: probability and counting principles; mean cognitive demand rating; mathematical competencies; mathematical task framework; descriptive statistical analysis; digital textbook; multiple representations; Grade 12.

Introduction

The worldwide spread of the coronavirus disease of 2019 (COVID-19) in the year 2020 brought changes to the way societies live, work and study. The institutions affected by COVID-19 responded by moving from physical, human-to-human interaction to virtual and online platforms. During 2020, South African schools were shut down for two and a half months before Grade 7 and Grade 12 pupils were permitted to return to school. During the shutdown, learning continued virtually and online for some schools, particularly the relatively well-resourced schools (Mohohlwane, Taylor, & Shepherd, 2020). In response to the closure of schools, and in a bid to ensure that learning was taking place especially for the candidate classes, the Department of Basic Education (DBE) issued guidelines with recommendations for children to take advantage of online learning resources to continue with schooling from home. To facilitate learning from home, some websites were zero-rated in partnership with mobile phone companies (DBE, 2020b).

Online learning resources include resources such as online video lessons on YouTube, digital textbooks (DT), and study guides. These resources can be accessed by learners working from home on a computer or on a smartphone connected to the internet. In this study, we focus on only one online resource, namely the DT used by Grade 12 mathematics learners and teachers in South Africa.

Due to the closure of schools in 2020 due the prevalence of COVID-19, we did not conduct classroom-based study involving the Grade 12 learners. We noticed that this online textbook was being heavily used by both Grade 12 teachers and learners and decided to investigate the teachers’ rating of learning tasks in the probability and counting chapter of the book. We were able to carry out an online survey with the Grade 12 mathematics teachers who are involved in teaching probability and counting principles (PCP).

Digital textbook

The DT approved by the DBE is freely downloadable on any mobile device such as tablets and mobile phones. The DT comprises 9 mathematics topics (sequences and series, functions, finance, trigonometry, polynomials, analytical geometry, Euclidean geometry, statistics, and probability) that are taught at Grade 12. The 9 topics are each written following the same format; thus, each topic begins with the revision of related concepts, followed by the content notes, a couple of worked out examples, and exercises at the end. The exercises have answers to enable learners to cross-check their solutions. In this article, we only discuss the topic of PCP.

Probability and counting principles

Probability theory is a mathematical modelling (Blum et al., 2007, p. 4) of the phenomenon of chance or randomness. Randomness has a specific meaning in probability and statistics (Batanero et al., 1997; Batanero & Sanchez, 2013). Suppose T is a finite probability space. We assume that the physical characteristics of an experiment in T are such that the various outcomes of the experiment have equal chance of occurring. Such a probability space, where each point is assigned the same chance of outcome, is called a finite equiprobable space (Moore, Notz, & Fligner, 2013, p. 262; Spiegel, Schiller, & Srinivasan, 2013). However, events in T are far from being random. For example, a coin does not generate random numbers because a tossed coin obeys the laws of physics depending on the force used, angle of toss, and surface of the coin. So, why then do the results of tossing a coin look random? It is because the outcomes are extremely sensitive to the inputs, so that very small changes in the forces one applies when tossing a coin do change the outcomes, say from heads to tails and back again (Moore et al., 2013, p. 262).

Regarding counting principles, these are techniques for determining without direct enumeration, the number of possible outcomes of a particular experiment (May, Masson, & Hunter, 1990, p. 189). For example, if T defined as above, has k elements, then each point in T is assigned the probability , and each event B ∈ T containing h points is assigned the probability . In other words, . We note that the formula for P(B) only applies to an equiprobable space T and not to a general space.

Multiple representations

By multiple representations, we mean techniques of teaching PCP that include various objects such as graphs, diagrams, texts, and 3D visualisations to facilitate learners’ grasping of the underlying meaning of the concepts.

Context

Probability and fundamental counting principles are relatively new topics in the South African mathematics syllabus (Zondo, Zewot & North, 2020), having been introduced for the first time in Grade 10 in 2012, and in Grade 11 in the following year. The topic was first examined in Grade 12 in 2014. Since their inclusion as compulsory topics in the South African mathematics syllabus, these principles have remained a challenge to many learners, with performance remaining generally poor over the years (DBE, 2015, 2016, 2017, 2018, 2019, 2020, 2021). For example, in 2013, the average score in the national examination was 30% (DBE, 2014). Although every year diagnostic reports are published stating specific concepts in PCP that learners show weakness in so that educators in schools can support them in overcoming such identified weaknesses, from 2014 to 2019, diagnostic reports strongly suggest that the same challenges faced by learners keep coming up in subsequent years.

There is also a lack of research to inform the teaching and learning of PCP at the school level. For example, a database search by the first author of articles in Pythagoras from 2016 to 2020 with probability as the keyword turned up only two outcomes, which were also not linked to the information on probability we were looking for. One article (Murray, 2017) sought to understand how the grades obtained at school for English and Mathematics affect the ‘probability’ of graduation at a university. Clearly, the context of said study was different from the current one. The second article (Prince & Frith, 2017) discussed quantitative literacy of South African school leavers who qualify for higher education. Again, this study is not related to PCP. These examples clearly confirm our assertion of little research in the topic of PCP at the secondary level. This study contributes to understanding mathematics teachers’ rating of the PCP learning tasks at Grade 12 in terms of the tasks’ cognitive demand levels. It may be that the underperformance in PCP at the national certificate is contributed to by the learning tasks that learners are prepared for on for the national examinations.

One of the proposals from the education authorities to try and reverse the poor performance at Grade 12 is the suggestion that the teaching and learning of PCP should incorporate multiple representations of tasks. Based on the mathematical task theoretical framework (Stein & Smith, 1998), we associate tasks with high cognitive demands with better prospects to enable learners to master PCP concepts, whereas tasks with low cognitive demand are associated with less chance of offering learners the opportunity to master PCP concepts. The objective is to understand the distribution of the cognitive demand levels of PCP learning tasks that are in the DT introduced earlier in the previous section of this study.

The Curriculum Assessment Policy Statements (CAPS) diagnostic reports (DBE, 2018, 2020) also recommend multiple representations of concepts as a strategy for teaching PCP at Grade 12. The effective learning of PCP at Grade 12 requires many resources. Teachers certainly play a vital role in supporting learners (Fennema & Franke, 1992).

Focus

This case study investigated the Grade 12 mathematics teachers’ rating of learning tasks in a PCP chapter in a popular Grade 12 mathematics online textbook. Teacher ratings were interpreted following Stein and Smith’s (1998) cognitive demand levels of learning tasks. Two research questions guided the study: (1) What is Grade 12 mathematics teachers’ rating of the PCP learning tasks in one popular Grade 12 mathematics online textbook? (2) From the teachers’ rating of learning tasks, and from the secondary data available on Grade 12 learners’ performance in PCP over the years, what can be said about the two pieces of data – might there be a link between the achievement in probability at the national level by Grade 12 learners, and mean cognitive demand level of the learning tasks that the learners popularly use to prepare for the national examinations?

Probability knowledge for teaching

The idea of probability is empirical. That is, probability describes what happens in very many trials, and we must observe many trials to pin down a probability. In this article, we use the definition based on the notion of proportion or relative frequency. Relative frequency of a score is obtained by dividing the frequency of that score by the total number of scores. Similarly, the probability of an experiment yielding a particular result (e.g. a coin toss yielding heads) can be defined as the number of equally likely and mutually exclusive outcomes, divided by the total number of possible, equally likely, and mutually exclusive outcomes. By equally likely, we mean that in the long run each of the possible outcomes will occur with approximately equal frequency (May et al., 1990, p. 179; Moore et al., 2013, p. 260).

Probability knowledge for teaching (PKT) includes content knowledge of probability, and various ways of presenting this content to the learners so that learning takes place (Batanero, Chernoff, Engel, Lee, & Sanchez, 2016). Like mathematical knowledge for teaching, PKT can be divided into probability content knowledge (PCK) and probability pedagogical content knowledge (PPCK). Probability content knowledge requires teachers to have specialised training in probability beyond the content covered in the high school.

Probability pedagogical content knowledge is knowledge about presenting the concepts to the learners so that learners easily understand them (Kazima & Adler, 2006; Hill et al., 2008). For instance, how does a teacher present to the learners concepts such as variation and randomness, aware that many learners come into a probability class with deterministic (concrete) understanding of the world around them? Take, for example, a probability experiment such as tossing a coin. In such an experiment, the outcome is not predictable every time the coin is tossed. It is, thus, not a straightforward case to generalise the probability of events arising from the experiment unless the experiment is repeated very many times. Nevertheless, although an individual trial has an unpredictable outcome, there is a predictable pattern of outcomes that will be obtained over a long series of trials (Moore et al., 2013, p. 260). Hence, for a teacher who may be unaware of the foundation principles of probability, moving their learners from theoretical probability to the concrete results can result in misunderstanding by the learners.

Mathematical tasks

In the mathematical task framework (MTF), a task is defined as a segment of classroom activity that is devoted to the development of a particular mathematical idea (Stein & Smith, 1998). A task can involve several related problems in each topic in mathematics, in this case PCP. Mathematical tasks used in the classroom, or used by learners in their homework, are the foundation for their learning (Doyle, 1988; Stein & Smith, 1998). Stein and Smith (1998) distinguish three phases through which tasks pass: the first phase includes tasks that are found in the instructional materials such as study guides, printed textbooks, and DTs. The second phase includes tasks that are prepared by the teacher, and the third phase involves tasks that the students engage with in the classroom or at home, as reflected in Figure 1.

FIGURE 1: The mathematics tasks framework.

This article will limit the discussion to phase 1 of Stein and Smith (1998) with a focus on PCP learning tasks in a DT. Learning tasks for PCP were chosen for two reasons. First, as noted earlier, PCP is a topic that learners show poor grades in at the matric level (see DBE, 2019, 2020). Second, through interaction with some Grade 12 mathematics teachers, we learned that the DT is widely used by learners and teachers. So, we wanted to understand the cognitive demand level of PCP tasks in the DT. This study contributes to providing research-based information on PCP at the matric level.

Mathematical task framework

As shown in Figure 1, the three phases of mathematical tasks are: (1) curriculum tasks found in the learning materials such as textbooks, and other CAPS-compliant learning materials, (2) tasks that teachers select and use in their classroom teaching, and (3) tasks that students implement in their day-to-day learning (Stein, Smith, Henningsen, & Silver, 2000). The three phases are interrelated.

Stein et al. (1998) use the MTF to classify tasks into four levels of cognitive demand, namely: (1) memorisation, (2) procedures without connections, (3) procedures with connections, and (4) doing mathematics (see Table 1). According to the authors, tasks that promote memorisation and procedures without connections do not present any challenge to the learners since they do not require deep reflection to solve. Tasks that involve procedures but require other information that is not obvious in the tasks are classified as procedures with connections. Finally, doing mathematics is a level at which tasks are highly cognitively demanding (Stein, Grover, & Henningsen, 1996). Based on Stein et al.’s (1996) classification, we associate high cognitive demand tasks with tasks that require multiple representations to solve. By ‘doing mathematics’, we mean engaging students with PCP tasks that give them the opportunity to develop their thinking and reasoning skills thus leading them to meaningful mathematical understanding (Stein & Smith, 1998, p. 13).

Cognitive demands of tasks

The MTF is used to classify PCP tasks found in a DT in terms of either high or low cognitive demand levels. Tasks that are set at a high cognitive demand level require multiple strategies to solve (Stein et al., 1996). Low cognitive demand tasks occupy learners with reproducing known facts. Tasks promoting memorisation (level 1) and procedures without connections (level 2) require less reflection to solve and are categorised as low cognitive demand level (Stein et al., 1996). For example:

The probability that Jabu likes tea is 0.6 and the probability that Jabu likes coffee is 0.3. If the probability that Jabu likes tea, coffee or both is 0.7, determine the probability that Jabu likes tea and coffee.

This task does not demand much more than using a formula and substituting in the respective values, then solving for the unknown. Let T represent tea, and C represent coffee, P(T) is the probability that Jabu likes tea and P(C) is the probability that Jabu likes coffee. Then, P(T∪C) = P(T) + P(C) – P(T∩C).

Tasks of high cognitive demand level require some thinking and reasoning to solve (Stein et al., 1996). Take an example adapted from Moore et al. (2013):

Government data in Country Z show that 10% of adults are full-time students and that 35% of the adults are age 50 years or older. Explain why we cannot conclude that because (0.10) (0.35) = 0.035, therefore about 3.5% of adults are college students aged 50 years or older.

One reason is that the two events are not necessarily independent, because not all 10% of adult full-time students are above 50 years of age. Moreover, it is reasonable to expect that younger adults are more likely than older adults to be college students. Hence, P(college student|over 50 years) < 0.10. This example fits in level 3 or level 4 of Stein et al.’s (1996) categorisation of learning tasks, namely procedures with connection, or doing mathematics.

Probability and counting tasks used in the study

The PCP tasks used in this article are obtained from a digital Grade 12 mathematics textbook. The textbook is endorsed by the DBE in South Africa. The book is freely available to South African users. Users are free to download and read the book on their mobile devices or print and read offline. The only restriction is for users to keep the book’s cover, title, contents, and short-codes unchanged.

We chose the digital book from among other books for three reasons. First, the book is used by many Grade 12 mathematics teachers and learners in South Africa, so it is a popular learning resource. Second, the book is freely available. Third, the book covers all mathematics topics taught in Grade 12 in South Africa. In this article, we focus only on the topic of PCP.

The PCP section is divided into eight sub-topics. For the purposes of this article, we limited our discussion to only four sub-topics, namely: the fundamental counting principles, factorial notation, tasks involving the application of counting principles, and tasks involving application of probability. We picked a total of 48 different learning tasks and asked five senior mathematics teachers at Grade 12 to rate the tasks according to the four levels of cognitive demand developed by Stein et al. (1996). Details of the study design are contained in the methodology section.

Methodology

Design

This study is a case study taking a descriptive statistical approach. This approach enables us to transform qualitative data into quantifiable form and use it to make sense of the cognitive demand levels of learning tasks in PCP.

Participants

Participants in the study are five secondary mathematics and probability teachers, pseudo-named A, B, C, D, and E to ensure anonymity. Initially, seven secondary mathematics teachers (six male and one female) were contacted by email to take part in the study. A questionnaire with clear instructions was emailed to all the seven teachers to complete. However, only five teachers, all male, from four secondary schools in two provinces in South Africa (Gauteng and KwaZulu-Natal) returned the questionnaire. The five questionnaires were entered into a spreadsheet (Table 2) for analysis.

Data gathering process

Table 2 has 48 rows and 8 columns. Each row represents one task taken from the DT. The first column provides the serial number of the task for identification during analysis. The second column gives the location of the task in the DT. An ‘exercise’ is a collection of tasks. For example, 10.4 (1) represents task 1 found under exercise 10.4 in the DT. It can also be observed from Table 2 that exercise 10.4 has a total of seven different tasks. The remaining 47 tasks are presented in a similar format. For instance, exercise 10.5 has a total of three PCP tasks. The next five columns after column 2 are the five participant teachers who independently rated the 48 PCP tasks according to the four cognitive demand levels on a scale from 1 to 4 for each task. The last column shows the mean rating for each task.

Consistency of measurements

The mean cognitive demand ratings (MCDR) by five senior PCP teachers from four schools in two provinces in South Africa were received by email by both authors of this article. Each teacher rated the 48 tasks independently of the other teachers. The first author entered the original data received from all teachers in Table 2 and the second author corroborated the entries with the original submissions. Table 2 was again cross-checked by the first author to ensure accuracy and consistency of measurements. In reporting the findings, we have rounded off the cognitive demand ratings of PCP tasks to the nearest digits.

Data analysis

Mean cognitive demand rating

In this article, we have categorised MCDR 1 and 2 as low, and MCDR 3 and 4 as high (Stein et al., 1996). An MCDR is a value (corrected to the nearest whole number) obtained from the five independent ratings of a learning task, divided by the total number of ratings. For instance, the MCDR for task number 48 in Table 2, is (corrected to the nearest unit), where x_i is a rating of teacher i. Table 3 provides the frequency distribution of the MCDR scores obtained from Table 2 and Figure 2 is the corresponding chart.

FIGURE 2: Bar graph of the mean cognitive demand rating of probability and counting learning tasks in the digital textbook by probability and counting principles teachers.

Findings

From Table 3 and Figure 2, teachers’ rating of the PCP tasks in the popular online textbook tasks reveal that 8.3% (n = 4) of the total learning tasks sampled in this study comprise facts that only require memory to solve; 12.5% (n = 6) of the tasks are procedures without connection; 64.5% (n = 31) are procedures with some connections; whereas 14.6% (n = 7) are learning tasks rated under doing mathematics, meaning, for example, tasks whose solutions are require learners to explore and understand the nature of mathematical concepts, processes, or relationships. Such tasks, according to Stein and Smith (1998) also demand self-monitoring or self-regulation of one’s own cognitive processes.

From Table 3, tasks requiring memorisation and procedures without connection together account for approximately 21% of the total number of PCP tasks sampled in this study. According to Stein et al.’s (1996) MTF, the above tasks are grouped under low cognitive demand level tasks which occupy learners with reproducing known facts. One hopes that these are not the kinds of task that take much of learners’ time when they prepare for PCP assessments at different school levels. However, until classroom-based studies are conducted, this remains an open question.

From Table 3, tasks that only need memorisation or procedures without connection are 6% more than tasks rated as doing mathematics. ‘Doing mathematics’ is conceptualised as engaging students in the learning tasks (Blumenfeld et al., 1991) that give them the opportunity to develop their thinking and reasoning skills (Stein & Smith, 1998, p. 13). An example of such tasks is:

The code to a safe consists of 10 digits chosen from the digits 0 to 9. Assuming that none of the digits is repeated, determine the probability of having a code with the first digit even and none of the first three digits is 0.

Such a task includes procedures with connection, but it also requires some reasoning skills from the learner to solve. Table 3 and Figure 2 clearly show that the majority (64.6%, n = 31) of the PCP learning tasks in the DT comprise procedures with connections. Only about 13% of the learning tasks are rated at the highest cognitive demand level of doing mathematics. Nevertheless, if tasks under procedures with connection, and tasks under doing mathematics are combined, it can be concluded that, overall, 79% of the PCP learning tasks in the DT are high-level cognitive demand tasks, and 21% are low-level cognitive demand tasks.

Discussion

This case study focused on and investigated Grade 12 senior mathematics teachers’ rating of learning tasks in a PCP chapter of a popular Grade 12 online textbook. Teacher ratings of tasks were interpreted following Stein and Smith’s (1998) cognitive demand framework. Two research questions guided the study: (1) What is Grade 12 mathematics teachers’ rating of the PCP learning tasks in one popular Grade 12 mathematics online textbook? (2) From the teachers’ rating, might there be a link between the achievement in probability at the national level by Grade 12 learners, and mean cognitive demand level of the learning tasks that the learners popularly use to prepare for the national examinations?

On the first question, 65% of the learning tasks in the chapter on PCP were rated by the teachers in this sample as procedures, but with some connections to other concepts and representations, which supported learning. Characteristics of such tasks include use of procedures, but after obtaining the numerical solutions, learners are expected to interpret the solutions. Other examples include interpreting the concepts of probability that have been represented in a diagram such as a tree diagram. We argue concepts such as the ones in our example engage learners beyond the procedures and can help them to understand underlying concepts in the tasks (Stein et al., 2000).

However, findings in this study also revealed that teachers in this study rated 79% of the learning tasks in the DT as having high cognitive demand. Only 21% of tasks in the DT the teachers rated as having low cognitive demand. If the teachers’ seemingly favourable rating of the learning tasks is true, the question that remains unanswered is: what explains learners’ general underperformance in the PCP topic in the Grade 12 national examination?

Under the new CAPS syllabus (DBE, 2011), probability has been examined in Paper 1 at Grade 12 since 2014 and contributes about 18% of the total marks in Paper 1 (Mutara & Makonye, 2014). However, since 2014, learners have performed poorly in this topic in matric examinations (see DBE, 2020). In fact, the mean percentage pass in probability is 34.7% for a period of seven years, from 2014 to 2020, respectively. In 2020 the percentage pass was 18% (see Table 4), the lowest since 2014 when the topic was first examined, and the pattern does not show signs of improvement.

This leads us to the second research question, which is: What are the implications of the teacher rating of the learning tasks in the online mathematics textbook on improved performance in PCP? Drawing on the secondary data from the CAPS document, and from our data from the teachers’ ratings of tasks, we can only offer two reflections on this question. First, the exceptionally low performance in probability in 2020 by Grade 12 learners partly speaks to the learning difficulty that learners could have faced during the COVID-19 closure of schools, but this observation has no direct link to our current data on teacher rating. The apparently favourable teacher rating of the online tasks is probably an indication of the confidence that teachers had (or still have) in the tasks. However, the learner performance as shown in Table 4 clearly shows that there is no link between the teacher ratings of tasks and learner performance. Looking at the teacher ratings and learner performance in PCP topic over the years, our study draws attention to the fact that there is a bigger problem in PCP that needs concerted effort to solve. The question of underperformance in PCP, our data have shown, cannot be fully explained by the hardships imposed on teaching during the COVID-19 lockdown. The issue must be about how probability is taught, how much time is allocated to it, who teaches the topic and at what time of the curriculum calendar year it is taught as well as the resources available to both teachers and learners. All these questions remain to be followed up in future research studies.

Limitations to the study

The study was carried out during the restrictions due to COVID-19, where physical contacts were restricted as recommended by the health authorities to keep individuals and the public safe from contracting the disease. Communication during the data gathering process depended mainly on email with an attached questionnaire for the teachers, and follow-up phone calls. We contacted seven PCP teachers, but in the end only five teachers returned the questionnaire. Although only five teachers responded, percentage wise, it still represented a reasonable percentage considering that our initial target was seven senior teachers of probability at Grade 12. We were not able to observe the teaching of PCP in the classrooms for the same reason explained above. Finally, this study focused only on the learning tasks that are available to learners in the online textbook, so we missed the teaching tasks and the nuances that the teachers incorporate in their actual lessons. Obviously, we also missed observing the tasks that learners implement in their learning in the classrooms (Stein et al., 1996). Future empirical studies should consider these uncovered areas with respect to PCP.

Conclusion and recommendations

The study opened our eyes to the challenges in the teaching of probability that we only have been hearing about but have not investigated for ourselves. This study suggests that teachers’ rating of tasks does not count until reflected in learner output in terms of learners’ performance in the tasks. It can also be argued that learners’ rating of learning tasks should precede the teachers’ rating. In other words, teachers should rely on, and respond to, the learners’ rating of learning tasks. One direct indicator is the learner scores in the tasks that are assigned to them. We recommend empirical classroom-based studies that support teachers with different ways of teaching PCP. One possibility is writing PCP teaching support materials that complement the online materials and the CAPS-recommended materials, focusing on the understanding of meanings in PCP, and their applications.

Acknowledgements

G.E. would like to thank Wits University, Faculty of Humanities, for the small research grant for this report.

The authors have declared that no competing interests exist.

S.M. compiled the data and wrote the first draft. G.E. added more research information on the draft, wrote and edited the final manuscript.

This article followed all ethical standards for research without direct contact with human or animal subjects.

G.E. received project fund number GEKL020 from the Faculty of Humanities Research Grant.

Data sharing is not applicable as no new data were created.

The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of any affiliated agency of the authors.

References