Original Research

Effective communication of learning intentions and success criteria in the mathematics classroom: MERLO pedagogy for Senior Phase South African schools

Lydia O. Adesanya, Marien A. Graham

Received: 16 Nov. 2021; Accepted: 20 June 2022; Published: 16 Aug. 2022

Copyright: © 2022. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

A well-designed assessment construct is critical for improving all aspects of quality education and validating the achievement of educational reform. The global prevalence of how teachers communicate learning intentions (LIs) and success criteria (SC) has been of great concern, particularly in the South African context. This study investigates how Meaning Equivalence Reusable Learning Objects (MERLO) pedagogy effectively transforms Senior Phase mathematics teachers’ daily practice in the classroom. The study adopted qualitative participatory action research to frame the evolution of teachers’ praxeologies such as teachers’ meta-didactical and didactical praxeologies, to improve teachers’ beliefs and practices to integrate MERLO pedagogy as assessment activities. Twelve Senior Phase teachers were purposively selected in Gauteng, South Africa. The methods used for data generation were interviews, classroom observation, document analysis, field notes and training sessions. Thematic analysis was used to obtain insight into teachers’ beliefs and practice of effectively communicating LIs and SC in the classroom. At the initial stage, teachers were examined with regard to their beliefs and practices of assessment practices in the classroom, which informed MERLO intervention. In the second stage, teachers were asked to learn about MERLO items by reading the MERLO handout provided to them, participating in the workshop and sharing their opinions and views with others. In the third stage, teachers had to design MERLO assessment items on their own to assess learners’ level of understanding of the mathematical concepts in Senior Phase. The findings revealed that the participating teachers acquired adequate knowledge and skills on MERLO techniques that allowed them to structure and integrate the lesson plan of assessment activities into their mathematics classrooms. This study contributes to the body of knowledge by introducing MERLO pedagogy to Senior Phase South African mathematical teachers as an assessment strategy. COVID-19 caused some teachers to drop out of the study after the pre-MERLO participation phase and, accordingly, future research suggests that more teachers be included in similar studies.

Keywords: assessment; MERLO pedagogy; learning intentions; success criteria; mathematics classroom.

Introduction

Quality assessment practices, specifically in mathematics, is a topic that is getting a lot of recent attention in the 21st century (Barana & Marchisio, 2021; Granberg, Palm, & Palmberg, 2021; See, Gorard, Lu, Dong, & Siddiqui, 2021). This study aimed to investigate how Meaning Equivalence Reusable Learning Objects (MERLO) pedagogy effectively transforms South African Senior Phase¹ mathematics teachers’ daily assessment practice in the classroom. Meaning equivalency is a concept that signifies shared meaning across representations: it is a polymorphous – one to many – transformation of meaning. As a pedagogical technique for teaching and assessment, MERLO asks learners to sort and map significant ideas using representative target statements of specific conceptual contexts and relevant statements that may or may not share the same meaning. When MERLO assessment items for various ideas are merged into a large database for a course of study, significant information about learners’ learning patterns can be obtained (Etkind, Kenett, & Shafrir 2010; Etkind, Kenett, & Shafrir, 2016). In a recent book chapter by Etkind, Prodromou and Shafrir (2021), it is pointed out that MERLO can be applied as a form of formative assessment (FA) and summative assessment (SA) to validate what learners know and get feedback regarding their conceptual understanding of mathematical concepts in the classroom. The MERLO pedagogy could aid teachers in developing new knowledge and skills relevant to designing their lessons (Etkind et al., 2021; Robutti, 2015). The implementation of MERLO pedagogy requires teachers to be skilful and competent to continually change their assessment practices in response to the actual requirements of their learners; updating or changing their assessment practices is critical as many studies have shown that assessment has considerable potential for enhancing learner performance (Nortvedt & Buchholtz, 2018; Polly et al., 2017; Suurtamm et al., 2016; Veldhuis & Van den Heuvel-Panhuizen, 2020).

Background of the study

Assessment practices in education have been a continuous focus for over a decade globally (Clarke & Luna-Bazaldua, 2021). The effective use of the assessment process allows one to elicit information on what learners need to know, understand and be able to do at the end of the lesson (Clarke, 2012). Scholars indicate that assessment, as an integral part of classroom practices, has the potential to effectively enhance learners’ learning and performance (Granberg et al., 2021; Heritage & Wylie, 2018). Igunnu (2020) believes that assessment should produce accurate information and validate concrete learning by learners. There are two types of assessment, namely FA and SA. The former is an assessment technique used to improve learners’ performance, whereas the latter is used to evaluate learners’ performance (Khechane, Makara, & Rambuda, 2020). South Africa participates in both national (Annual National Assessment [ANA]; Van der Berg, 2015) and international assessments (Trends in Mathematics and Science Study [TIMSS]; Human Sciences Research Council [HSRC], 2020, and South and Eastern Africa Consortium for Monitoring Educational Quality [SACMEQ]; Department of Basic Education [DBE], 2017), and South African learners constantly perform far below the excepted standard in mathematics across all grades. Authors have tried to explain this poor performance, with many of them (e.g. Chavalala, 2015; Outhred, 2022) linking it back to poor assessments in the schools; for example, Chavalala (2015) states that the ‘low performance of South African learners in various national and international assessments can be linked to lack of quality assurance of assessment practices in schools’ (p. 7). In fact, in the ‘Action Plan to 2024: Towards the realisation of Schooling 2030’ issued by the DBE in 2020, the DBE directly links better learning outcomes to ‘more focussed assessment practices’ (DBE, 2020, p. 37). Scholars have indicated that to improve education quality, teachers need to be trained, supported through professional development, and they should be willing to improve their assessment practice in terms of improving learners’ learning skills, learners’ involvement and learners’ goals and objectives (UNESCO Institute for Statistics, 2016). The latter speaks more to FA than SA (thus, the focus of this study was on FA although MERLO can be used for both FA and SA), as recent research debates that SA causes teachers and learners to be overly concerned with performance rather than learning goals (Ishaq, Rana, & Zin, 2020; Karaoğlan-Yilmaz, üstün, & Yilmaz, 2020), whereas FA that asks probing questions helps learners to deepen their understanding (Kyaruzi, Strijbos, Ufer, & Brown, 2019) and provides ‘opportunities for further learning and conceptual development through feedback, interpretation, and dialogue between teachers and students’ (Arifuddin, Turmudi, & Rokhmah, 2021, p. 242). Kyaruzi et al. (2019), who conducted a study on mathematics FA practices in Tanzania, remark that FA supports learners’ learning, which is positively related to their use of deep-level learning strategies, and Knight, Shum and Littleton (2014) highlight the fact that FA plays a ‘crucial role in guiding a student’s epistemic beliefs’ and that FA may be the ‘disambiguation of the epistemic requirements of questions — in terms of understanding the question, its context, and the knowledge required to answer the question’ (p. 28). These preceding arguments led to our study of introducing the use of MERLO items to assess the epistemic quality of what learners need to know, understand and be able to do in the mathematics classroom (Hudson, Henderson, & Hudson, 2015).

The importance of conceptual thinking skills is now recognised as a cornerstone of effective learning, understanding facts and ideas in the context of a conceptual framework (Bransford, Brown, & Cocking, 2004), as ways of thinking that explore patterns of equivalence-of-meaning in ideas, relations, and underlying issues.

Several studies on MERLO pedagogy development and the nature of its reflective practice have been evolved, validated, tested and implemented across different countries (Australia, Canada, Israel, Italy, Russia and the Netherlands) and various content areas and disciplines, including mathematics (Arzarello et al., 2015; Etkind et al., 2010, 2016; Etkind & Shafrir, 2013; Persoons & Di Bucchianico, 2020; Prodromou, 2015; Robutti, Carante, Prodromou, & Kenett, 2020a). However, MERLO pedagogy used as an assessment strategy has not been developed and implemented in South Africa. South Africa is one of the most unequal countries in the world, not only when referring to the fact that approximately half of South Africans live in poverty, that economic growth has stagnated, and the unemployment rate is almost one-third of South Africans, but also in terms of its education system (Francis & Webster, 2019). In addition to this, South Africa is performing poorly in mathematics (as mentioned earlier), and it is imperative to introduce an inexpensive and effective method into South African schools to enhance mathematics performance. Therefore, this study contributes to the body of knowledge by introducing MERLO pedagogy to Senior Phase South African mathematical teachers, which can be implemented inexpensively as an assessment strategy to promote the conceptual higher-skills thinking and understanding of mathematics in their daily practices.

Since the new democratic era was implemented in 1994 (DBE, 2009; Kanjee & Sayed, 2013), the topic of assessment has been deliberated in the South African educational system (Mouton, Louw, & Strydom, 2013; Pahad, 1999). In the past decade, curriculum changes are not an easy task to integrate and work on, as teachers are struggling to make sense of the demands placed on them in South African schools (Govender, 2018; Mouton et al., 2013). Govender (2018), who conducted a study on teachers’ views on the curriculum changes, reported that teachers felt that the increased number of assessments increased their administrative duties. Poliah (2019) asserts that assessment differs from province to province in the South African context, including regions, districts and schools. Some scholars also reflect on the differences in the international external standardised testing results in TIMSS across several years (Howie, 2002; HSRC, 2011, 2020; Reddy et al., 2020). These differences create doubts about the consistency of assessment. Nevertheless, Vandeyar and Killen (2007) and Poliah (2019) indicate that the dominant challenge in assessment is to find strategies that will provide an equal opportunity to all learners while allowing credible, reliable, valid and effective outcomes. Furthermore, researchers have identified several obstacles associated with the application of assessment techniques in South Africa (Dube-Xaba & Makae, 2021; Kanjee & Sayed, 2013; Poliah, 2019; Vandeyar & Killen, 2007; Van Staden & Motsamai, 2017). These difficulties range from inadequate training and severe workloads to policy demands that are sometimes difficult to meet. In their critical reviews of the use of practising assessment in the teaching and learning process, scholars have observed that a well-designed assessment construct is vital for improving all aspects of quality education and validating the achievement of educational reform (Adesanya & Graham, 2021; Sayed, Kanjee, & Rao, 2014). With the focus of the current article being on mathematics teaching and learning, authors have established that assessment is valuable in mathematics teaching and learning (Nortvedt & Buchholtz, 2018; Polly et al., 2017; Suurtamm et al., 2016; Veldhuis & Van den Heuvel-Panhuizen, 2020). The contribution of this study goes even further than providing South African teachers with assessment techniques that could ultimately improve learner performance. It should be noted the MERLO assessment technique is something that South African teachers have not been exposed to before this study, and a recent study conducted within a South African context (Warnich & Lubbe, 2019) has shown that by applying innovative and alternative assessment practices, enjoyment is brought into the classroom that alleviates learner stress and enhances learner engagement. Furthermore, since the adoption of assessment practice across schools in South Africa, few empirical studies on learning intentions (LIs) and success criteria (SC) have been conducted. This study thus investigates how MERLO pedagogy as an assessment strategy can be used to understand teachers’ beliefs and practice of effectively communicating LIs and SC in the South African Senior Phase mathematics classroom. Crichton and McDaid (2016) summarise the difference between LIs and SC succinctly:

‘LIs tell the learners what the intended outcome of the lesson is with regard to their learning. SC provide examples of their expected performance as a result of the lesson, ‘closing the gap’ between learners’ previous knowledge and their developing understanding.’ (p. 190)

Teachers’ beliefs on assessment practices in the South African context

Teachers’ beliefs have been a topic of great interest in mathematics didactics (Lepik & Pipere, 2011; Ramnarain & Hlatswayo, 2018). According to scholars, beliefs are the most important psychological element that should guide teacher education (Grossman, 1990; Holt-Reynolds, 1992). Teachers’ beliefs reflect how they conceptualise mathematics and its teaching and learning. It is clear that researchers, communities and policymakers around the world are interested in understanding the different ways in which teachers’ beliefs have contributed to and influenced learners’ academic performance (Lepik & Pipere, 2011). However, the terminologies that immersed knowledge and belief into a single construct defined teachers’ beliefs about assessment (Barnes, Fives, & Dacey, 2015). Ideally, Binns and Popp (2013) and Ramnarain and Hlatswayo (2018) emphasise the importance of teacher beliefs by asserting that it is not only a teacher’s educational experience that impacts whether a teacher would utilise a pedagogy that supports learner-centred learning, but also teachers’ beliefs, values and attitudes about knowledge and how it is acquired. Scholars have indicated that pedagogical tactics are influenced by teachers’ notions about assessment strategies, teaching and learning, the nature of mathematics and classroom assessment practices (Kuze & Shumba, 2011; Ramnarain & Hlatswayo, 2018; Sikko, Lyngved, & Pepin, 2012).

Scholars reveal that some of the biggest challenges to integrating assessment strategies in the mathematics classroom are teachers’ beliefs regarding effective planning and preparation, classroom management, inequity, lack of teachers’ training, lack of teaching and learning aids and materials, and lack of time (Martin, Mraz, & Polly, 2022; Panthi & Belbase, 2017; Schoen & LaVenia, 2019). Teachers’ beliefs influence their perceptions and judgment, which in turn influence their collection of pedagogy techniques and classroom behaviour (Pajares, 1992). Meanwhile, Harwood, Hansen and Lotter (2006) claim that teachers’ beliefs have been found to impact teachers’ classroom practices, how they feel content should be taught, and how they believe learners learn. Beliefs are thus likely to play a significant role in whether teachers aim to carry out the practice of teaching mathematics by asking questions to seek out information (Crawford, 2014; Karim, 2015).

In a South African study, Van der Nest, Long and Engelbrecht (2018) used a qualitative approach to capture the experiences and perceptions of four Grade 9 mathematics teachers who participated in a professional development programme that focussed on the use of FA activities in mathematics teaching and learning. They found that, although some of the teachers saw the potential for a deeper conceptual understanding developing in their learners using these new FA techniques introduced to them, their current focus was on their learners’ good performance in the ANAs, and this was taking attention away from the focus of implementing these new FA strategies meant for developing a deeper conceptual understanding of mathematics. This argument makes a direct link between South African teachers’ beliefs and learners’ mathematics performance in that, when the choice is there between creating a deeper conceptual understanding of mathematics or ‘teaching to the test’ (the ANAs), the focus falls on the latter, thus showing learner improvement in a SA (the ANAs), the results of which are released in public domains. Van der Nest et al. (2018) go on to say that the focus of teachers having their learners perform well in the public eye (in external tests that are not aligned with classroom teaching and learning) has a negative impact on mathematics education in South Africa.

An exploratory study conducted on South African teachers revealed that the achievement and failure of assessment practices are affected by teachers’ belief in the practices that they employ in the classroom (Kanjee, 2020). In another South African study, it was found that teachers believe that various factors in terms of teacher workload, ineffective lesson preparation and planning, disruptive classrooms, large class sizes, and time constraints affected teachers’ inconsistent practice of quality assessment in the classroom, which they believe negatively affects learners’ mathematics performance as they believe that assessment is advantageous for identifying learners’ misconceptions in learning (Adesanya & Graham, 2021). More so, they also believe that assessment outcomes can be utilised to transform their teaching strategies to meet the learning intentions (LIs) (Adesanya & Graham, 2021). However, some teachers do not use assessment outcomes effectively, and their measures to follow up learners’ performance are ineffective. These imply that teachers did not thoroughly practise assessment as they believed. Therefore, for assessment practice to be effective, teachers need further training with guidelines revision, monitoring and periodic assessment (Adesanya & Graham, 2021; Kanjee & Croft, 2012; Poliah, 2019; Vandeyar & Killen, 2007).

Communicating and sharing of learning intentions and success criteria with learners in mathematics classrooms

Regarding the role that understanding and ascertaining LIs and SC plays in ensuring effective mathematics teaching, Jones and Edwards (2017) state:

‘Learning to plan effective mathematics lessons is one of the most important capabilities you can acquire in becoming a successful teacher of mathematics. Having a good lesson plan is significant for a whole host of reasons, not least in providing the structure which helps you to be confident that mathematics learning takes place during your lessons. Not only does good planning result in lessons that are interesting, challenging and motivating for your students, but also good planning is closely linked to the equally demanding (but often more overt) issue of effective classroom management.’ (p. 70)

Graham, Van Staden and Dzamesi (2021), who conducted a study in Ghanaian mathematics classrooms, emphasise the importance of communicating LIs and SC with learners and state that the LIs and SC are the forces that drive the process of assessment practices in the mathematics classroom. Various scholars of classroom practice concur that quality assessment requires teachers to understand, clarify, share and communicate LIs and SC with their learners during the lesson (Bartlett, 2015; Graham et al., 2021; Pryor & Crossouard, 2008; Wiliam, 2016). However, teachers need to carefully design a lesson plan that measures the quality of the instructional objective, which directs them to aid learners in accomplishing their learning goals. Stating and clarifying LIs stipulate what learners will learn during teaching (Moss & Brookhart, 2019). Teachers need to have an action plan for what they will teach their learners in the classroom by understanding and ascertaining LIs and SC in the process of teaching and learning (Bennett, 2011; Heritage, 2010; Moss & Brookhart, 2019). For instance, mathematical teachers’ pedagogical decisions about how to involve learners in higher-order conceptual thinking skills, reasoning and problem-solving have a direct impact on their learning outcomes. Many learners in schools can answer simple mathematical problems, but they lack critical thinking, reasoning and problem-solving skills, especially when working on higher cognitive level open-ended questions (Hoogland & Tout, 2018).

According to Heritage (2010), the LIs direct learners’ attention to what they will learn rather than the activities they will do. The teacher’s attempt to clarify and share LIs with their learners which promotes them to be actively engaged in the learning process rather than passive recipients of knowledge. The LIs and SC must be communicated to learners properly and in a language that they can understand. The use of easy words related to cognitive domains of learning that explain the LIs and SC should be communicated to the learners so that they grasp the purpose of the class and can simply share it with their peers in the classroom (Heritage, 2010). Learners are so driven to learn new skills and knowledge through active engagement to prevent learning by memorisation that will not improve learners’ performance (Brabeck, Jeffrey & Fry, 2017).

Based on communicating and sharing LIs and SC to learners, scholars highlight that teachers should try to employ effective techniques before, during and after the FA process by ensuring that learners understand, know and communicate LIs and SC with others (peers) (Moss & Brookhart, 2019). Moss and Brookhart (2019) further assert that when teachers state and communicate LIs and SC with their learners appropriately, they would have a starting point to ‘plan their lesson with effective strategies that scaffold learners’ activities, act and monitors their teaching, and help their learners to become self-regulated as well as assessment-capable learners’ (p. 8). To this end, some scholars indicate that implementing effective assessment strategies during teaching and learning plays a vital role in increasing the progress and level of learners’ achievement in learning (Nortvedt & Buchholtz, 2018; Polly et al., 2017; Suurtamm et al., 2016; Veldhuis & Van den Heuvel-Panhuizen, 2020). This argument implies that if teachers do not grasp the LIs and SC, it will impede the consistent practice of assessment because assessment is determined by teachers’ understanding of the aim and objectives of a content.

Meaning Equivalence Reusable Learning Objects pedagogy approach

Since the 1990s, MERLO has been a pedagogy and teaching technique developed, validated and experimented with within different countries and across different content areas and disciplines (Etkind, Shafrir, Kenett, & Roytman, 2016; Etkind & Shafrir, 2013; Etkind et al., 2010). As a pedagogical tool, MERLO is appropriate for different versions of core content based on sharing the meaning across different forms of representation (Arzarello et al., 2015; Robutti et al., 2016, 2020a, Robutti, Prodromou, & Aldon, 2020b). Additionally, MERLO is a powerful tool for problem-solving mathematical concepts known as duplication obstacles, extensive in all mathematics classrooms. Generally, MERLO items are made up of five statements, namely an unmarked target statement (TS) and four other statements that are developed by sharing meaning equivalence with TS and sharing surface similarity with TS (Etkind et al., 2016). The four quadrants are Q1, Q2, Q3 and Q4. The relevance of the four quadrants to the TS tries to identify learners’ needs in learning, which provides teachers with the opportunities to pay much attention to how they plan their lessons and design good, effective questions for mathematics teaching. It also leads learners to choose two representations of objects that share the same mathematical meaning in the questions illustrated. Q1 is about the representation that shares meaning equivalence and surface similarity with the TS. Q2 is about the representations that are not similar in appearance to the TS but share meaning equivalence with the TS. Q3 focuses on the representation that is similar in appearance to the TS but does not share meaning equivalence with the TS. Q4 focuses on the representation that is not similar in appearance to the TS, and does not share meaning equivalence with the TS. Etkind et al. (2016) recommend that Q1 statements be excluded as they are extremely straightforward and ‘give away the shared meaning due to the valence match between surface similarity and meaning equivalence, a strong indicator of shared meaning between a Q1 and the target statement’ (p. 318). The main TS can be written in various semiotic or symbolic systems (text, image, map, decimal, percentage fraction and others).

Scholars have demonstrated that designing MERLO patterns requires some steps to acquire different equivalence forms of representations that share the same mathematical meaning with the TS (Arzarello et al., 2015; Prodromou, 2015; Robutti et al., 2016, 2020a, 2020b). According to Arzarello et al. (2015) and Prodromou (2015), the basis for designing MERLO items is to identify a close link that relates to the concept because it needs a change from old-style questions into present-day questions. Furthermore, the TS was designated as an open question because ‘teachers acquired the practice of elaborating the TS (i.e., TS) as a statement, graph, or table’ (Arzarello et al., 2015, p. 5). Several studies have indicated that in designing MERLO items, there are some difficulties in choosing items that share the same mathematical meaning with the TS (Robutti et al., 2016, 2020a, 2020b). This implies that teachers must choose representation items that are linked with one another before designing MERLO activities. These steps are described below:

Designing and preparing the statement of TS and Q2 requires the facilitators to select a concept from the core content in mathematics and write down the statement as a TS, then design one or more statements that share the same meaning with the TS in different representations (i.e., tables, texts, numbers, equations diagrams) to signify Q2. To design and prepare the statement for Q3, the facilitators have to use concepts that appear similar (surface similarity) to the TS but do not have the same meaning as the TS. This implies that Q3 does not share the same meaning as TS and Q2. To design and prepare the statement for Q4 is different because it does not have the equivalence of meaning and surface similarity to the TS and Q2. These characteristics imply that Q4 does not share the same meaning as the TS and does not appear to be the same as the TS (Robutti et al., 2016, 2020a, 2020b).

Theoretical framework

The constructivist philosophy was embraced to underpin the study as the instructional process of this study was designed under the principles of constructivism. This is because the teacher facilitates a process of learning in which learners are encouraged to be responsible and autonomous in their learning. Paying attention to the growing trend of engaging practising teachers under investigation, Reis-Jorge (2005) proposes that the goal is to educate reflective practitioners ‘who are more acquainted with theoretical discourse and more skilful readers of research literature’ (p. 303). As a result, we regard teachers’ engagement with research literature as an activity aimed at professional development through knowledge sharing among communities in mathematics education rather than a path to complete absorption in the research endeavour. This conceptualisation led us to structure our work with the meta-didactical transposition theoretical model (Arzarello et al., 2014), which was developed to describe the complex dynamic that occurs when teachers and researchers interact with one another.

Meta-didactical transposition

Meta-diactical transposition (MDT) is highly relevant to describing present actions and interactions among researchers and mathematics teachers relating to the MERLO pedagogy after participation in workshop training sessions (Arzarello et al., 2014; Robutti, 2018). The term meta-didactical ‘refers to the fact that important issues related to the didactical transposition of knowledge are faced at a meta-level’ (Robutti, 2018, p. 4). This framework was suitable for describing teachers’ praxeologies such as teachers’ meta-didactical (i.e. teacher professional development) and didactical praxeologies (i.e. mathematics praxeologies) that consist of four interrelated components of the task, technique, technology and theory (Arzarello et al., 2014; Robutti et al., 2020a). The task and the corresponding techniques are discussed as the practical counterpart (i.e. the praxis), while the technology and theory, in the sense of justification, are the theoretical correspondence that uphold the use of those techniques (i.e. the logo).

With the purpose of the study, MDT offers an interpretative model of mathematics teachers’ praxeologies. A mathematics praxeology is made of tasks requiring teachers to use questions and actions, representing a didactical praxeology. For instance, teachers need to choose a mathematical concept (i.e. fraction) in the CAPS documents or a question from a test as a TS and incorporate it to design any form of representation; teachers need to repeat the same mathematical concept with a different form of graph or number line area, by using statements that have the role of Q2; teachers also need to complete the items with statements that have the role of Q3 or Q4, which act as a distractors or guessing (i.e. the practical component). The theoretical components made by various theoretical frameworks and aspects could justify the practical components: the MERLO pedagogy approach (made by the main criteria of meaning equivalence and surface similarity) and the epistemic nature of the mathematical contents, which are intertwined with pedagogical, didactical, curriculum and assessment (Chevallard, 2019; Shinno & Yanagimoto, 2020). This framework implies that the contribution from mathematics education research dealing with learners struggling to understand mathematics concept (i.e. fractions) is closely linked with the Senior Phase teachers’ direct experience in the classrooms.

Research methodology

This study was part of a larger participatory action research (PAR) project, that draws on the paradigms of constructivism. According to scholars, PAR is constructionist and knowledge is socially created (Armstrong, 2019; Baldwin, 2012; Florian & Beaton, 2018). We used PAR because it builds up opportunities to empower and support participants to re-think and change their practices in the education sector; it focuses on social transformation that promotes democracy and combats inequity (Chevalier & Buckles, 2019; Kemmis, McTaggart, & Nixon, 2014; Riel, 2019). This larger project investigated the impact of how MERLO items were used in Senior Phase South African mathematics classrooms for teaching and learning.

Twelve Senior Phase mathematics teachers were purposively selected from six public schools due to the participants’ uniqueness in their qualities (i.e. mathematics teachers with at least two years teaching experience of teaching mathematics) (Maree & Pietersen, 2019). Two Senior Phase mathematics teachers were chosen from six public schools. The reason for adopting a purposive non-probability sampling technique in the current study was due to the qualities of the teachers’ skills and knowledge. Although 12 participants were initially part of this study, due to COVID-19, eight dropped out after the critical evaluation of problem identification in terms of understanding teachers’ beliefs and practice of effectively communicating of LIs and SC in the South African Senior Phase mathematics classroom, which left only five teachers in a MERLO participation programme.

The South African mathematics teachers were also provided with a handout about the MERLO items and their relevance in education as teaching, learning and assessment. The handout was adopted from the research papers of Arzarello et al. (2015) and Robutti et al. (2016). They were further provided with some examples of how to construct the different statements of a variety of MERLO items. Subsequently, they were asked to design a MERLO item that could be used to assess learners’ understanding of the learning results across their lesson plans. During the completion of their MERLO items, teachers shared their views in the training sessions; they discussed the correctness of their MERLO statement with the other teachers and the researchers (i.e. the facilitators). After the completion of designing the MERLO items, the teachers were required to explain the design strategies and justify the series of the steps they followed to construct the MERLO items that they would communicate or share to assess their learners’ conceptual understanding of mathematical concepts.

The data of this study consist of MERLO items designed by South African Senior Phase teachers, audio-tape recordings of teachers’ face-to-face MERLO presentations in the classroom, audio-tape recordings of semi-structured interviews (pre and post interviews), field notes, reflective journals and the MERLO handout for intervention (the content relates to the production of MERLO and the explanations and the justifications of the steps they need to follow to design MERLO questions).

Due to the COVID-19 pandemic, stricter measures were put in place by each school to keep their learners safe. Ideally, we wanted to observe each teacher at least three times, but due to the situation in schools (i.e. COVID-19 cases), each teacher was observed at least twice. The purpose of the first observed lesson was to allow the teachers to develop a good knowledge and understanding of using the MERLO pedagogy as a form of assessment activity in their classroom. During the first lesson, which was the first cycle of the class, we focused primarily on how the teachers applied the components that constituted the mathematical praxeologies with their learners, which related to teachers’ didactical praxeologies. At the end of each lesson, feedback was given to the teachers on the areas that needed improvement.

The second lesson observation aimed at checking whether there was any progressive improvement in presenting the MERLO pedagogy in class. During this stage, the researchers used audio-tape recordings and written notes to assess how the teachers present and communicate MERLO pedagogy as a form of assessment activity in the teaching and learning of mathematical concepts. During teachers’ didactical praxeologies (i.e. classroom implementation), the teachers integrated the knowledge acquired from the MERLO pedagogy involvement into their lesson plan to guide them when teaching the concepts. By the end of the second cycle of presenting and communicating MERLO pedagogy in the classroom, only one of the teachers involved the researchers in the lesson to clearly explain MERLO. He said, ‘Please can you explain to learners for clarity’ (SCH2-FOB-MT2).

Thematic analysis was used because it provides a way to look for patterns in the data set, connecting into meaningful categories collectively and themes that represent the study being investigated (Braun & Clarke, 2019; Dolgobrodova, 2016). The researchers of this study listened carefully to all the audio-tape recordings and analysed the transcribed data, including coding of teacher MERLO implementation in the teaching and learning of mathematical concepts. The data includes pre and post semi-structured interviews, reflective journals and MERLO items designed by the five Senior Phase South Africa mathematics teachers and MERLO classroom implementation in the mathematics classroom. The analysis was based on the study of research literature in terms of teachers’ beliefs and practices of assessment which informs how MERLO items were effectively presented and communicated in the mathematics classroom in the South African context (i.e. the process corresponds to classroom observation).

Ensuring trustworthiness

Trustworthiness was ensured through data collected from several sources to corroborate the facts and multiple methods. Trustworthiness was ensured by member checking: participants were given copies of their transcripts to confirm the accuracy (Nieuwenhuis, 2019).

Ethical considerations

Approval to conduct this study was obtained from the Ethics Committee of the Faculty of Education, University of Pretoria. Pseudonymns were used to project participants’ identities; for example, SCH1-PrInt-MT1 stands for a male teacher from the first school and SCH1-PrInt-FT1 stands for a female teacher from the first school. Anonymity and confidentiality were ensured by not revealing any identifying information of the participants. All participants signed consent forms indicating voluntary participation. The potential participant was assured that participation in the research is entirely voluntary and that they were free to withdraw at any moment.

Presentation and discussion of the data

Theme 1: Concept of assessment and sharing of learning intentions and success criteria

The finding of this study informed MERLO intervention to enhance effective communication of LIs and SC in the mathematics classroom. The data were presented through pre interviews, teachers’ lesson plans and classroom observations. The following themes emerged.

Sub-theme 1.1: Teachers’ beliefs on assessment practices

Teachers’ beliefs about assessment practices in the classroom play a vital role, as most teachers consistently believe that assessment is used to check learners’ understanding at least most of the time during teaching. One teacher (SCH1-PrInt-MT1) mentioned that it is very challenging to effectively practise assessment in the classroom because of the pressure to finish the scheme of the subject that needs to be presented within a single lesson period:

‘It will be very difficult for you to come up with the real real real assessment in class because sometimes you are pressured by accomplishing of the lesson, you come with the plan that we have, the mental maths, the presentation of the subjects, the classwork and the homework which we must just complete them within a single period.’ (SCH1-PrInt-MT1)

This shortcoming of teachers’ belief to effectively practise assessment in the classroom could contribute to the inconsistency of planning, preparing and implementation as studies reviewed that the achievement and failure of assessment practice are affected by the teacher’s belief in assessment practices (Barnes et al., 2015; Kanjee, 2020; Karim, 2015).

Sub-theme 1.2: Communicating learning intentions and success criteria

This sub-theme is concerned with teachers’ understanding of communicating the LIs and SC to the learners. Pre interviews and classroom observations were used to determine whether the teachers communicated the LIs and SC to their learners in the mathematics classroom. From the pre interviews, only two out of the 12 teachers (SCH2-MT2 and SCH6-MT2) mentioned that it is used to verify whether objectives for the lesson have been reached. The LIs identify what the teacher wants the learners to know at the end of the lesson. Furthermore, none of the teachers mentioned the SC. According to Graham et al. (2021), the sharing and clarification of LIs and SC by teachers and peers at the beginning of the mathematics lesson (to achieve an understanding of the LIs and SC by the learners from the beginning of the lesson) is vital for assessment to achieve an effective result. One of the teachers gave the following response:

‘Assessment is a tool that you use to check whether how far the learners learned. Understanding has gotten. Have you reached the outcome or have you reached the objectives rather than you planned to proceed into other chapters without really checking do they understand? So, assessment helps us check whether learners understand and whether we can proceed to the next level of learning.’ (SCH2-PrInt-MT2)

Informing and clearly stating the LIs and SC to the learners in the classroom offers a strong starting point for teachers to plan their lesson with effective teaching techniques that support learners’ activities, integrate and assess their teaching, and aid learners in becoming independent in their learning (Moss & Brookhart, 2019). Teachers were observed in the classroom to see whether they communicated the LIs and SC with the learners. Interestingly, none of the teachers communicated the LIs and SC clearly with the learners when teaching, which identified a gap that needed to be addressed (Bartlett, 2015; Heritage, 2010; Wiliam, 2011).

The scaffolding activities are based on temporal planning to support learners in fostering their level of understanding in learning rather than responses that provide an opportunity to reach reactions from their own learning (Aydeniz, 2009; Nasr, Bagheri, & Sadighi, 2020; Sadler, 1989). Teachers’ teaching plans were examined to know whether the LIs and SC are clearly stated. It is interesting to note that none of the teachers planned their lesson notes; instead, they depended on the Annual Teaching Plan (ATP) provided by the DBE. From the pre interviews, one of the teachers mentioned that planning for the lesson is difficult since the learners in the classroom are not always the same; he gave the following response:

‘Yeah, but it’s always, hum, can be tricky to plan them because the classes are not always the same. [Laughing]. They are not always the same, and the kids are not the same, and even on that same that you think you know it; the class’s mood is not always the same. Yes. Most of the time, because myself, as a teacher, when I am planning a lesson, I don’t plan it down to the detail right.’ (SCH2-PrInt-MT1)

Another teacher stated that his planning and preparation depend on the topics that would be communicated or shared with the learners in the mathematics lesson; he gave the following response: ‘My planning and preparation is based on the topic I want to be introduce to my learners during the period of the lesson’ (SCH6-PrInt-MT2).

Another teacher stated that limited time, overcrowded classes and work overload impeded them not to plan their lesson; he gave the following response:

‘Okay, yeah. Sometimes you will find out maybe you want to assess, but because of the overcrowded classes, lack of time and overloaded work, then you cannot plan for learners.’ (SCH1-PrInt-MT2)

Another teacher stated that the problem they have is time to make up their lesson plan: ‘It’s time. Time is one of our biggest obstacles’ (SCH1-PrInt-MT1).

Based on their teaching plans, teachers’ responses identified some gaps that needed to be addressed in Senior Phase mathematics. More so, teachers could not effectively demonstrate the LIs and SC in their lesson plans and mathematics teaching. According to Moss and Brookhart (2019), teachers need to carefully design a lesson plan that measures the quality of the instructional objective, which directs them to aid learners in accomplishing their learning goals. It is clearly seen from pre interviews, classroom observations and teaching plans that none of the teachers wrote out the LIs and SC in their teaching plans. In addition, none of the teachers communicated the LIs and SC with their learners during mathematics teachings. The fact that teachers did not plan or communicate the LIs and SC with the learners could be due to inadequate training or skills to consistently plan and communicate the LIs and SC with their learners.

Theme 2: Development of mathematics teachers praxeologies

The theme shows the development of the South African mathematics teachers’ praxeologies when designing and implementing MERLO in Senior Phase classrooms. The data were presented through post semi-structured interviews, reflective journals and MERLO items designed by the five Senior Phase South African mathematics teachers and MERLO classroom implementation in the mathematics classroom. The following themes emerged.

Sub-theme 2.1: Design of MERLO items of Senior Phase mathematics praxeology

The sub-theme analyses example of MERLO items designed by Senior Phase South African mathematics teachers. In Figure 1, we show an example of MERLO objects that are framed to communicate LIs and SC with learners in the mathematics classrooms. The data presented, which were created by Senior Phase teachers, also validated their MERLO item selections during the question design process.

FIGURE 1: Example of MERLO assessment items, on fractions.

The example is designed in the mathematics content area of fractions, and the question developed was in MERLO patterns. The teachers mentioned that planning the topic of fractions was a result of learners that were struggling to understand the concept of fractions. One teacher provided the following viewpoint:

‘I believe that the pedagogical tool MERLO is another way of presenting new mathematical concepts to the learners because one of the challenges we face is crossing the bridge from abstract to reality. You know mathematics is one of the difficult subjects to produces contents that means real life to the learners.’ (SCH1-MT1)

This view was supported by Moyo and Machaba (2021):

‘Learners’ definitions of fraction were neither complete nor precise. Particularly pertinent were challenges related to the concept of equivalent fractions that include fraction elements, namely the numerator and denominator in the phase of rational number.’ (p. 1)

During the MERLO implementation time, the mathematical concepts of fraction were also included in their weekly teaching plan. The lesson presentation was done during regular mathematics periods, and the process is discussed in Sub-theme 2.2.

Sub-theme 2.2: Effective communication of learning intentions and sharing success criteria with the learners through mathematics praxeology

This sub-theme analysed the introduction, presentation and communication of LIs and SC with their learners through teachers’ mathematics praxeology. The MERLO assessment items that focused on mathematics praxeology were on the topic of fractions (see Figure 1). Teachers introduced and presented a layout of the lesson and conveyed the LIs. Teachers explained and presented the content of the lesson and demonstrated the knowledge and skills obtained during the MERLO lesson. Teachers also explained and related the concept of fractions on the board by using the MERLO pattern with an equivalent form in a different representation. Teachers explained the terminologies associated with MERLO, such as TS, surface similarity, meaning equivalence and quadrants (Arzarello et al., 2015; Prodromou, 2015; Robutti et al., 2016, 2020a, 2020b).

The following comment is one of the teachers’ interactions in the classroom:

‘I want to do with you the following example. But in our case, because we are following a new method, in our question we are going to call it a target statement. So, our target statement, in this case, it is .’ (SCH2-FT2)

The implication of this analysis was that the teachers introduce the task on the topic of fractions to their learners. The task was further linked to the use of the MERLO pedagogy (technique) by following the criteria of meaning equivalence and surface similarity.

Afterwards, the teacher explained that Q1 would not be included when solving the MERLO question because they are straightforward and effortless to understand. Teachers explained that when starting a MERLO question, Q2 would be used. Teachers seemed pleased that learners were actively involved in class and understood the lesson. Teachers demonstrated that as Q stands for quadrant and that Q2A represents the decimal fraction of 0.75. Teachers demonstrated that another Q2 example, as well as Q3 and Q4 examples, would be illustrated:

‘So here I want us to look ehm, at the board, I want us ehm, I want to call this one because this one and this one, they are written differently. This one is my opening statement which is my target statement. So, I want to call this statement 0.75; I want to call it Q2A. If you look at here in your paper, there is Q1; there is Q2, there is Q3, there is Q4, right. We are not going to use Q1 because Q1 those are very simple questions. So, what we are doing now, we are only beginning at question Q2, not only question Q2 but at quadrant 2 let us call it like that. Because the questions there it has, but I am very happy you people are actually following and understand. So that why I am calling this one 0.75 now because 0.75 is going to make my Q2A. So, I want to make another Q2, I want to make another example of Q2 then I will also make an example of Q3 and Q4.’

The presentation of mathematics praxeology in the classroom, which is the ‘technique’, indicates that teachers further explained that the first Q2 example was indicated as Q2A, but another Q2 example would be indicated as Q2B. Teachers asked learners to look at the board and identify whether the TS of the representation of has a similarity with the picture in Q2B. Most learners were able to identify that the representation of the TS has no surface similarity with Q2B, which represents in a diagram. Teachers asked learners to give a reason why they are not fractions, and the representation of Q2B is in the form of a picture. Another learner said that the representation of the TS is written as a fraction, and the representation of Q2B is in the form of a pie chart. The teacher showed an appreciative word to learners by saying ‘very good, they are not similar’. The teacher encouraged class participation as she asked learners to identify whether the representation of the TS has the same mathematical meaning as the diagram in Q2B. Most of the learners answered ‘yes, they have the same mathematical meaning’. Teachers appeared keen and enthusiastic about how learners were actively involved in answering the question. Teachers also asked learners to represent the diagram in Q2B in the form of a fraction. The majority of the learners indicated that the diagram in Q2B is a representation of . Teachers demonstrated on the board that the representation of the picture form and the representation of a fraction have the same mathematical meaning, but they have no surface similarity (Arzarello et al., 2015; Prodromou, 2015; Robutti et al., 2016, 2020a, 2020b).

The conversation between the teacher and the learners is given below:

Teacher: So am repeating Q2, I am saying, I am giving you something like this because that one is Q2A, but I am giving you Q2B, right.

Teacher: Again, am looking at my target statement and my Q2B, I am saying, are this two similar?

Learner 1: No.

Learner 2: No.

Teacher: Why they are not similar?

Learner 1: Because is written in fraction and other is in a picture.

Learner 2: Because is written in fraction and the other one is written in a pie.

Teacher: But are they having the same mathematical meaning?

Learners: Yes, they have mathematical meaning.

Teachers: Okay, I like the yes answer. So, what is the meaning of this diagram in Q2B in a fraction way?

Learners: It’s .

Teacher: Can you see, it means this one is written in a picture form, and this one is written as a fraction, so they have the same mathematical meaning, but the similarity does not exist.

The teacher demonstrated the Q3 example on the board and stated that Q3 represents . Learners were asked to interpret the representation of and the representation of the TS and to identify whether the representation of and the TS appeared the same. Most of the learners were able to identify that the representation of and the target statement look the same (i.e. surface similarity). The majority of the learners were able to interpret that the representation of and the TS are written in an equivalent form of a fraction. The teacher also asked learners whether the representation of and the TS have the same mathematical meaning. Most of the learners were able to identify that the representation of and the TS do not have the same mathematical meaning. Teachers explained that the representation of and the TS do not have the same mathematical meaning because three-quarters and one-quarter are not the same (Arzarello et al., 2015; Prodromou, 2015; Robutti et al., 2016, 2020a, 2020b).

The exchanges between the teacher and the learners are given below:

Teacher: I am going to make another example; this one is Q3. Can you relate now to our open question, which we called our target statement? Let us relate this two. Are these two seem similar?

Learners: Yes, they are similar.

Teacher: Why?

Learners: Because they are both written in fractions.

Teacher: Are they having the same mathematical meaning?

Learners: No, they don’t have the same mathematical meaning?

Teacher: No right, because this one has one quarter and the other one has three quarters so they do not mean the same.

Teachers demonstrated a Q4 example on the board. Teachers said to learners that Q4 represents 50%. Learners were asked to interpret the representation of 50% in relation to the representation of the TS and to identify whether the representation of 50% and the TS have surface similarity. Most of the learners were able to identify that the representation of 50% and the TS do not look the same (i.e. have no surface similarity). Learner 1 said that the numbers are different. Another learner said that the TS is written in a fraction form, and the representation of 50% is written as a percentage. Some of the learners did not understand the MERLO items but the teachers re-explained (Arzarello et al., 2015; Prodromou, 2015; Robutti et al., 2016, 2020a, 2020b). The statements require learners to identify fractions with equal value from different representation objects.

The conversation between the teacher and the learners is given below:

Teacher: Now let us look at Q4, I am writing her 50%. Let’s relate our Q4 here to our target statement. Are these two similar? Is and 50% similar?

Learners: No, they are not similar.

Teacher: Why they are not similar?

Learner 1: Because the numbers are different.

Learner 2: Because is in fraction and 50% is in percentage.

Teachers also asked learners whether the representation of 50% and the TS have the same mathematical meaning. Most of the learners were able to identify that the representation of 50% and the TS do not have the same mathematical meaning. Teachers made it clear in an explanation that the representation of 50% and the TS do not have the same mathematical meaning and do not have surface similarity (Arzarello et al., 2015; Prodromou, 2015; Robutti et al., 2016, 2020a, 2020b).

The conversation between the teacher and the learners is given below:

Teacher: What about the mathematical meaning, did it carry the same meaning?

Learners: No, they do not have the same meaning.

Teacher: So both numbers do not have similarity and do not have the same meaning. This is Q4 means similarity is no, the same meaning is no. So, the question that says No/No, they fall on the categories of Q4.

Based on the teacher’s mathematics praxeology (i.e. MERLO class implementation), the teacher re-explained how a MERLO question should be answered. Teachers demonstrated the process of relating Q2, Q3 and Q4 to the TS (i.e. open question) by saying:

‘This is Q4, Q4 means similarity is no, the same meaning is no. so the question that says No/No, they fall on the categories of Q4. Let us go to Q3; in Q3, we say the similarity is there (i.e. yes) but not the same meaning (i.e. you say it’s no).’ (Arzarello et al., 2015; Prodromou, 2015; Robutti et al., 2016, 2020a, 2020b)

Learners asked questions that linked to the content of the lesson in the class. Teachers also politely worked around the class to check learners’ activities. This was done to allow learners to better understand what the teacher expects them to know, understand, or be able to do at the end of the lesson. This was also done to establish the SC. Other teaching skills demonstrated by the teachers during the didactic delivery were learners’ involvement, classroom management, and addressing learners who are having learning difficulties and providing immediate feedback to the learners.

The implication of the preceding analysis is that mathematics praxeology is made up of a task which consists of a problem that learners must solve (for example, learners might be asked to convert the representation of fractions to decimals), the technique used, and the more-or-less clear reason for applying it in the mathematics classroom.

According to the MDT framework, the South African mathematics teacher’s praxeology consistently follows the guiding principles when designing MERLO items. It seems that the design of MERLO assessment items process follows the sequential order of TS-Q2-Q3-Q4. Teachers’ mathematics praxeology that is linked to MERLO didactical praxeology provides insight into learners’ conceptual reasoning about fractions by designing an activity that focuses on identifying learners’ strengths and weaknesses. Furthermore, the TS was designated as an open question because ‘teachers acquired the practice of elaborating the TS (i.e., TS) as a statement, graph, or table’ (Arzarello et al., 2015, p. 5). According to Arzarello et al. (2015) and Prodromou (2015), the basis for designing MERLO items is to identify a close link that relates to the concept because it needs a change from old-style interactions into present-day interactions.

Conclusions and recommendations for future research

This study was part of a larger PAR project aimed at investigating how MERLO pedagogy as an assessment strategy can be used to understand teachers’ beliefs and practice of effectively communicating of LIs and SC in the South African Senior Phase mathematics classroom. This approach aimed at developing teachers’ adequate knowledge and skills to design MERLO assessment items independently. Due to the small sample size, transferability to the broader population is not possible. At the initial stage, teachers were examined with regard to their beliefs and assessment practices in the classroom, which informed MERLO intervention. In the second stage, teachers were asked to learn about MERLO items by reading the MERLO handout provided to them, participating in the workshop, and sharing their opinions and views with others. In the third stage, teachers had to design MERLO assessment items on their own to assess learners’ level of understanding of the mathematical concept in Senior Phase. We observed from the teachers’ mathematical praxeology that, even though teachers from different school contexts had separately worked on designing MERLO assessment items, they progressively introduced and communicated similar mathematics praxeology which relates to didactical knowledge with their learners in the classroom. For instance, teachers were able to design and integrate MERLO pedagogy across their lesson plan through the practical component, as well as the theoretical component. We assume that sharing ideas and experiences with others can be justified by invariants of mathematical objects, which remained unchanged after the involvement of designing MERLO assessment items, which appears the same in various countries, and the design of MERLO items was found in all groups because the items possess the same criteria in terms of meaning equivalence and surface similarity. These findings imply that MERLO items can be employed in multiple nations as well as other course syllabi by modifying them to institutional contexts while holding on to their significant structure. The study further suggests the need for a professional learning programme based on MERLO technology in Grades 4–12, post-secondary institutions, and public and private contexts (Etkind et al., 2016; Shafrir, 2020). Future research could involve having more teachers in the South African context due to the small sample size.

Acknowledgements

The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.

L.O.A. is a doctoral student at the University of Pretoria and conceptualised the idea and carried out the research. M.A.G. was L.O.A.’s supervisor for her doctoral degree and, as sole supervisor, was involved with all steps in the research. The themes generated in this article were discussed in detail between the authors to further ensure trustworthiness.

Ethical clearance was obtained from the University of Pretoria Research Ethics Committe SM 19/05/01.

M.A.G.’s research was funded by the National Research Foundation (NRF) [reference: CSRP190415430728, grant number: 120401].

The transcriptions are not made available publicly to protect the participants; this is done to ensure anonymity and confidentiality.

The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of any affiliated agency of the authors.

References

Footnote

1. The Senior Phase in the South African schooling system is Grades 7 to 9; the interested reader is referred to southafricaeducation.info (2021) for a layout of the South African education structure.