Frontline Learning Research vol. 7 No 3 (2019) 64
- 90
ISSN 2295-3159
aFaculty of Mathematics and Physics, Charles
University, Czech Republic
b Faculty of Arts, Charles University, Czech
Republic
c Institute of Psychology, The Czech Academy of
Sciences, Czech Republic
b Film and TV School, Academy of Performing Arts in
Prague, Czech Republic
Article received 17 February 2019/ revised 20 June/ accepted 19 July / available online 16 August
Game-based learning is supposed to motivate learners. However, to what degree does motivation driven by interest in playing an instructional game affect learning outcomes compared to motivation driven by interest in the very learning process? This is not known. In this study with a unique design and intervention, young adults (N = 128; a heterogeneous sample) learned how to control an electro-mechanical device in a 40-minute-long learning session integrated into a 2-hour-long educational live action role-playing game (edu-LARP). Edu-LARPs are supposedly engaging games where players take part in team role-playing by physically enacting characters in a fictional universe. In our edu-LARP, players had to understand how the to-be-learned device worked in order to win the game. Departing from typical game-based learning research, learning- and playing-related variables were assessed for each learner separately (i.e., a within-subject design). Affective-motivational factors related to playing (rather than learning) predicted learning outcomes in a positive, but considerably weaker, way compared to learning-related, affective-motivational factors. Developed interest in LARP-like games was primarily related to enjoying the game rather than better learning outcomes; whereas, developed interest in the instructional domain was primarily related to enjoyment of learning and better learning outcomes. Overall, autonomous motivation to play was connected to higher learning outcomes, but this connection was weak.
Keywords: game-based learning; autonomous motivation; learning-outcomes; developed interest; live action role-playing games; edu-LARPs
Game-based learning experiences are widely supposed to boost the autonomous motivation of learners. Autonomous motivation refers to doing an activity for its own sake or for its perceived personal importance (Vansteenkiste et al., 2009). Higher autonomous motivation is supposed to increase cognitive engagement and thereby facilitate learning (e.g., Moreno, 2005; see also Cordova & Lepper, 1996; Grolnick & Ryan, 1987). However, to what degree does motivation driven by interest in playing an educational game impact learning outcomes? Is the influence of this motivation on learning outcomes higher, lower or the same compared to how much motivation driven by interest in the instructional topic impacts learning? This research question is addressed in this study.
Participants played a 2-hour-long educational live action role-playing game (edu-LARP) with a sci-fi plot, that included a 40-minute-long learning session organically integrated into the middle of game play. Edu-LARPs are supposedly engaging game-based learning experiences that emphasize team role-play and an element of physical enactment (Bowman, 2014; Bowman & Standiford, 2015; Hyltoft, 2010; Montola, 2008; Vanek & Peterson, 2016). To win the game, players had to learn (in the integrated learning session) how to control an electro-mechanical device. The device was fictitious, having meaning only in the game context (to minimize the influence of prior knowledge). From an educational perspective, learning how the device worked represented general mental models acquisition in science, technology, and engineering contexts.
Participants included young adults having varying degrees of developed interest in sci-fi LARP-like games and developed interest in electro-mechanics/ICT. Developed interest refers to a relatively enduring predisposition to re-engage with particular types of content over time (Hidi & Renninger, 2006). The edu-LARP was designed to appeal primarily to those with the former developed interest. The learning session and the device were designed to engage primarily those with the latter developed interest. The reason for this was to create two distinct drives of autonomous motivation: one being interest in playing and the other interest in the instructional domain.
Enjoyable instructional experiences may not only increase autonomous motivation; they can also cause greater cognitive load (e.g., Mayer, 2014; Rey, 2012; Um et al., 2012). Cognitive load refers to the amount of mental activity imposed by the educational experience on working memory (Sweller, Ayres, & Kalyuga, 2011). High cognitive load, especially if triggered by too complex or distractive elements in the learning environment, has opposite effects on learning processes than does motivation. It can overwhelm limited working memory resources and thereby hamper learning (e.g., Sweller, Ayres, & Kalyuga, 2011; see also Um et al., 2012).
In this study, we measured autonomous motivation as well as cognitive load variables. We measured them twice during the game: with respect to game play (i.e., before the learning session) and with respect to the learning (i.e., after the learning session). We contrasted how motivation to play versus motivation to learn, and game-engendered versus learning-engendered cognitive load, influenced learning outcomes (i.e., within-subject design). Learning outcomes were measured both after the edu-LARP and a month later. We also examined whether developed interest in the instructional domain, and in the game, affected the two autonomous motivations, two cognitive loads, and learning outcomes. Finally, we explored whether the (possible) relationship between the two developed interests and learning outcomes was mediated by affective-motivational or cognitive load variables.
There are many definitions of games. In this paper, we understand game in the terms of Jull’s definition (2003): as a rule-based system with variable, quantifiable outcomes, in which actors (i.e., players) have the possibility to “attach” themselves intrinsically to different outcomes of the playing process. At the same time, they can influence the game state while working toward the goal (with or without real-life consequences). Players must make an effort to influence the game state. One of the key drivers for exerting such effort is autonomous motivation to play the game.
Various types of learning exist; for instance, skill training, learning of facts, or mental models acquisition. In this study, we focus on mental models acquisition, as understood in constructivist frameworks (e.g., Mayer, 2009). Mental models are knowledge structures that represent processes and/or systems and that enable drawing inferences about the processes/systems. These structures are built in learners’ minds within the context of knowledge structures previously acquired. The process is not automatic: Learners have to exert mental effort to do so.
One of the key premises of game-based learning is that motivation to play the game can positively influence learning processes and thereby enhance learning outcomes. How exactly motivation to play should do this (and whether it can actually do this, and if so, how strongly) has rarely been examined in game-based learning literature. This is discussed in detail in the next section. For example, when can effort invested into playing (driven by autonomous motivation to play) be transferred to effort invested into learning? This is not known. The present study examines one possible mechanism through which motivation derived from playing can enhance learning, as outlined in Section 2.4.
The game-based learning field has been dominated by digital games. However, other approaches, such as edu-LARPs, are also becoming popular (see Bowman, 2014). As said above, the motivational potential of game-based learning has been generally assumed, but evidence substantiating it is limited. For example, digital games slightly enhance learning compared to traditional instructional approaches (e.g., meta-analyzed in Clark, Tanner-Smith, & Killingsworth, 2016; Wouters, van Nimwegen, van Oostendorp, & van der Spek, 2013), but the extent to which they do so through affective-motivational factors is unclear. Many studies included in the meta-analyses have not researched affective-motivational factors (see Sitzmann, 2011; Wouters et al., 2013). The affective-motivational dimension has thus been omitted from most meta-analyses. Wouters and colleagues, who did examine the effects of games in this dimension, reported that instructional games are more motivational compared to traditional types of education, but the difference only approached significance. 1 Narrative reviews of game-based learning literature (e.g., Bowman, 2014; Boyle et al., 2016; Jabbar & Felicia, 2015) also did not provide information about how much game-derived motivation influences learning outcomes.
Claims about the possible influence of game-derived motivation on learning outcomes are substantiated by experimental studies that have examined the effects of individual game design elements (see, e.g., Clark et al., 2016; Wouters & Oostendorp, 2017). Some of these elements have been shown to elevate affective-motivational factors as well as enhance learning outcomes. These include personalization and choice (Cordova & Lepper, 1996), intrinsic integration of the learning content with game mechanics (Habgood & Ainsworth, 2011), and team role-playing activities (Brom, Šisler, Slussareff, Selmbacherová, & Hlávka, 2016).
However, correlational studies examining the strengths of associations between game-derived affective-motivational factors and learning outcomes have offered a mixed picture (e.g., a negative influence: Iten & Petko, 2014; a positive influence: Sabourin & Lester, 2014). The reason behind this ambiguity could be that the motivational–learning correlations have been confounded by contextual factors, most notably, by different levels of cognitive load caused by different game designs. Even in comparative studies, experimental and control conditions could differ in the levels of cognitive load (often uncontrolled) imposed upon learners. To take the next step in answering the question on the strength of game-derived motivation’s learning impact, it would be useful if motivational effects triggered by an educational game were contrasted with effects triggered by an unquestioned, robust, “baseline” motivational factor while all participants were undergoing the same intervention. This would make levels of cognitive load dependent on differences between individuals rather than between contextual factors. What should this robust “baseline” be?
High interest in a learning domain, and/or in an instructional topic, is straightforwardly implied by theories of motivation and interest in enhancing learning processes (e.g., see Eccles & Wigfield, 2002; Hidi & Renninger, 2006; Keller, 2010). The positive effects of this interest on motivation to learn and learning outcomes have been repeatedly demonstrated (e.g., Brom et al., 2017; Fulmer, D’Mello, Strain & Graesser, 2015; Schiefele & Krapp, 1996; Schiefele, 1999). Therefore, we used motivation driven by interest in the learning domain as our “baseline”. We contrasted effects of the “baseline” motivation on learning outcomes with the effects of motivation to play the game. The latter motivation was triggered using one of the approaches having been previously shown to be instrumental in doing this: team role-play activities in an edu-LARP.
Edu-LARPs derive mostly from leisure time social role-playing games (Bowman, 2014). Social role-playing games emphasize players’ interaction, which distinguishes them from single-player role-playing games. Social role-playing games are organized around a modeled scenario/narrative. In edu-LARPs, players enact this scenario physically, unlike in table-top and digital role-playing games.
Edu-LARPs present an old educational approach (Kot, 2012; Vanek & Peterson, 2016), which recently saw a surge in interest (see Bowman, 2014). They have been utilized for a range of curricular objectives and they are supposed to have various instructional benefits (Bowman, 2014; Hyltoft, 2008). For instance, they seem to be useful when a model scenario is to be re-enacted: such as for skills-training (e.g., Hayden, Smiley, Alexander, Kardong-Edgren, & Jeffries, 2014) or understanding complex, socio-historical relationships (e.g., Brom et al., 2016; Mochocki, 2014). Motivational potential is yet another possible advantage of edu-LARPs (e.g., Bowman & Standiford, 2015; Vanek & Peterson, 2016).
As in the case of digital game-based learning, researchers are still developing empirical evidence on edu-LARPs’ effectiveness (Bowman, 2014). The key evidence substantiating the claim about the motivational potential of edu-LARPs comes from the experimental study with a large sample by Brom and colleagues (2016), who demonstrated that learning outcomes and affective-motivational variables were enhanced when high school students learned through an edu-LARP compared to a discussion-based control without the role-playing game element. The affective-motivational variables partially mediated the game’s positive effect on learning outcomes. Supplementary evidence for the motivational potential of edu-LARPs comes from anecdotal reports (see Bowman, 2014; Bowman & Standiford, 2015).
Different learners are interested in different things. Learners’ interests may influence how much they will be motivated to study a particular topic or learn using a game-based approach. For example, not all learners are equally motivated by edu-LARPs. Some learners do not want to get involved in role-play (Vanek & Peterson, 2016); especially those with low prior experience with this game format (Mochocki, 2014). Learners with applied study backgrounds found an edu-LARP approach more appealing than learners with social sciences backgrounds (Brummel et al., 2010). Whereas some learners enjoyed an edu-LARP experience, others were stressed by the necessity to interact socially during the game (Brom et al., 2014).
On a theoretical level, these ideas can be embraced using the notion of well-developed individual interest (called developed interest here for brevity). According to the four phase model of interest development (Hidi & Renninger, 2006), this is the most enduring form of interest: a relatively stable predisposition to re-engage with particular types of content. Two developed interests are important here. The first one is interest in sci-fi LARPs and similar games and game-like experiences (hereafter also called gamer scores). The second one is developed interest in the instructional domain/topic, i.e., ICT/electro-physics (hereafter also called techie scores).
How can gamer scores and techie scores be theoretically connected to motivations to play and/or to learn? How can these motivations be theoretically linked to learning outcomes? (We note that we do not focus here on leveraging general school motivation, but on motivation and learning outcomes related to a particular learning experience.)
A useful way of organizing different forms of motivation provides a framework, which differentiates between autonomous and controlled motivations (Ryan et al., 2006; Vansteenkiste et al., 2009). This framework is based on self-determination theory (Deci & Ryan, 1985). Autonomous motivation, which is of present interest, is characterized by an internal perceived locus of causality (deCharms, 1968; Ryan & Deci, 2000): Learners perceive this motivation as originating within themselves. (Controlled motivation is characterized by an external perceived locus of causality.)
Autonomous motivation is the desired type of motivation (Deci & Ryan, 2008), because it is linked to several advantages: including better learning outcomes (e.g., Cordova & Lepper, 1996; Grolnick & Ryan, 1987; Vansteenkiste et al., 2005; see also Schiefele, 1999; Vansteenkiste et al., 2009). We share this assumption here (General Prediction). Within self-determination theory, autonomous motivation has two subcomponents (Vansteenkiste et al., 2009; see also Ryan & Deci, 2000): intrinsic motivation that refers to doing an activity for its own sake (because it is inherently enjoyable) and identified regulation, a form of extrinsic motivation that refers to doing an activity because of its perceived importance.
Self-determination theory maintains that autonomous motivation is fostered when learning environments facilitate learner satisfaction of needs for autonomy, competence, and relatedness (Deci & Ryan, 1985; Vansteenkiste et al., 2009). As concerns intrinsic motivation to learn, the need for competence would be satisfied more often among persons with a high developed interest in the learning domain (i.e., techies) than for those with a low developed interest therein (i.e., non-techies). The reason is that techies would typically feel more competent in solving the learning task than non-techies. Therefore, self-determination theory predicts that techie scores will positively relate to intrinsic motivation for learning how an electro-mechanical device works (Prediction SDT1a; SDT = self-determination theory). Consequently, the techie scores will positively relate to learning outcomes (Prediction SDT1b).
Based on self-determination theory, intrinsic motivation to play would be hampered when a game undermines one of the above needs: it will be lower for those who find it hard to play an edu-LARP (competence) or who feel uncomfortable during role-playing/social interaction (relatedness). Intrinsic motivation to play will thus positively relate to the gamer score (Prediction SDT2a).
As concerns identified regulation to learn, the following applies: High autonomous motivation to play, presumably more prevalent among gamers, can be transferred to identified regulation to learn, because players motivated to play may invest more into learning (they feel it is important as part of the game). Consequently, the gamer score should relate positively to learning gains (Prediction SDT2b). However, the gamer scores → learning outcomes link may be weaker than the techie scores → learning outcomes link, because it is not guaranteed that motivation to play would project to identified regulation to learn for all participants (Prediction SDT3).
Cognitive load theory (Sweller, Ayres, & Kalyuga, 2011) is a theoretical framework based on a model of human cognitive architecture, which enables educational designers to construct instructionally efficient learning environments. Its key assumptions are that working memory has limited capacity and duration; whereas, long-term memory serves for permanent storage with unlimited capacity and duration. During learning, incoming information is first represented in working memory and eventually integrated with pre-existing knowledge structures in long-term memory. These knowledge structures also organize temporary representations in working memory: More advanced structures enable the representation of incoming information using fewer information elements.
Following its recent adjustment (Kalyuga, 2011), cognitive load theory posits two types of working memory load: intrinsic and extraneous. Learners must allocate cognitive resources to deal with these loads. If they fail to do so, or if total load overwhelms working memory resources, learning is hampered.
Intrinsic load is imposed on learners by the complexity of the learning task. Intrinsic load is essential for comprehending the learning message: Dealing with it results in learning. In determining the task’s complexity, one has to consider learner’s prior knowledge (i.e., knowledge structures in long-term memory prior to learning). What is complex for a novice may not be complex for an expert.
In this study, learners learn to operate a fictitious electro-mechanical device. Prior knowledge of the device is null for everyone, but techies will possess high-quality knowledge structures concerning certain general electro-mechanical concepts; e.g., “electrical signal”. These structures will enable them to represent information in their working memory more efficiently. Within cognitive load theory, this means techies will have lower intrinsic load (Prediction CLT1a; CLT = cognitive load theory). Therefore, it is less likely that techies’ working memory would be overloaded compared to non-techies. Consequently, in agreement with Prediction SDT1b, techie scores will be positively related to learning outcomes (Prediction CLT1b).
Extraneous load is caused by the processing of sub-optimally designed features of learning environments. It should be minimized, because accommodating it depletes cognitive resources that could otherwise aid in dealing with intrinsic load. Extraneous load can arise from two sources within a game-based learning environment: a) from the sub-optimal design of the instructional content embedded in the game and b) from game play as such. Those with high techie scores will likely cope better with possible sub-optimal design, which adds weight to Prediction CLT1b. As concerns the game-related source of extraneous load, a portion of players’ cognitive resources will be devoted to thinking about playing the game. These thoughts will deflect learners’ attention away from learning. Game-related, but learning-irrelevant, thoughts are likely to be amplified for non-gamers, who do not yet have well-developed game-related schemata/skills. Therefore, game-engendered extraneous load will be negatively related to gamer scores (Prediction CLT2a). Because it may cause cognitive overload and hamper learning, gamer scores will relate positively to learning outcomes (Prediction CLT2b).
This study examines how much autonomous motivation driven by developed interest in an educational game influences learning outcomes compared to motivation driven by developed interest in the instructional domain (i.e., within-subject design). Autonomous motivation is referred to hereafter as motivation for the sake of brevity; the first motivation is referred to as motivation to play and the second one as motivation to learn. The two developed interests are called gamer and techie scores, respectively. The study also investigates how these two scores affect motivation to play, motivation to learn, overall game enjoyment, difficulty of game play (as a proxy variable to game-engendered extraneous cognitive load), cognitive loads engendered by the learning experience, and learning outcomes.
Directional hypotheses:
H1: The techie scores will relate
• (H1a) positively to motivation to learn (based on Prediction SDT1a);
• (H1b) negatively to cognitive loads engendered during learning (Prediction CLT1a);
• (H1c) positively to learning outcomes (Predictions SDT1b and CLT1b).
H2: The gamer scores will relate
• (H2a) positively to motivation to play and also overall game enjoyment (Prediction SDT2a);
• (H2b) negatively to difficulty in playing the game (Prediction CLT2a);
• (H2c) positively to learning outcomes (Prediction SDT2b and CLT2b).
The directional hypotheses concerning influences of the techie and gamer scores are summarized in Table 1. There relationships for the remaining pairs of variables (i.e., in columns and rows from Table 1) will be explored (Exploratory Goals E1, E2).
With respect to the influences of both motivations on learning outcomes, the following hypotheses will be examined (Table 2):
H3
• (H3a) motivation to play will be positively related to learning outcomes (Prediction SDT2b and General Prediction);
• (H3b) motivation to learn will be positively related to learning outcomes (General Prediction).
The link between cognitive loads and learning outcomes will be explored (Exploratory Goal E3). We put forward these mediation hypotheses (Table 3):
H4: The relationship between techie scores and learning outcomes will be mediated
• (H4a) positively by motivation to learn (Prediction SDT1a and SDT1b);
• (H4b) negatively by cognitive loads evoked during learning (Prediction CLT1a and CLT1b).
H5: The relationship between gamer scores and learning outcomes will be mediated
• (H5a) positively by motivation to play and also overall game enjoyment (Prediction SDT2a and SDT2b);
• (H5b) negatively by difficulty in playing the game (Prediction CLT2a and CLT2b).
Finally, we will explore (E4) whether the relationships between a) gamer scores and learning outcomes and b) motivation to play and learning outcomes is weaker compared to complementary relationships between (a) techie scores or (b) motivation to learn and learning outcomes (cf. Prediction SDT3).
Table 1
Hypotheses/exploratory goals related to techie and gamer scores
a Indexed by proxy variables.
Note: + positive relationship expected; – negative relationship expected; ? no expectation.
Table 2
Hypotheses/ exploratory goals related to how variables assessed in situ predict learning outcomes
aIndexed by proxy variables.
Note: + positive relationship expected; ? no expectation.
Table 3
Hypotheses/exploratory goals related to mediation
aIndexed by proxy variables.
Note: Expected relationship: + positive; – negative.
Participants were recruited from university pools and via the Facebook pages of LARP/sci-fi communities and using short-term job advertisement servers. We emphasized that we were seeking both novice and seasoned LARP players. Participants received financial compensation (400 CZK, ~15 EUR).
Prospective participants completed an online questionnaire, which provided demographic data (Mage = 24.7; SDage = 3.72) and data on prior LARP-related experience. We invited selected participants for one of 11 game runs, such that people with both low and high prior LARP-related experience and with diverse study/employment backgrounds (see Suppl. Mat. A) participated in every run (10-13 participants per run). Ultimately, 128 participants were included in the analysis. Two additional participants were excluded due to health issues. Seventeen participants were excluded from the analyses of delayed learning outcomes data because they did not attend delayed testing session.
The sample was heterogeneous with respect to two techie and gamer scores (Figure 1). These two variables also did not correlate (r = .06); i.e., we recruited techie gamers, techie non-gamers, non-techie gamers, and non-techie non-gamers.
Figure 1. Participants’ techie and gamer scores.
Edu-LARP. We designed the LARP as a 2-hour-long game that can teach a scientific topic by means of an embedded learning session. The LARP was a sci-fi space opera with a plot created by a seasoned LARP script writer. The story started in the midst of a journey on a generation spaceship. The players took on roles of technical school students therein. Certain events triggered a mutiny, during which the fighting parties damaged input cables to a device for controlling correction thrusters (i.e., sideways-pointed motors used to make small corrective movements when a spaceship is already in space). This left the ship on a collision course with an asteroid. Only one access route to the device remained passable: it led from the technical school via an escape corridor. The players had to learn how the device works (Figure 2, 3), locate it, set it to manual control, and tweak its cables to avoid hitting the asteroid and save the ship’s population.
The complexity of roles was determined during pilot experiments. The roles were less complex than in a typical LARP for seasoned players, but still relatively complex for novice players (i.e., a compromise).
Figure 2. A: Model of the device available during the learning session. B: Rewired cabling. C: The actual device. D: Schematic drawing of device.
Figure 3. A: Players discussing game options. B: Players interacting during the learning session. C: Players rewiring cabling on a model.
In the middle of the game, the teacher (game master) initiated a 40-minute-long learning session, during which he taught the students how to operate the device (Figure 3b, c). The plot was designed so that all players were motivated to learn this (see Suppl. Mat. B for details). The players worked with functional models of the device. They did not have access to these models outside the session, and the teacher never commented on the device’s functioning, except during this session. A clear beginning and end point for the session also enabled the teacher to administer questionnaires during the game at well-defined moments.
Toward the end of the game, the players gained access to the actual device. They had 9 minutes to tweak it in order to correct the ship’s course.
Teaching session. This was a 40-minute-long frontal lecture with slides interspersed with teacher-guided hands-on-practice segments. During the lecture, players stayed in their game roles the whole time. Each player was given the device’s model and schematic drawing (Figure 2a, 2d). This was the first time they saw the model/device. To solve the final game task and win the game, players had to understand how the device works, i.e., acquire its mental model (mere superficial memorization was insufficient).
The model and the learning task. The key part of the model/device, the so-called control calculator a) controlled whether the ship’s course correction could be made given the current status of the correction thrusters and b) converted the course correction given in angular units to the force of the thrusters. Input wiring to the calculator relayed a signal from the deck carrying the correction command and status reports from the thrusters. Output wiring carried signals to the thrusters with force data and a command (either to perform the maneuver or test if it was possible to carry out the maneuver). The device featured a manual control which could partially override commands from the deck. With a sufficient level of understanding, one could re-wire the cabling (Figure 2b, 3c); by-passing certain calculations and/or changing the partial override mode to full override mode (see Suppl. Mat. C for further details). It was necessary to do these steps in solving the final task.
The actual device and the final task. The device was similar to the model (Figure 2c): but larger. It had a similar layout, but different graphics. It contained a timer that counted down the time to the ship’s final maneuver. During this time (9 minutes), the players, as a group, had to tweak the cabling and set the ship’s new course. This was a near transfer task with respect to the tasks assigned in the learning session.
We faced a challenge because instruments assessing relevant constructs in the context of edu-LARPs were lacking. Also, the instruments needed to be short; especially those to be used during the game. Therefore, we adjusted several instruments from neighboring research fields, established face validity, and fine-tuned the questions during pilot experiments. All questions and scales are detailed in Suppl. Mat. D.
4.3.1 Participant variables
Demographic data. When expressing interest in participating in this research, participants reported online their gender, age, prior LARP-related experience, study background, and possible employment. Prior LARP-related experience was measured using six questions we developed, which assessed participants’ experience with various types of LARP-like games.
Techie scores. This variable should reflect participants’ developed domain interest. Such interest was, in the present case, related to developed interests in ICT and electro-physics. It was also related to participants’ study types, as study type can generally be assumed to reflect the person’s interests. Therefore, the techie score was computed as a weighted sum of these two interests and a score assigned based on the participant’s study type (see Suppl. Mat. E for the exact equation). Developed interest in electro-physics (4 items; α = .87) and in ICT (4 items; α = .89) was assessed based on work done by Renninger and Schofield (2014). A score for the participant’s study type was assigned based on a rubric detailed in Suppl. Mat. E. ICT and electro-physics developed interests were strongly inter-correlated (r = .68) and moderately-to-strongly correlated with the study type score (r = .42, .44).
Gamer scores. This variable should reflect participants’ developed interest in the type of games exemplified by our intervention; i.e., a sci-fi LARP. It was computed as a weighted sum of participants’ prior LARP-related experience (as this reflects voluntary experience with LARP-like games) and developed sci-fi interest (α = .73; see Suppl. Mat. E for details). Prior LARP-related experience was reported online when participants expressed interest in our research (see above). Sci-fi developed interest was assessed similarly to the ICT/electro-physics developed interest (i.e., based on Renninger & Schofield, 2014) (5 items; α = .95).
4.3.2 Dependent variables
Autonomous motivation: positive affect, flow, and learning enjoyment. Motivations to play and to learn were examined separately. Because it would be difficult for learners to distinguish between these two motivations after the game ended, we assessed them in situ: at appropriate moments during the game play but without disrupting the play (i.e., using “gamified” questionnaires). We measured motivation to play through two proxy variables: flow and generalized positive affect (referred to hereafter as positive affect). We measured motivation to learn through three proxy variables: flow, positive affect, and learning enjoyment. Positive affect is related to various positively-valenced, activating feelings (e.g., excitation, activity). We measured it using Positive and Negative Affect Schedule (i.e., PANAS; Watson et al., 1988) (α = .82 – .93).2 Flow refers to pleasant absorption of an activity that one takes part in (Csikszentmihalyi, 1975). We measured it using three items from the Flow Short Scale (Rheinberg et al., 2003) (α = .85 – .89). Learning enjoyment was assessed using two questions from the Interest/Enjoyment subscale of the Intrinsic Motivation Inventory (McAuley et al., 1989) (r = .88). Only subsets of questions were used for brevity.
Learning-engendered cognitive load. In addition to learning-related motivation, we measured learning-engendered intrinsic load (2 items; r = .82) and extraneous load (3 items; α = .78) using questions adopted from the questionnaires by Leppink and colleagues (2014) and Naismith and colleagues (2015).
Overall game difficulty, overall game enjoyment. As a proxy variable to game-engendered extraneous load, overall game difficulty was measured using three items we created (α = .84). Overall game enjoyment was assessed with 10 questions we created (α = .89); based on motivation/enjoyment items from other questionnaires (e.g., McAuley et al., 1989; Schraw et al., 1995). Items were tailored for the specifics of LARPs.
Retention test. It had one question: “Draw a diagram of the device for controlling the ship’s correction thrusters showing all elements of the device”. A point was awarded for correctly drawing an element, for correctly positioning it with respect to other elements, for correctly naming it, and for correctly drawing a cable crossing (scale: 0 – 80). Two independent raters scored the answers with a nearly perfect agreement (immediate: Weighted Cohen’s κ = .995; delayed: κ = .999; Cohen, 1986).
Transfer test. We developed two complementary versions of the transfer test (for immediate versus delayed testing; counterbalanced across participants). One version had four and the other five open-ended questions, e.g., “Imagine that the spaceship does not have three correction thrusters, but four instead. What changes would you have to make to the device controlling the correction thrusters in order for the device to function with four thrusters?”. Participants were awarded 1 point for each correct solution or 0.25 or 0.5 points for a partially-correct solution (scales: 0 – 26 and 0 – 27, respectively). Two raters scored the answers with a substantial agreement (immediate: κ = .977; delayed: κ = .968).
Raters’ scores were averaged for the subsequent analysis. Prior to averaging, transfer test scores were z-transformed for each version of the test to obtain comparable values.
Participants received general LARP rules, a description of the setting, and brief descriptions of all roles in advance. They selected online which roles they would prefer to play.
Upon arrival, participants filled in the initial questionnaire (see Figure 4 for the experimental schedule). Afterwards, a warm-up period started: the LARP rules were recapitulated and participants could decorate the rooms with supplied thematic set pieces. Participants were then assigned roles (based on the preferences they had previously expressed), read their descriptions, and introduced their roles to fellow players.
Figure 4. Experimental schedule.
Afterwards, the game started. The teaching session started about an hour into the game. Before it began, the teacher distributed the game-related flow and PANAS questionnaires (with a cover story about the ship’s supreme inspectorate evaluating the quality of his teaching). After the lecture, the teacher distributed the questionnaire that yielded post-learning flow, positive affect, enjoyment, and intrinsic/extraneous cognitive load data. After roughly 20 minutes, players eventually found access to the escape corridor and located the device, wherein they solved the final game task.
Participants then filled in the final questionnaire, which primarily yielded overall game enjoyment and difficulty assessments. Afterwards, retention and transfer tests were distributed. Finally, a game debriefing and an interview were organized.
About three weeks after the LARP, participants arrived for the delayed testing session. They were given retention and transfer tests and developed interest questionnaires.
Post-learning positive affect and flow tapped at-the-moment experienced affective-motivational states. These states could be influenced not only by learning, but also by game play that preceded the learning session (this was not the case of enjoyment/cognitive load scales, because the respective questions referred to the learning session as such, see Suppl. Mat. D). We wanted to use, as proxies to motivation to learn, positive affect/flow-derived variables that satisfied two requirements: They were a) related to pre-/post-learning change in positive affect/flow and b) independent of pre-learning positive affect/flow. Therefore, learning-related positive affect/flow were computed as pre-/post-learning positive affect/flow residual differences (by regressing the post-learning positive affect/flow on the pre-learning positive affect/flow).
The means and averages of the dependent variables are included in Suppl. Mat. A. The correlation matrix is also provided therein.
We used multiple linear regressions with two independent factors: gamer and techie score. When entered together with gamer scores into the models, techie scores modestly predicted motivation to learn (Table 4), so Hypothesis 1a was supported. Techie scores strongly predicted cognitive load induced by learning (in the negative direction) and learning outcomes, so Hypotheses H1b and H1c were also supported.
As concerns Exploratory Goals E1a and E1b, techie scores were unrelated to game-induced flow and positive affect. However, they were modestly related to game enjoyment. This can be explained by the fact that game enjoyment was measured after the LARP ended, so it was arguably also influenced by liking the learning session (unlike game-induced flow/positive affect measured before the learning session started). Techie scores were unrelated to perceived game difficulty.
We conclude that techies liked the learning session more (compared to non-techies), it was easier for them to learn, and they learned better. However, playing a LARP was not more motivating for them.
Gamer scores strongly predicted motivation to play, overall game enjoyment, and (in the negative direction) game difficulty (Table 4). Hypotheses H2a and H2b were thus supported. Gamer scores were modestly related to learning gains (except for immediate retention). Hypothesis H2c was thus partially supported.
As concerns Exploratory Goals E2a and E2b, gamer scores were unrelated to motivation to learn and learning-engendered cognitive loads. We conclude that participants with higher gamer scores were relatively more motivated to play the LARP and playing was easier for them. They also learned slightly better. However, gamer scores were not showed to be connected to motivation to learn and cognitive load.
Table 4
Standardized beta coefficients for a multiple regression model (predictors: techie/gamer scores)
Note: PA = positive affect. Hypotheses: ✓ supported; [✓] partially supported. Relationships found: ○ no; + positive; – negative.
a Pre-post residuum.
*p < .05 **p < .01 ***p < .001
Which in situ measured variables predicted learning outcomes? The effects of the following variables were investigated: game-induced positive affect/flow, learning-induced positive affect/flow, learning enjoyment, intrinsic/extraneous load. To facilitate interpretation, we reduced the number of variables using exploratory factor analysis. The Kaiser-Meyer-Olkin adequacy was .65 (above the recommended value .6) and Bartlett’s test of sphericity was significant (p < .001). Factors were extracted using the Ordinary Least Squares method with varimax rotation. Both scree plot analysis and parallel analysis suggested the presence of three factors. These factors corresponded to our three umbrella constructs: motivation to play, motivation to learn, and cognitive load; and the factors were thus labeled so. In our main analysis, we computed regression models with the newly-created factors (all three factors where entered together into each model)3.
Both Motivation factors significantly predicted transfer; however, only Motivation to learn predicted also retention (Table 5). Hypothesis H3b was thus supported and H3a was supported only as concerns transfer.
Table 5
Hypotheses, predictions, and exploratory goals related to the motivation–learning link
Note: Hypothesis: ✓ supported; ( ✓ ) partially supported; X not supported. Relationship found: – negative.
aThe factor to which game-induced positive affect/flow primarily load.
bThe factor to which learning enjoyment/positive affect/flow primarily load.
cThe factor to which intrinsic/extraneous load primarily load.
As concerns Exploratory Goal E3, cognitive load predicted well all learning outcome variables in the negative direction. As concerns Exploratory Goal E4, gamer scores as well as motivation to play were weaker predictors of learning outcomes compared to techie scores and motivation to learn (and cognitive load). However, they still played certain roles (Tables 4, 5).
We conclude that participants with higher motivation to learn and/or with lower cognitive load learned better than those with lower motivation and/or higher cognitive load. Also, those motivated to play the game performed somewhat better than those less motivated to play, but only on transfer test tasks.
We used the package mediation (Tingley et al., 2014) for causal mediation analysis. We computed estimates for indirect effect using quasi-Bayesian Monte Carlo Simulations (N = 10,000) (Preacher & Hayes, 2004).
The relationship between techie scores and learning outcomes was mediated both by motivation to learn and cognitive load engendered during learning (Table 6). Therefore, Hypotheses H4a and H4b were supported.
The relationship between gamer scores and transfer was mediated by overall game enjoyment and, for immediate transfer, marginally mediated by overall game difficulty. No other variable was confirmed as a mediator (p > .106). Hypothesis H5a was thus supported only with respect to transfer and overall game enjoyment, and Hypothesis H5b with respect to immediate transfer.
Table 6
Mediation analysis
Note: CI = confidence interval. Hypothesis: ✓supported; [✓] partially supported; X not supported.
†p < .10 *p < .05 **p < .01
aThe FA factors
We inspected negative evaluations of the LARP and whether motivation to learn transformed to identified regulation to play, as gauged by participants. To summarize these results, we split participants into four groups based on median split of transfer (average value of immediate and delayed transfer test scores; i.e., high vs. low transfer) and motivation to play (average value of Z-scores from game-induced flow and positive affect; i.e., high vs. low motivation). Sample statements from the subgroups’ participants are shown in Table 7. Generally, qualitative data showed: i) higher learning outcomes were not necessarily connected to excitation from the game; ii) some learners (around 20% of the sample) did not like the approach; iii) some participants (around 15%) claimed the game had a substantial positive motivational effect on their learning.
Table 7
Sample statements during interviews and participant interests
aActual range.
bTwenty-seven participants could not be classified as hard vs. social science students or had partly missing data.
For control purposes, we also conducted a supplementary study, in which we tested how much participants would learn from the same 40-minute-long learning session as embedded in the LARP: but outside the game. We recruited 48 participants to match a selected sub-sample from the main study (i.e., a quasi-experimental comparison without randomization). Participants were matched based on their study backgrounds (see Suppl. Mat. G for further details). At the beginning of the learning session, these non-game participants were told that they should imagine they are students on a generation ship (to contextualize the device). The same learning outcomes and autonomous motivation measures were used.
Positive affect of non-LARP learners measured immediately after the narrative introduction, i.e., immediately before the learning session, was significantly lower compared to positive affect of the matched LARP participants measured in the game, before the learning session (d = 0.45). This demonstrates a medium effect size motivational advantage of the edu-LARP (see Suppl. Mat. G for descriptive data and analyses). Motivation to learn (i.e., measured immediately after the learning session ended) was comparable for both groups of learners. Non-LARP learners exhibited a faster decline of transfer learning outcomes (immediate – after three weeks) compared to the matched LARP participants (d = 0.67). No other significant between-group difference as concerns learning outcomes was found (d = –0.16 – 0.20) (Suppl. Mat. G). This means that LARP players forgot less in terms of conceptual knowledge (moderate-to-strong effect).
We investigated how much motivation driven by interest in playing an educational game impacts learning outcomes compared to motivation driven by interest in the instructional domain (while controlling for levels of cognitive load). Motivation to play was shown to be related to learning outcomes, but its influence was dwarfed by the effects of the natural motivation to learn the given topic exhibited primarily by participants with developed domain interest. Specifically, developed domain interest was clearly related to autonomous motivation to learn and learning-evoked cognitive load, which were clearly related to learning outcomes. However, autonomous motivation to play, exhibited primarily by participants with developed interest in sci-fi LARP-like games, was only slightly related to transfer learning outcomes (and unrelated to retention). Also, motivation to play did not noticeably mediate the relationship between interest in sci-fi LARP-like games and learning outcomes.
This pattern of results – that of a relatively strong effect of domain interest on learning outcomes and weaker, sometimes even negative, effect of interestingness of framing the educational message on learning outcomes – is generally consistent with findings from the fields of multimedia learning (e.g., Rey, 2012) and hypermedia learning (e.g., Moos & Marroquin, 2010). This study has demonstrated this pattern in the context of game-based learning, for which large motivational benefits have been envisioned: it is better to enjoy learning than playing.
6.1.1 Theoretical contributions
From the self-determination theory perspective, it is noteworthy that we distinguished between autonomous motivations to learn versus to play, which is rare in game-based learning literature. In agreement with self-determination theory, our results demonstrated that motivation to learn was a better predictor of learning outcomes compared to motivation to play; yet the study also provided provisional evidence suggesting that intrinsic motivation to play can transform to identified regulation to learn. Future studies should explore in more detail how motivation derived from playing transfers to identified regulation to learn, as this is one of the key ways how motivation to play can influence learning processes within game-based learning. Other ways how motivation to play can impact learning processes should also be considered and examined (e.g., through enhanced self-efficacy).
From the cognitive load theory perspective, it was equally important that we distinguished between game-evoked extraneous load and learning-evoked loads. In agreement with cognitive load theory, the results confirmed learning-evoked loads as mediators of the effect of developed domain interest on learning outcomes. Complementary meditational analysis showed that perceived game difficulty (a proxy variable to game-engendered extraneous load) only tended to mediate the effect of developed interest in sci-fi LARP-like games on immediate transfer. Either the distraction from the game was not a big deal for participants, or our measurement did not assess the game-engendered cognitive load well. Validated methods for measuring this construct are needed, because the extent to which different types of games or game attributes evoke extraneous cognitive load is a pressing issue.
With respect to the four-phase model of interest development (Hidi & Renninger, 2006), our results corroborated the idea that developed interest in a learning domain is connected to enhanced learning in (at least) two different ways: cognitive and affective-motivational ones. This interest was linked to both lower cognitive load (presumably due to pre-existing, task-fitting schemata in the long-term memory) and higher values of motivation variables. Both cognitive load and motivation also mediated the effect of domain interest on learning outcomes.
6.1.2 Practical contributions
Our results showed that edu-LARPs may enhance learning through positive affective-motivational factors for some learners, but this educational method is not unanimously liked. This study and some prior research (Brummel et al., 2010; Mochocki, 2014; Vanek & Peterson, 2016) thus indicate that edu-LARPs are accepted differently by different learner-types. Acceptance-/suitability-related problems should be addressed in future research as well as in applying this educational method in practice (cf. Mochocki, 2014; Vanek & Peterson, 2016). Practitioners should also keep in mind that LARPs take a long time to prepare and complete. On a more positive note, a single edu-LARP can have multiple educational objectives at the same time (Bowman, 2014).
6.1.3 Methodological contributions
This work capitalized on the fact that game-related versus learning-related developed interest, affective-motivational, and cognitive load variables were assessed separately. We suggest considering this approach in future game-based learning research, as this can help elucidate complex roles different components of interest, motivation, and cognitive load play in learning processes.
No work is without limitations. The key thorny issue is the nearly-absolute lack of validated measurements for edu-LARP contexts (and beyond, as detailed below). First, developed instruments, such as EGameFlow (Fu et al., 2009), are very long. A construct needs to be assessed with a few questions within a game, and, should learning tests be administered, also after the game (to avoid fatigue). Second there is an on-going discussion about how to measure cognitive load and distinguish between different types of load (e.g., Leppink et al., 2014; Naismith et al., 2015). Cognitive load instruments that would be validated in the same way as, for example, PANAS are lacking. Finally, there is no agreed-upon method for measuring developed interests (Renninger & Pozos-Brewer, 2015). We were more satisfied with the gamer score variable, because it was more internally consistent compared to the techie score variable (despite all three subcomponents of the latter variable were theoretically related to developed domain interest). We believe that the pattern of our data is clear enough and consistent with underlying theories to warrant our interpretations of the findings. Nevertheless, valid instruments would be useful in future.
In an ideal world, this study would have had a control, non-LARP, condition and participants would have been randomly assigned to the LARP and non-LARP conditions. This would enable the contrasting of learning within the game to learning outside the game, whilst the content and the method (i.e., frontal lecture with hands-on practice) would be the same. Unfortunately, true randomization is rarely possible in research using edu-LARPs due to practical and ethical reasons (further detailed in Suppl. Mat. G). We have focused here on the within-subject comparison part (done within what would be an experimental condition) and have drawn conclusions from this part. We also recruited a quasi-experimental control group, but data on comparing performance of LARP learners to non-LARP learners should be treated cautiously because of the lack of randomization.
This study contributes to game-based learning literature, but also to the research base on productive enhancement of interest and motivation in academic contexts. Its key message is that learners’ developed domain interest and motivation to learn a particular topic contribute more toward enhancing learning outcomes than supposedly appealing, game-based augmentations of the educational message. Our results can be directly generalized probably only to games that have a similar level of complexity as our LARP. However, there is growing, parallel evidence suggesting that the message above is quite general and concerns many different types of learning environments and materials, such as textbooks or hypermedia.
Supplementary materials include: supplementary data and analyses (Suppl. Mat. A, F), detailed description of the edu-larp and procedure (Suppl. Mat. B), description of the function of the experimental device (Suppl. Mat. C), questionnaire items (Suppl. Mat. D), description of developed interest variables (Suppl. Mat. E), the supplementary study (Suppl. Mat. G).
We thank research assistants who helped to conduct the experiments, most notably: T. Zoulová, N. Frollová, K. Koppová, N. Střádalová, and P. Šustová. We thank Suzanne Hidi and K. Ann Renninger for discussing methods for measuring interest with us. We also thank Sarah Lynne Bowman for discussing this project with us and for commenting on early versions of this manuscript. We thank Lucie Filipenská for making the videos and all the actors: Anna Kratochvílová, Anežka Rusevová, Pavol Smolárik, and Luboš Veselý; including actors in pilot videos: Břetislav Dufek, Tereza “Tess” Kovanicová, and Jan Kovanic. We also thank David Obdržálek, Jan Hrach, Tomáš “Jethro” Pokorný, and Jana Stárková for making the devices, and Brmlab, a community-run hackerspace in Prague, for assistance. This study was primarily funded by Czech Grant Science Foundation (GA ČR), Project nr. 15-14715S. Work of F. D. was supported by RVO 68081740 by The Czech Academy of Sciences.
1 The p-value, unreported in the paper, is .076 (Pieter Wouters; email dating from 16 Dec 2013).
2 Some questions administered in this study were not analyzed and reported here. Examples include negative affect (from PANAS), which was out of present scope (see Suppl. Mat. A for descriptive data), and various manipulation check questions (e.g., on initial interest).
3 See Suppl. Mat. A for factor loadings and supplementary analyzes.
Bowman, S. L. (2014). Educational live action role-playing games:
A secondary literature review. In: The Wyrd Con Companion
Book (pp. 112-131): Wyrd Con.
Bowman, S. L., & Standiford, A. (2015). Educational LARP in
the Middle School Classroom: A Mixed Method Case Study. International
Journal of Role-playing, 5, 4-25.
Boyle, E. A., Hainey, T., Connolly, T. M., Gray, G., Earp, J.,
Ott, M., ... & Pereira, J. (2016). An update to the systematic
literature review of empirical evidence of the impacts and
outcomes of computer games and serious games. Computers &
Education, 94, 178-192. DOI: 10.1016/j.compedu.2015.11.003
Brom, C., Buchtová, M., Šisler, V., Děchtěrenko, F., Palme, R.,
& Glenk, L. M. (2014). Flow, social interaction anxiety and
salivary cortisol responses in serious games: A quasi-experimental
study. Computers & Education, 79, 69-100. DOI:
10.1016/j.compedu.2014.07.001
Brom, C., Šisler, V., Slussareff, M., Selmbacherová, T., &
Hlávka, Z. (2016). You like it, you learn it: affectivity and
learning in competitive social role play gaming.
International Journal of Computer-Supported Collaborative
Learning, 11 (3), 313-348. DOI: 10.1007/s11412-016-9237-3
Brom, C., Děchtěrenko, F., Frollová, N., Stárková, T., Bromová,
E., & D’Mello, S. K. (2017). Enjoyment or involvement?
Affective-motivational mediation during learning from a complex
computerized simulation. Computers & Education, 114,
236-254. DOI: 10.1016/j.compedu.2017.07.001
Brummel, B. J., Gunsalus, C., Anderson, K. L., & Loui, M. C.
(2010). Development of role-play scenarios for teaching
responsible conduct of research. Science and Engineering
Ethics, 16(3), 573-589. DOI: 10.1007/s11948-010-9221-7
Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S.
(2016). Digital games, design, and learning a systematic review
and meta-analysis. Review of Educational Research, 86(1),
79-122. DOI: 10.3102/0034654315582065
Cordova, D. I., & Lepper, M. R. (1996). Intrinsic motivation
and the process of learning: Beneficial effects of
contextualization, personalization, and choice. Journal of
Educational Psychology, 88 (4), 715-730. DOI:
10.1037/0022-0663.88.4.715
Csikszentmihalyi, M. (1975). Beyond Boredom and Anxiety:
Jossey–Bass, San Francisco, CA.
Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation
and self-determination in human behavior. New York: Plenum.
Deci, E. L., & Ryan, R. M. (2008). Facilitating optimal
motivation and psychological well-being across life's domains. Canadian
Psychology, 49(1), 14-23. DOI: 10.1037/0708-5591.49.1.14
deCharms, R. (1968). Personal causation: The internal
affective determinants of behavior . New York: Academic
Press.
Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs,
values, and goals. Annual review of psychology, 53(1),
109-132. DOI: 10.1146/annurev.psych.53.100901.135153
Fu, F.-L., Su, R.-C., & Yu, S.-C. (2009). EGameFlow: A scale
to measure learners’ enjoyment of e-learning games. Computers
& Education, 52(1), 101-112. DOI:
10.1016/j.compedu.2008.07.004
Fulmer, S. M., D'Mello, S. K., Strain, A., & Graesser, A. C.
(2015). Interest-based text preference moderates the effect of
text difficulty on engagement and learning. Contemporary
Educational Psychology, 41, 98-110. DOI:
10.1016/j.cedpsych.2014.12.005
Grolnick, W. S., & Ryan, R. M. (1987). Autonomy in children's
learning: An experimental and individual difference investigation.
Journal of Personality and Social Psychology, 52(5),
890-898. DOI: 10.1037/0022-3514.52.5.890
Habgood, M. J., & Ainsworth, S. E. (2011). Motivating children
to learn effectively: Exploring the value of intrinsic integration
in educational games. Journal of Learning Sciences, 20(2),
169-206. DOI: 10.1080/10508406.2010.508029
Hayden, J. K., Smiley, R. A., Alexander, M., Kardong-Edgren, S.,
& Jeffries, P. R. (2014). The NCSBN National Simulation Study:
A longitudinal, randomized, controlled study replacing clinical
hours with simulation in prelicensure nursing education. Journal
of Nursing Regulation, Supplement 5(2), S1-S64. DOI:
10.1016/j.ecns.2012.07.070
Hidi, S., & Renninger, K. A. (2006). The four-phase model of
interest development. Educational Psychologist, 41(2),
111-127. DOI: 10.1207/s15326985ep4102_4
Hyltoft, M. (2008). The Role-Players’ School: Østerskov Efterskole
Playground Worlds: Creating and Evaluating Experiences of
Role-Playing Games (pp. 12-25): Ropecon ry.
Hyltoft, M. (2010). Four reasons why edu-larp works LARP:
Einblicke (pp. 43-57): Zauberfeder Verlag.
Iten, N., & Petko, D. (2014). Learning with serious games: is
fun playing the game a predictor of learning success?
British Journal of Educational Technology, 47(1), 151-163.
DOI: 10.1111/bjet.12226
Jabbar, A. I., & Felicia, P. (2015). Gameplay engagement and
learning in game-based learning: A systematic review. Review
of Educational Research, 85(4), 740-779. DOI:
10.3102/0034654315577210
Juul, J. (2003). The game, the player, the world: Looking for a
heart of gameness. PLURAIS-Revista Multidisciplinar, 1(2),
248-270.
Kapp, K. M. (2014). Do Not Use Games for “Stealth Learning”.
Retrieved from http://karlkapp.com/do-not-use-games-for-stealth-learning/
(Acessed 15-06-2019)
Kalyuga, S. (2011). Cognitive load theory: How many types of load
does it really need? Educational Psychology Review, 23(1),
1-19. DOI: 10.1007/s10648-010-9150-7
Keller, J. M. (2010). Motivational design for learning and
performance: The ARCS model approach: New York: Springer.
Kot, Y. I. (2012). Educational LARP: Topics for consideration. In:
Wyrd Con companion book (pp. 118-127): Wyrd Con.
Leppink, J., Paas, F., Van Gog, T., van Der Vleuten, C. P., &
Van Merrienboer, J. J. (2014). Effects of pairs of problems and
examples on task performance and different types of cognitive
load. Learning and Instruction, 30, 32-42. DOI:
10.1016/j.learninstruc.2013.12.001
Mayer, R. E. (2014). Incorporating motivation into multimedia
learning. Learning and Instruction, 29, 171-173. DOI:
10.1016/j.learninstruc.2013.04.003
McAuley, E., Duncan, T., & Tammen, V. V. (1989). Psychometric
properties of the Intrinsic Motivation Inventory in a competitive
sport setting: A confirmatory factor analysis. Research
Quarterly for Exercise and Sport, 60(1), 48-58. DOI:
10.1080/02701367.1989.10607413
Mochocki, M. (2014). Larping the past: Research report on
high-school edu-larp. In S. L. Bowman (Ed.), The Wyrd Con
Companion Book (pp. 132-149): Wyrd Con.
Montola, M. (2008). The invisible rules of role-playing: the
social framework of role-playing process. International
Journal of Role-playing, 1(1), 22-36.
Moos, D. C., & Marroquin, E. (2010). Multimedia, hypermedia,
and hypertext: Motivation considered and reconsidered. Computers
in Human Behavior, 26(3), 265-276. DOI:
10.1016/j.chb.2009.11.004
Naismith, L. M., Cheung, J. J., Ringsted, C., & Cavalcanti, R.
B. (2015). Limitations of subjective cognitive load measures in
simulation-based procedural training. Medical Education, 49(8),
805-814. DOI: 10.1111/medu.12732
Pekrun, R. (2006). The control-value theory of achievement
emotions: Assumptions, corollaries, and implications for
educational research and practice. Educational Psychology
Review, 18(4), 315-341. DOI: 10.1007/s10648-006-9029-9
Renninger, K. A., & Pozos-Brewer, R. K. (2015). Interest,
psychology of. In: International Encyclopedia of the Social
& Behavioral Sciences, 2nd ed. (pp. 378-385). Oxford:
Elsevier.
Renninger, K. A., & Schofield, L. S. (2014). Assessing STEM
Interest as a Developmental Motivational Variable. Poster
presented as part of a structured poster session. In: Current
approaches to interest measurement. Philadelphia, PA:
American Educational Research Association.
Rey, G. D. (2012). A review of research and a meta-analysis of the
seductive detail effect. Educational Research Review, 7(3),
216-237. DOI: 10.1016/j.edurev.2012.05.003
Rheinberg, F., Vollmeyer, R., & Burns, B. D. (2001). FAM: Ein
Fragebogen zur Erfassung aktueller Motivation in Lern-und
Leistungssituationen [in German]. Diagnostica, 47,
57-66.
Rheinberg, F., Vollmeyer, R., & Engeser, S. (2003). Die
Erfassung des Flow-Erlebens [in German]. In: Diagnostik von
Motivation und Selbstkonzept (pp. 261-279): Hogrefe.
Ryan, R. M., & Deci, E. L. (2000). Intrinsic and extrinsic
motivations: Classic definitions and new directions. Contemporary
Educational Psychology, 25(1), 54-67. DOI:
10.1006/ceps.1999.1020
Ryan, R. M., Rigby, C. S., & Przybylski, A. (2006). The
motivational pull of video games: A self-determination theory
approach. Motivation and Emotion, 30(4), 344-360. DOI:
10.1007/s11031-006-9051-8
Sabourin, J. L., & Lester, J. C. (2014). Affect and Engagement
in Game-Based Learning Environments. IEEE Transactions on
Affective Computing, 5(1), 45-56. DOI:
10.1109/T-AFFC.2013.27
Schiefele, U. (1999). Interest and learning from text. Scientific
Studies of Reading, 3(3), 257-279. DOI:
10.1207/s1532799xssr0303_4
Schiefele, U., & Krapp, A. (1996). Topic interest and free
recall of expository text. Learning and Individual
Differences, 8(2), 141-160. DOI:
10.1016/S1041-6080(96)90030-8
Schraw, G., Bruning, R., & Svoboda, C. (1995). Sources of
situational interest. Journal of Literacy Research, 27(1),
1-17. DOI: 10.1080/10862969509547866
Sharp, L. A. (2012). Stealth learning: Unexpected learning
opportunities through games. Journal of Instructional
Research, 1, 42-48. DOI: 10.9743/JIR.2013.6
Sitzmann, T. (2011). A meta‐analytic examination of the
instructional effectiveness of computer‐based simulation games.
Personnel Psychology, 64(2), 489-528. DOI:
10.1111/j.1744-6570.2011.01190.x
Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive
load theory. New York: Springer.
Tingley, D., Yamamoto, T., Hirose, K., Keele, L., & Imai, K.
(2014). Mediation: R package for causal mediation analysis. Journal
of Statistical Software, 59(5). DOI: 10.18637/jss.v059.i05
Um, E. R., Plass, J. L., Hayward, E. O., & Homer, B. D.
(2012). Emotional Design in Multimedia Learning. Journal of
Educational Psychology, 104(2), 485-498. DOI:
10.1037/a0026609
Vanek, A., & Peterson, A. (2016). Live Action Role-Playing
(Larp): Insight Into an Underutilized Educational Tool. In:
Learning, Education and Games. Volume Two: Bringing Games into
Educational Contexts (pp. 219-240): ETC Press.
Vansteenkiste, M., Sierens, E., Soenens, B., Luyckx, K., &
Lens, W. (2009). Motivational profiles from a self-determination
perspective: The quality of motivation matters. Journal of
Educational Psychology, 101(3), 671-688. DOI:
10.1037/a0015083
Watson, D., Clark, L. A., & Tellegen, A. (1988). Development
and validation of brief measures of positive and negative affect:
the PANAS scales. Journal of Personality and Social
Psychology, 54(6), 1063-1070. DOI:
10.1037/022-3514.54.6.1063
Wouters, P., van Nimwegen, C., van Oostendorp, H., & van der
Spek, E. D. (2013). A Meta-Analysis of the Cognitive and
Motivational Effects of Serious Games. Journal of Educational
Psychology, 105(2), 249-265. DOI: 10.1037/a0031311
Wouters, P., & van Oostendorp, H. (2017). Overview of
instructional techniques to facilitate learning and motivation of
serious games. In Instructional techniques to facilitate
learning and motivation of serious games (pp. 1-16):
Springer.