The Necessary Nine: Design Principles for Embodied VR and Active STEM Education
Shortened version of original: Johnson-Glenberg, M. C. (2019). The Necessary Nine: Design Principles for Embodied VR and Active STEM Education (pp. 83-112). In P. Diaz, A. Ioannou, K.K. Bhagat, & J.M. Spector (Eds.), Learning in a Digital World: Perspective on Interactive Technologies for Formal and Informal Education. Singapore, Springer. https://link.springer.com/chapter/10.1007/978-981-13-8265-9_5 The Necessary Nine: Design Principles for Embodied VR and Active STEM Education Dr. Mina C. Johnson-Glenberg Abstract: This chapter explores the two profound affordances of VR for learning; namely 1) the sense of presence attendant with immersive VR, and 2) the active learning associated with movement/gestures in a three dimensional virtual world. The chapter highlights several theories supporting embodied education and two examples of mediated STEM lessons which have been designed to maximize active learning. The first example explores the journey of redesign when a 2D tablet game is transformed into a 3D immersive VR lesson. The second example highlights how the new generation of hand controllers in VR can be used with constructivism to scaffold complex topics (chemistry and fireworks). The chapter ends with a set of optimal design principles for immersive VR in STEM education. The most important are called the Necessary Nine. Key words: Virtual reality, VR, embodiment, STEM education, multimedia design principles, XR (To read full version with references, go to above link for chapter or www.embodied-games.com and search under BLOGS.)
The Two Profound Affordances of VR for Education
For several decades, the primary input interfaces in educational technology have been the mouse and keyboard; however, those are not considered highly embodied interface tools (Johnson-Glenberg, Birchfield, Koziupa, & Tolentino, 2014). Embodied, for the purposes of education, means that the learner has initiated a physical gesture or movement that is well-mapped to the content to be learned. As an example, imagine a lesson on gears and mechanical advantage. If the student is tapping the s on the keyboard to make the gear spin that would be considered less embodied than the student spinning a fingertip on a screen to manipulate a gear with a synchronized velocity. With the advent of more natural user interfaces (NUI), the entire feel of digitized educational content is poised to change. Highly immersive virtual environments that can be manipulated with hand controls will affect how content is encoded and retained. Now learners can spin a virtual hand crank with full arm movements (circles) and engage with 3D complex gear trains from any vantage point desired. One of the tenets of the Embodied Games lab is that doing actual physical gestures in a virtual environment will have positive, and lasting, effects on learning in the real world. Tremendous opportunities for learning are associated with this latest generation of virtual reality (VR) (Bailenson, 2018) and one of the most exciting aspects of VR is its ability to leverage interactivity (Bailenson et al., 2008).
Now that many of VR’s affordability and sensorial quality issues are being addressed, it is reasonable to assume that VR experiences will become more ubiquitous in educational settings. When the demand comes, the community should be ready with quality educational content. There are few guidelines now for how to make optimal educational content in VR; this chapter will begin by explicating several relevant pedagogical theories. The chapter includes two case studies of lessons that have been built already, and it ends with 18 tenable design principles. The guidelines have been further pared down to the “Necessary Nine” for education.
First, what makes VR special for learning? Two attributes of VR may account for its future contributions to education. These we call the two profound affordances. The first profound affordance is 1) the feeling of presence which designers must learn to support, while not overwhelming learners. Slater and Wilbur (1997) describe presence as the feeling of being there. It is a visceral transportation that, in many individuals, occurs immediately; when surrounded in 360 degrees by the virtualized unreal environment, players often lose sense of time.
The second profound affordance pertains to 2) embodiment and the subsequent agency associated with manipulating content in three dimensions. Manipulating objects in three dimensional space gives a learner unprecedented personal control (agency) over the learning environment. Gesture and re-enactments using the hand controls (and tracked fingers) should increase agency and positively impact learning. The basis for this prediction is the research on embodiment and grounded cognition (Barsalou, 2008). Although other methods for activating agency can be designed into VR learning environments (e.g., using eye gaze and/or speech commands), it may be the case that gesture plays a special role. Gesture kinesthetically activates larger portions of the sensori-motor system and motoric pre-planning pathways than the traditional modalities for learning (i.e., the visual and auditory). Gesture may lead to stronger memory traces (Goldin-Meadow, 2011). Another positive attribute of engaging the learner’s motoric system via the hand is that the use of hand controls is associated with a reduction in cybersickness (Stanney & Hash, 1998).
VR for education should take full advantage of 3D object manipulation using the latest versions of handheld controllers (as well as, gloves and in-camera sensors to detect joints, etc.). This shortened chapter focuses on design practices that the author has learned from creating content in mixed and virtual realities over the past 12 years. An early, and evolving, set of design principles for VR in education is provided at the end, with the hope is that the guidelines will assist this nascent field as it matures.
The 18 Primary Guidelines for STEM Education in VR
Assume Every Learner is a VR Newbie – Start slow
Not everyone will know the hand controls. Not everyone is a gamer. Not everyone will look around. Users are now in a sphere and sometimes need to be told to turn their heads. However, they should not turn too far, nor too quickly. Do not place important interface, HUD components, or actionable items, too far from each other.
Part of starting slow includes being gentle with the user’s proprioceptive system (where the body is in space). For example, if your user captures a butterfly at 10°, then do not force the next capture to be at 190°. Watch out for large body-action disconnects as well, e.g., the learner is standing, but the avatar is running, or lying in a bed. If the content includes varying levels of difficulty, allow the user to choose the level at the start menu. (This also gives a sense of agency.)
Introduce User Interface (UI) Components Judiciously – fewer is better
· Keep the as screen clean as possible. Permanent objects (i.e., a timer that stays center-screen as players turn their heads) will unnecessarily disrupt presence. Be creative about health bars (e.g., when the game Snow Fortress ported to VR, the designers got rid of pinned health bars, now the amount of snow accumulating on the users’ mittens serves as health state feedback – cool). When users build the first fireworks in the chemistry lesson (next section), they can only make simple one stage rockets. The more complicated multi-stage components are not available in the interface until users show mastery of the simpler content. Designers should add visual complexity to the interface when the user is acclimated and ready (Johnson-Glenberg, Savio-Ramos, Perkins, et al., 2014).
Scaffold – Introduce Cognitive Steps One at a Time
· Build up the user interface as you build up in cognitive complexity. This is a form of scaffolding (Pea, 2004). In the electric field series of seven mini-games, users are not immediately exposed to the multi-variable proportionality of Coulomb’s Law. Each component, or variable, in the Law is revealed one component at a time and reinforced via gameplay. Users explore, and eventually master each component successively before moving to the final lesson that incorporates all the previously learned content and culminates in the formation of lightning (Johnson-Glenberg & Megowan-Romanowicz, 2017).
Co–design with Teachers
· Co-design means early and with on-going consultations. Let the teachers, Subject Matter Experts (SMEs), and/or clients play the game at mid- and end stages as well. Playtesting is a crucial part of the design process. Write down all comments made while in the game. Especially note where users seem perplexed, those are usually the breakpoints. Working with teachers will also ensure that your content is properly contextualized (Dalgarno & Lee, 2010), i.e., that it has relevance in, and is generalizable to the real world and to relevant educational content standards.
Use Guided Exploration
· Some free exploration can be useful in the first few minutes for accommodation, and to incite curiosity, but once the structured part of the lesson begins, it is your job to guide the learner. Guide using constructs like pacing, signposting, blinking objects, constrained choices, etc. To understand why free exploration as an instructional construct has not held up well in STEM education, see Kirschner, Sweller, & Clark (2006).
Minimize Text Reading
· Rely on informative graphics or mini-animations whenever possible. Prolonged text decoding in VR headsets causes a special sort of strain on the eyes, perhaps due to lens muscle fatigue or the vergence-accomodation conflict. In the Catch a Mimic game (available in the Oculus store), players do not read lengthy paragraphs on butterfly cocoons and emerging, instead a short 2-second cut-scene animation of butterflies in chrysalis and then emerging is displayed.
Build for Low Stakes Errors Early On
· Learning often requires errors to be made. Learning is also facilitated by some amount of cognitive effortfulness. In the Catch a Mimic game, the player must deduce which butterflies are poisonous, just like a natural predator must. In the first level, the initial butterflies that appear on screen are poisonous. Eating them is erroneous and slightly depletes the player’s health score, but there is no other way to discern what is toxic, some false alarms must be made. In psychology, this is called ‘learning from errors’ (Metcalfe, 2017); in the learning sciences, it has been called productive failure (Kapur, 2016).
Playtest Often with both Novices and End-users
· It is crucial that designers playtest with multiple waves of age-appropriate learners for feedback. This is different from co-designing with teachers.
· Note, playtesting with developers does not count. Human brains learn to reinterpret visual anomalies that previously induced discomfort, and over time users movements become more stable and efficient (Oculus Developers Guidelines, 2018). Developers spend many hours in VR and they physiologically respond differently than your end-users will.
Feedback – Unobtrusive, Actionable and Well-timed
· This does not mean giving constant, on screen, feedback (Shute, 2008). Feedback should be high level, and if text is included, it should be evaluative but short. Some proportion of users will be colorblind, so you cannot rely on only red and green colors for feedback. Feedback should be paced because it takes time for the cognitive adjustments to be integrated into the learner’s ongoing mental model. This leads to the next guideline on reflection.
Design in Opportunities for Reflection (it should not be all action and twitch! Include metacognition.)
· Education game designers are currently experimenting with how to do this in VR. Reflection allows the learner’s mental model to cohere. Some ongoing questions include: Should the user stay in the headset or not? How taboo is it to break immersion? Should short quizzes be embedded to induce a retest effect (Karpicke & Roediger, 2008)? Perhaps screencasting/mirroring with dyads where one partner is outside the headset could be conducive to learning and the interleaving of new knowledge.
Encourage Collaborative Interactions
· Synced, multiplayer experiences are still expensive, but their creation is a worthy goal. Until the cost drops, designers should explore workarounds to make the experience more social and collaborative. Some ideas include: using a preprogrammed non-player character (NPC), having a not-in-headset partner interact via a screencast, or building sequential tasks that require back-and-forth asynchronous activities. Education game designers are currently experimenting with how to do this in VR. Reflection allows the learner’s mental model to cohere. Some ongoing questions include: Should the user stay in the headset or not? How taboo is it to break immersion? Should short quizzes be embedded to induce a retest effect (Karpicke & Roediger, 2008)? Perhaps screencasting/mirroring with dyads where one partner is outside the headset could be conducive to learning and the interleaving of new knowledge.
Excited about Embodiment! Using Hand Controls/Gestures
The final design guidelines (numbers 12 through 18) focus on using the hand controllers in VR for learning.
Use the Hand Controls to Encourage the Users to be “Active”
· Incorporate into lessons opportunities for learners to make physical, kinesthetic actions that manipulate content. Where appropriate, try to include representational gestures and/or re-enactments.
· In this lab’s previous research, the group that was instructed in centripetal force and made kinesthetic circles (either with the wrist or arm) retained more physics knowledge, compared to the group that made low embodied, less active motions (Johnson-Glenberg, Birchfield, et al., 2014). Active learning has been shown to increase STEM grades by up to 20% (Waldrop, 2015).
How Can a Body-based Metaphor be Applied?
· Be creative about ways to incorporate kinesthetics, or body actions, into the lesson. At first blush, it may not be apparent how to make a traditional bar chart become more embodied. But with a VR hand control, the learner can now use a gesture to fill a bar to the correct height. An upward swipe is also congruent with our cultural concept of more (see Dor Abrahamson’s work on embodied and mediated examples of proportional reasoning). In the “Catch a Mimic” game, learners are instructed to make a prediction about species survivability using the hand controls (see Figures 5 and 6, next section). Additionally, prediction is a well-researched metacognitive comprehension strategy (Palincsar & Brown, 1984).
· The gesture/action should be congruent, i.e., it should be well-mapped, to the content being learned (Black et al., 2012; Johnson-Glenberg & Megowan-Romanowicz, 2017). The action to start a gear train spinning should involve moving the hand or arm in a circle with a certain velocity; the action should not be pushing a virtual button labeled “spin” (Johnson-Glenberg et al., 2015).
Actions Strengthen Motor Circuits and Memory Traces
· Performing actions stimulates the motor system and appears to also strengthen memory traces associated with newly learned concepts (Refer to the Appendix B on embodiment, the Goldin-Meadow corpus, or Johnson-Glenberg & Megowan-Romanowicz, 2017).
Ownership and Agency
· Gestural control gives learners more ownership of, and agency over, the lesson. Agency has positive emotional affects associated with learning. With the use of VR hand controls, the ability to manipulate content and interactively navigate appears to also attenuate effects of cybersickness (Stanney & Hash, 1998).
Gesture as Assessment – Both Formative and Summative
· Design in gestures that reveal the state of the learner’s mental model, both during learning (called formative or in-process) and after the act of learning (called summative).
· For example, you might prompt the learner to demonstrate negative acceleration with the swipe of a hand controller. Does the hand controller speed up or slow down over time? Can the learner match certain target rates? This is an embodied method to assess comprehension that includes the added benefit of reducing guess rates associated with the traditional text-based multiple choice format. For an example of hand movements showing vector knowledge on a tablet, see the Ges-Test in Johnson-Glenberg and Megowan-Romanowicz (2017).
Aspirational – Personalized, more Adaptive Learning
· Finally, try to include adaptivity. This is acknowledged to cost more, but the learning content level should often reside a fraction beyond the user’s comprehension state, also known as the learner’s Zone of Proximal Development (ZPD) (Vygotsky, 1978).
· Gesture research on younger children shows they sometimes gesture knowledge before they can verbally state it. Gesture-speech mismatches can reveal a type of readiness to learn (Goldin-Meadow, 1997). Thus, gestures can also be used as inputs in adaptive learning algorithms. Adding adaptivity (dynamic branching) based on performance is difficult and time-consuming to design, but it is considered one of the best practices in educational technology (Kalyuga, 2009); it is something to strive for.
Because all studios and developers have fiscal constrains – the 18 guidelines have been condensed into the “Necessary Nine”. (Engagement is not included because all designers obviously strive for engagement!)
THE NECESSARY NINE
1- Scaffold cognitive effort (and components in interface) – one step at a time
2- Use guided exploration
3- Give immediate, actionable feedback
4- Playtest often, with correct user group
5- Build in opportunities for reflection
6- Use the hand controls for active, body-based learning
7- Integrate gestures that map to the content to be learned
8- Gestures are worth the time and extra expense – they promote learning, agency, and attenuate cybersickness (be creative about using motion and gesture for assessment)
9- Embed assessment, both during and after the lesson
It is an exciting time for education and VR, filled with opportunity and enlivened by a rapidly changing hardware landscape. Besides issues around how to design optimal lessons, there are important questions regarding when to insert a VR module. Aukstakalnis (2017) shares an anecdote about a student in a design class who regrets designing his first project in a VR headset during the year-long course because he missed watching classmates work in the real world and being able to learn from his peers’ collective mistakes. The design guidelines presented here will be refined as the hardware and its affordances change. This chapter focused on the two profound affordances associated with the latest generation of VR for educational purposes: 1) presence, and 2) the embodied affordances of gesture in a three dimensional learning space. VR headsets with hand controls allow for creative, kinesthetic manipulation of content, those types of movements and gestures have been shown to have positive effects on learning. Hand controllers can be used for innovative types of assessment. Hopefully, the case studies and design guidelines here will help others to create effective immersive VR lessons.