I am taking two courses this semester to finally finish up the requirements so I can file for teacher certification in Idaho. In one of the courses, my reading this evening included this except from the National Center for Fair and Open Testing (NCFOT):
Black and William report that studies of formative assessment show an effect size on standardized tests of between 0.4 and 0.7, larger than most known educational interventions. (The effect size is the ratio of the average improvement in test scores in the innovation to the range of scores of typical groups of pupils on the same tests). . . . Formative assessment is particularly effective for students who have not done well in school, thus narrowing the gap between low and high achievers while raising overall achievement (NCFOT, 1999, ¶ 4).
Which reminded me, forcefully, that I do not have enough formative assessment built into my Nolat Labs course game. It’s integrated thoroughly in my Gnimmargorp game; nearly every activity is a formative assessment. But it’s nearly missing from Nolat Labs. I need to build it in as thoroughly as I have for my other game.
But as I wrote that, I realized just how much work it will be to properly integrate. To have formative assessment done well, there need to be clear and frequent opportunities for students to check their own progress. That will mean, among other things:
- Building multiple versions of exams covering the same topics, so students can take the formative exams and receive feedback about their areas of strength and weakness without compromising the final exam(s).
- Writing built-in feedback about the areas of strength and weakness, which will involve adding feedback information to every question in the question-bank.
- Creating exemplars of each project, in a range of results from mediocre to outstanding, so students can compare their work against a know standard and determine their current success and further needs.
Whew! That’s going to be a lot of work. But I believe…no, I know it will be worth the effort and time. Because I already have that in Gnimmargorp and I know just how powerful it is for student learning. Seeing their own results and knowing exactly what to do to improve them, or what to ask in order to understand more fully, is one of the most powerful tools I’ve given students in my games. So I’ll get started.
This is an under-construction ongoing post that will be regularly modified as the summer progresses. It’s a place for me to capture my thoughts about how I’ll be structuring my two major course-as-game efforts in the coming school year (16-17). There may also be responses posted here to some of my friend and colleague’s efforts along similar lines, over at Game On.
A tale of two games
My two game for next school year will be: a revised version of Gnimmargorp, my Introduction to Computer Science course, and a brand new site, Nolat Labs, which is for my IB Design Technology courses. I’ll talk in much more detail about each of these below. What they will both share are the basic mechanics and structure of Game-On, a plug-in for the WordPress website development environment. Game-On enables a gamified learning management system, or GMLS, which gives the capability to run my courses as a game by setting up assignments in parts, tracking players’ progress and earnings, awarding rewards, and providing oversight. For details, see this post at Game On.
These two courses are rather different, too. Gnimmargorp is well established, having gone through 3 full-year iterations so far. Many of the components will be familiar to my prior players, but several will be tweaked this year to improve classroom dynamics and engage players in more interaction. It will also be easier to navigate, start, and track your own progress.
On the other hand, Nolat Labs will be a brand new game and will be structured substantially differently than Gnimmargorp. In part, this is due to the nature of the course (having a more clearly defined curriculum and external moderation, for example). A larger part is due to me wanting to set up both a linear sequence – to accommodate learners who prefer this approach – and an open structure – to accommodate those who’d rather jump around.
This will be continued
One of my students wrote this in their final reflection: “It was fun because I really didn’t have to memorize things for a test or a quiz, instead some things were remembered because they were interesting. I didn’t have to stuff information in my brain in order to understand or do something, information about electronics just was in my head because it wanted to be there.”
I count that as a win for the year.
What a great way to end a week. I assigned a test this week to my Introduction to Computer Science players, in the form of a “Boss Challenge.” I have several fairly quiet students, who come in to class, sit down, get to work, and rarely say much even during our semi-weekly one-on-one conferences. But one of them wrote this in her journal after the period:
When I first heard there’s going to be an assessment my initial reaction was “Oh no, I’m really not that good.” But, last class I did the warmups and it was actually kind of fun because they were all sort of logic based questions and basic python coding so I thought “Maybe, I’m going to be okay.”
As it turns out, I was okay as a matter of fact I actually enjoyed it. After a bit of struggle , I breezed through it and even mastered it. I didn’t use the internet that much because the problems I did were sort of similar and I remembered more than I thought I did from CodeAcademy. So, I went from an “Oh no, I’m really not that good” to a“Maybe, I’m going to be okay” and by the end of it all I was at an “That was actually sort of fun.”
My response was simply: “I’m glad it was fun! And you are that good.”
“Sun behind clouds” by Superflewis at Wikimedia commons
Today I was thinking a lot about iteration. It’s a concept from computer science that describes the process of building something, then going back to revise it, again and again. It’s like my English teacher told me in junior high: just start writing and revise it as many times as you need.
This is something I’ve been teaching many of my students, in every subject I teach: Yearbook, Computer Science, Electronics, and Design Technology. It’s also something I’ve been doing, with my writing in various forums, with my websites, and in the design of each of my courses.
Iteration is all about not being afraid to start, even though you know it’s going to suck at first. In my experience, the best stuff I’ve created has sucked for around a couple of years before it really starts being something other folks notice. This matches what an amazing creative, Ira Glass, has said about his own work (not that his sucks, but that this is true in general). So it’s important to get over the fear of it sucking and just get started.
Then iterate. A lot. Revise it frequently. Get feedback from lots of people whose opinion you trust. Take criticism as a kindness, because praise doesn’t help you make it better (and you already know it’s not great, yet, so be skeptical of anyone who tells you it is, they probably just don’t want to see your feelings hurt). Keep iterating. And you’ll gradually see something you recognize as much better than when you started.
About two years into it, if you’re like me, you’ll start feeling like “yeah, this isn’t so bad after all.” You’ll also have a much better idea of all the amazing things you could still do to improve it, which are the most important, which are the ones you really want to do, and so on. Plenty more iteration to go!
Just before spring break, I asked my students playing the Introduction to Computer Science course (gnimmargorp.com) to add a note in their journal in which they thought either about their own thinking or about themselves, rather than just in-game observations. One young man wrote this:
I have learnt much from this class. This class has not only taught me the basics of coding, but how to time manage my time, and focus my attentions on my studies. In the beginning of the class I just wanted to follow the footsteps of my older brother who now studies Computer Science, and Engineering in Penn State. But a few month into the subject, and I began to actually really like it. I thought it was easy, but what was easy turned out to be both fun, and challenging. I finished all the Lullaby lyrics a few weeks ago, and had recently completed the tough quest of secrets. I have set a personal goal for myself, in which I would spend 30 minutes doing anything related to this class. I have been recently busy, thus unable to complete that goal. I have learnt that working with my classmates, especially Mohamed Abo Aiad in coding, has proven to be efficient, because we are both at the same level, allowing us to help each other in every quest and activity. I had been sick for a week, and traveling for another, so I think I might be behind in terms of the Code Academy completion, but I hope to catch up the class. I learnt that sometimes sitting near friends can cause disruption, and lack of focus. Although other times, I end up learning from them. This class I will continue working on quests, in order to earn some gold, and experience.
He packed a lot of stuff into 250 words! Most of his prior posts had been about his game progress or a problem on which he was working. Boy, was I pleased to read this. I should have begun asking for more thoughtful, reflective posts much earlier in the year. Lesson plans will be changed for next year. While this is the most exciting example, several other students similarly increased the depth of their reflection. I’m chalking this one up as a complete win.
The initial impetus for asking for more specific reflection came from a post at Mindshift: When Kids Have Structure for Thinking, Better Learning Emerges. I’m planning on re-reading that at least a couple times this summer in order to mine it for other ideas on asking better reflection questions and eliciting more specific responses from my students when they complete their weekly journal entries for our game.
36 – High Five by Melanie at Flickr
Apologies, I’ve just moved to a new host and several things, including image links, broke during the transition. I should have the site repolished soon.
About a month ago I began playing World of Warcraft (WoW) with Mike Skocko, a friend of mine (who also happens to be the guy that inspired and shepherds the Game On plugin that I use for running my course-game). We play once a week and discuss the way the game runs, aspects we notice (positively or negatively) and how those might translate into classroom practice. Our goal is to improve our course games by observing closely how one of the most engaging games of all time works.
Mike is an old hand at WoW, having built up and topped out multiple characters, while I am a newby with just a handful of hours played, total. Some of the things we’re noticing are already built into our course games, some we already knew about and plan to incorporate (or not), and others are new to our respective games. It’s been a pretty fun journey and one I’m eager to continue each week.
There are several things that we’ve noticed already (we’re kind of tracking the ongoing conversation over in our Game On forum). There are three that I think really differentiate the game environment from a typical school environment, two of which I’ve seen written about numerous places and one of which I haven’t.
Complete Freedom to Fail
The first really big mechanic I’ve noticed is that in WoW, I have complete freedom to fail. I can try as many times as I want to complete a quest or craft a product. There are no time limitations nor attempt limitations. I first noticed this when I foolishly took on a foe larger than I could handle. It was actually accidental: I was trying to catch up to Mike and ran through a village of Murlocks before I noticed they were there. They quickly surrounded me and started attacking. I saw my health meter dipping precipitously and my heartbeat rose while I started breathing faster. Shortly, my character died.
My heart dropped. Now, I thought, I’d have to start all over. But instead, the world turned to shades of gray and another “quest” appeared on my dashboard. To my surprise, and unlike other video games I’d played in the past, getting killed wasn’t the end of the game. I had to shepherd my spirit back to the place my body lay, finding my way there from a graveyard. I waited until there weren’t any more Murlocks by my body, then resurrected and continued playing. Wow! Even death isn’t the end, it’s just a chance to learn. The whole game, at least as far as I’ve played to date (level 25) has been like that. Endless chances to come back and learn how to do better.
Rich story/world with no driving purpose
Another really interesting thing I’ve noticed is that in WoW, there is a very rich story/world, which so far seems completely lacking in a driving storyline or purpose. There are definitely quests I pursue, but after the initial introduction, I choose which ones and there seem to be a wide variety of opportunities. There are also mini-stories related to every single quest shown. I have yet to get a quest that just says “do this” without at least a minimal bit of storytelling thrown in as well.
This is in great contrast to the way I developed my story/world for the ’14-’15 school year. Gnimmargorp definitely had a driving story line, although there were a few side quests, deviations and amusements. I had a vision of where I wanted students to go and steered consistently toward that spot. Some of the quests I built had information directly leading into the story line. Others had none, and no story.
When I started playing WoW a few weeks ago, I was pleased to notice that the game clearly had a storyline, which is introduced from the very beginning via a cinematic style fly-over of the land with a narration of your character’s race’s recent history. But as I played, I realized that every single quest had a little storyline associated with it. They didn’t seem to tie together in other than a really loose fashion, contributing to the overall sense of a world recovering from a major disaster. Some of the story bits tied together into small narratives, such as one in which I accomplished several small quests gathering materials for a fellow in what looked like a run-down church at the edge of town. Then, it turned out, he used those to raise a zombie and sent it into town on a rampage, so I had to help kill it to save the townsfolk.
That series of 5 quests, and others like it, confirmed my belief that storyline enhances the game experience. It contrasted starkly with my prior belief that an overarching storyline is needed in order to achieve that enhancement. There may still be an overarching storyline I simply haven’t seen enough of the game to understand. But this experience has given me the idea and the courage to build small storylines within the overall experience of my world. I enjoy both the connection with the world and the feeling of freedom the lack of a driving storyline allows.
Tiny chunks with lots of repetition and iteration
A sample quest. Note that you don’t go get the outfit, this is just about pulling the required money out of the bank. One small step at at time.
One thing I noticed right away and which I continue to notice is how tiny the quests are. I’m at level 25 and I have yet to receive a quest that requires me to really puzzle through what I’m supposed to do. The game scaffolding is so fine-grained that I’m never in the slightest doubt, yet, regarding how to accomplish the quests I’ve been given. The steps I need to take are both tiny and clearly laid out for me. There is a progression, but it’s a very slow ramp and I always feel I am more that capable of finishing the next quest.
As I built my quests for Gnimmargorp, I planned them based on the assignments I would have given had it been a regular class. To begin with, this worked out fine. Later on, the assignments became larger and more complicated, as I wanted to provide sufficient challenge as students’ skill grew.
After the initial skills were learned, I offered students the chance to choose whether they followed my game or used one of several other options to structure their path, such as codecademy or The New Boston. When students did choose another option, I asked why. The overwhelming answer was that my game was too difficult, with too little step-by-step instruction. Now that I’ve played WoW, I can understand better how I might restructure my game to lead players through learning the skills they need in smaller chunks.
The flip-side of that tiny-ness is the vast number of repetitions I go through for every skill in WoW. Literally dozens of quests to practice every single new skill that is introduced, each of which is so small that I know I can complete it. In order to level up, I need to do many of them, so iterative practice of my skill is essential to my game progress (sound like any classroom pedagogies you’re familiar with?). This is a model I can replicate, but it’s going to take a lot of quest writing to ensure I have enough quests to make this work.
Renovations & refurbishment
Now that I’ve sorted out these three major differences between my game world and WoW, I have a lot of work to do this summer. I need to build a completely new set of quests which each contain a bit of story but don’t necessarily have to connect in a direct way with my overarching storyline, and which are numerous, so iterative practice of the concepts and skills in the course are built into the game. Whew!
Earlier this week, I had my most succesful single class, ever. Hunters and Healers. Woot!
I opened by telling the players that a Repaer had been seen in the river near Esab (our city). Repaers catch the weak and the outliers (had to explain that vocab word), so we want our cohort to be strong and cohesive. “In order to ensure that you are all strong,” I said, “we’re going to play Hunters and Healers.”
“To hunt, you will look at other players’ Codex entries (our name for blogs) and find any missing entries, any that are too short, weakly composed, etc. If you successfully hunt a Codex author, you must then help Heal by ensuring the author is aware of the problem and writes or improves the entry. As the Hunter, you must document the original version (or lack thereof) and the revised version in an entry on your own Codex.”
I offered a reward for any successful hunts (I think I’ll have to reduce it for future episodes of this sort, to keep progress better balanced). I warned that the Repaer is coming on Tuesday to wind through the Codex; anyone it catches will loose health and have to pay a healer as well. I set it elicit high motivation, at a nerf (subtraction) that most of the players don’t quite have yet, so they’d end up in the negative. It’s not too stressful, though, because I don’t grade based on points this year (more about that soon).
Both of the players who had previously had zero Codex entries filled out their blogs completely today. Many others who were marginal (on either side of the line) substantially beefed up their entries. In spite overly analytical player pointing out that this is “just peer editing, right?” everyone had fun. Lots of fun! Most of the players will be receiving a reward for a successful hunt. The most ironic moment of the day came when two players approached to inform each other that their Codex entry needed work, and the entries were the same ones on both blogs. 🙂
It was 100% engagement, lots of good reflection and peer coaching, and a great success in my book. (Best of all, my principal chose that moment to come by for a classroom observation – what could be better than a fully engaged classroom of kids eager to explain how they were peer editing each other’s reflection websites?!?). Thanks to Mike Skocko and Rob Schwartz at the Adobe Education Exchange Game-On forum for putting me on to this idea; it’s been a really powerful one.
Follow-up: I had 4 players get caught by the Repaer and loose XP and Gold as a result. In all 4 cases, there were no arguments about it at all. I discussed it with the players that they’d have many chances to catch up and could even work outside of class if they wanted to do so. They all leaped right back into their work and don’t seem phased by the setback in the slightest. Disappointed, yes, but not disheartened. And some of the hunters are a bit chagrined to realized that they overlooked those players and feel like they let their team-mates down.
So in the end, it remains my most successful class and I’ll definitely be using this technique again.
Initially, I was a bit afraid my game structure would push students too far, too fast. Like many of my preconceptions about inquiry-based and game-based-learning, this proved to be completely unfounded. In fact, the opposite has been true. More than once this year, I’ve had to scramble to keep building ahead of the students as one or more progressed through a portion of the material much faster than I expected.
The big takeaway for me has been that I need to ask them to reach further. In a gamified class, I can build out ahead of time (especially when I’m not building the plane as we fly it). For next year, I’m planning to build out substantially more material on the advanced side, as well as fleshing out material for the sections that some students breezed through. In providing more options for any particular section, I hope to encourage people to linger longer on a topic they might otherwise have brushed past; I’m also planning to plant some easter eggs so that if I notice someone brushing by a topic I can drop a hint that they’re missing something and thus help encourage them to linger further. Similarly, I plan to build the upper end of the class out further than I previously did. This is related to expanding the points ladder, but is more focused on ensuring there is enough content to keep the rapid players engaged within the game world. There will also be quests specifically designed to encourage the advanced players to work with their classmates to further everyones’ knowledge of the subject matter (of course, that will only help them, too, but I won’t be telling them that). These will only open for the advanced students – an elder game of sorts. I’m working on developing this idea further; if you have ideas or have done something similar I’d love to hear about it in the comments.
On the opposite side, I’ve discovered that a bit more scaffolding when I do ask for a big reach will be helpful, too. The second-to-last project has been a fairly large one, with very minimal scaffolding for them to base it on. I just provided a description and a minimal rubric. My thought was that this would give them the widest scope within which to imagine their project and build it, which would lead to better engagement due to ownership. Unfortunately, my students do not have much experience with a wide-open field, and so floundered a bit, especially at the beginning, with that freedom. Next year I plan to offer the same wide open field, but have an “I’m not sure what to do next” button that will lead to more specific scaffolding to guide those students that need a bit more structure within which to build their project.
Reach by Steve Mishos on Flickr
Helping Hand by Asarum Images on Flickr