Reinforcement Concepts for Designers

All game designers should take a few courses in psychology. This will help a lot more than you think, especially if you’re aiming to be a systems designer (as opposed to a level or content designer). Game design is an application of psychology: the goal of a game is typically to entertain and/or engage the human being playing the game, and studying psychology helps explain how we entertain and engage people.

Don’t believe me? Let’s just peek into our Psych 101 course book for some examples. Let’s talk about positive and negative reinforcement! Then we’ll follow up with the role of tedium in behavior management.

Positive Reinforcement versus Negative Reinforcement

Positive reinforcement is when you reward good behavior with a cookie. Negative reinforcement is when you reward good behavior by making the pain stop.

These can be mathematically equivalent, but they have psychological ramifications you need to consider.

Suppose you’re designing the food system for a new MMO, and you decide that characters become physically weaker if  they don’t eat a piece of food every few hours. (Eating food is thus a negative reinforcement because it removes the weakness that comes from being hungry.) Another way to design the same system is to give players a buff when they eat food. This is a positive reinforcement.

Mathematically they look like the same system, because you’re going to balance the game based on the “fed state” — in other words, when you figure out how tough monsters should be, you’ll assume the player is properly fed. It matters not to the spreadsheet whether food is a buff or just the absence of a debuff.

But it matters to the player! Modern gamers expect all punishment to cease when they are playing your game the right way. You can take advantage of this to train players to play your game the right way.

In this example game, since players NEED to eat in order to play correctly, you should use negative reinforcement. You’d put an “I’m Hungry!” icon on the screen whenever they needed to eat, and you’d remind them that they are weaker due to lack of food. When they eat, these indicators go away and the character pats her stomach like she’s happy to have finally eaten. This helps teach players when they aren’t playing optimally. Even if players don’t get the hint right away, it will sink in eventyally. If they’re dying a lot, they’re likely to think “I should really eat something. That will help!”

On the other hand, if eating provides only tiny effects (and you didn’t balance encounters assuming that players were fully satiated all the time), then you would present eating as a “buff” instead. This teaches players that eating is a good thing, but not something mandatory.

Now, if you secretly decided that players should always be eating food or else they are sub-par statistically (and unable to complete quests for their level), you’d really confuse them by making food a buff instead of debuff. Players intuit that a buff is an optional power benefit. They do not expect you to have factored this optional benefit into every combat encounter… if you did that, it wouldn’t be optional! Be wary of sending mixed signals like that. It’s pretty common, but then again it’s also pretty common for MMO’s to be unintuitive.

Both positive and negative reinforcement are very useful in every MMO — you just have to decide which to use in each situation. ”But I don’t want negative reinforcement in my game!” you say. It’s got that word negative in there. That can’t be positive! Actually, when used in moderation, negative reinforcement is fine. For instance, World of Warcraft treats armor repair as a negative reinforcement, and it works great. You only get into trouble when you overdo it. When a game has too many negative reinforcements, the player can feel like they’re having to do too much “work” to play the game.

The Role of Tedium

Punishment and negative reinforcement often go hand in hand, because usually you can’t negatively reinforce unless you do something unpleasant first, which the player will perceive as punishment. In WoW, you’re punished for letting your armor break: you take a lot more damage in combat. Repairing your armor is negatively reinforced by removing that punishment.

So normally they are directly related. But there’s one major case where this isn’t true: players don’t generally perceive being bored as a punishment per se. This means that developers can inject tedium into their game, and then negatively reinforce behaviors in order to remove the tedium. Brilliant! Well, sort of.

Walking from city to city in World of Warcraft is torturously boring. Using a gryphon is moderately less tedious, and having your own flying mount is less tedious still. WoW uses tedium — and the slow removal of it — as a balance and reward mechanism.

Balancing through tedium works! But it won’t keep working forever: its acceptance among gamers is quickly disappearing. Tedium is the last bastion of the old punishing mechanisms from the early days of computer games. Remember those games? Man, they were punishing.

Back in the day, if you died, you were severely punished. In Ultima Online, you lost every item you had, and you could even lose the deed to your house and all your belongings. Holy shit that was punishing. This worked great, though, because the fun of the game outweighed the punishment. No, that’s not it… let’s see… what was it? Oh yeah: players didn’t know of any other game they could go to.

Since then, punishments have mostly disappeared from games. Why? Because all other things being equal, players tend to go to the less-punitive game. From a designer standpoint this is actually a little frustrating. One of our most potent psychological tools — punishment — is slowly dwindling away! This actually makes it harder to train players to find the fun in our games. But we are never going back to the time when mainstream MMO’s punish you heavily for making mistakes. That time is over. As a designer, this is a tiny bit saddening. But as a player, this is a much happier time to be playing MMOs.

However, there’s one major kind of punishment still in the toolbox. Tedium is the last type of punishment in MMO games. It’s survived because it’s so insidious that players don’t really think of it as a punishment. Nevertheless, tedium was actually on its way out when WoW came along. WoW revived tedium for its travel system (and, to a lesser extent, for its food/drink system), and it got away with it because the rest of the game was so damned good. (In comparison to the other games available.)

The designers didn’t arbitrarily add tedium to travel. They had clear goals in mind. They believed that slow travel was the key to making the world seem “large” and immersive. With instantaneous travel, they reasoned, every place in the world would just be a hop, skip, and a jump away, and this would reduce the immersiveness of the world, which they believed was key to making the game fun over the long term. As an added bonus, having this tedium in the game meant they could remove the tedium over time as a reward mechanism.

They pulled it off, but don’t count on being able to follow their lead. Unless your game is so good in other ways that you can get away with an annoyance like tedious travel, you’re better off having instant travel mechanisms. Your world might be less immersive, and you will definitely have fewer rewards to give out because you can’t remove the tedium that isn’t there. But your game doesn’t exist in a vacuum, so you really have no choice. Heck, even WoW is busy removing tedium over time. The game gets less tedious practically every month. (Today’s Penny-Arcade is coincidentally about this very thing.)

I’m not saying all tedium must go. Like other types of punishment, it’s a great way to get players to think the way you want them to. Unforced tedium is not in any danger of going away. Suppose a player wants to keep stabbing low-level orcs in the newbie zone for as long as possible. It’s okay if this is a tedious way to play… assuming the player knows full well that they have made the choice to do something tedious. But tedium that can’t be avoided (such as a quest that makes players run for 15 minutes from the newbie city to a capital city) is not something players respect anymore.

Tedium is on its way out in games. Use tedium if you think you can get away with it, but … well, you probably can’t. Heck,  I personally prefer a jolt of explicit punishment over a jolt of tedium. That’s the ADHD crowd for you — tedium is worse than pain.

Punishment and tedium will never completely go away, but they have clearly waned in their influence. Keep that in mind when you design your game. If you go heavy on negative reinforcement, your game is likely to be perceived as “old-school”, and that may be exactly what you’re going for, if you want a niche title. If you want a broad-spectrum game, use sparingly.

Negative Reinforcement = Potent Spice

I want to reiterate that you shouldn’t plan to remove all negative reinforcement! Just treat it like you would a potent spice when cooking: a little goes a long way. Decide the handful of places where negative reinforcement will give your game a big benefit, and use it there. Remove it from all the places where it’s less important.

I really think psychology is a useful field for designers to study. I wanted to go into reinforcement schedules a bit, but that’ll be another time, I guess. Anyway, if you’re wondering “what should I study in college to learn to be a systems designer?”, a dozen credits in well-chosen psych courses are a good investment.

This entry was posted in Design. Bookmark the permalink.

18 Responses to Reinforcement Concepts for Designers

  1. Kirk says:

    I don’t disagree, but I will add a touch of complexity. How much negative and positive reinforcement players will not only tolerate but desire will also depend on where on the entertainment-hobby line the players are located. (See Andrew’s excellent explanation of that here.)

  2. Ravious says:

    Great article! I also think that conditioning (teaching through reinforcement) is often overlooked. Designers condition the gamer throughout the advancement game, and in so many MMOs a bait and switch occurs in the late game or end game. Designers should be thinking about how to condition gamers (with positive or negative reinforcement) towards whatever final, sticky content they have from the very beginning. I sometimes wonder if some designers even know how they are actually conditioning the players to play their game.

  3. Ninetytwo says:

    Fantastic post. I’d argue, though, that there’s a concomitant flip side to parceling out negative reinforcement that we see in a lot of MMOs today:

    It’s very easy to succeed at most current-gen MMOs playing suboptimally with little or no negative reinforcement attached to poor or less-than-optimal play. Games are so solo-friendly that players often aren’t taught the skills they will need to know how to use when actually challenged, in pvp for example, or in group / raid play.

  4. Ferrel says:

    I actually miss punishment in MMOs. I recognize that the demographic at large won’t accept it any more but it has changed the way we play the game. You use the best example of over doing a death penalty. UO’s death penalty was so frustrating it was bad for business. The day I canceled my account was a day that I was PK’d and had the entirety of my being stolen. It was excessive.

    EverQuest also had quite a heavy death penalty. You could lose all your gear, had to waste a ton of time getting to your body, and you lost experience no matter what you did. Most importantly, however, you did not want to die and you played accordingly. You actively avoided death like you might in reality.

    I honestly don’t want a UO or EQ style death penalty again but I certainly want more than what we do have. Players use death for transportation now! The penalty is so insignificant that we no longer act out of self preservation. It has cheapened the experience for me. Since death is so meaningless I rarely feel the high of victory when I achieve something. It almost seems that our MMOs have been evened out. No extremely lows and thus no extreme highs.

  5. doubledutch says:

    Ninety two:

    Taking away negative reinforcement does not equal making a game easy or visa-versa.

    For example, if you take a look at a Counter-Strike deathmatch server. There is no punishment for dieing, yet you still get better and learn how to play / succeed from doing the action itself (playing).

    I would say that practice > negative reinforcement in the realm of improving skill (though not in necessarily in the will to succeed or not-fail).

    If someone is succeeding in an MMO playing suboptimally, then they are playing just fine.

    ___________________________________________________________________________________________________

    Back on topic:

    I’m going to enter this discussion continuing with the setting of Ac2 vs. WoW that I was making use of in my comment on your last entry.

    As you were saying, WoW basically re-invented tedium. Before WoW, many games like Ac2 and PlanetSide were trying to move away from that concept.

    You are correct in saying that WoW brought the tedium back into MMO gaming because it was attached to the most solid product that we had ever seen (still have?).

    It really is dissapointing. That era of about 2 years up to WoW’s release, was such a promising innovative time in MMO’s and the huge success of one of them created the industry belief that everything it did was the right way to do it.

    ___________________________________________________________________________________________________

    …aaannnddd fast travel can be great in creating immersion. When you have fast travel, such as portals, players WANT to explore what’s out there away from the portal destinations. You could actually explore the landscape… with tedious travel, you’re forced through every area that you don’t want to be in just to get somewhere. It reduces the adventure.

  6. doubledutch says:

    oh and i’m about to finish my degree in psychology.

  7. Wayne says:

    Myself, I’m with the developers as much of the fun of MMO’s has gone away when punishment started to go away. Positive rewards and punishment are your ying and yang, they serve to add an overall sense of balance to the game (at least for me).

  8. Psychology is definitely a good thing to know, a useful tool in the designer’s toolbox. But, this should come with a warning: with great power comes great responsibility. When you get to discussing reward schedules we’ll see this in more detail.

    I think the issue of tedium in game design is a bit more nuanced than you discuss here. I like how you say that negative reinforcement is the removal of punishment to encourage behavior instead of the usual layperson’s definition of being punishment to deter negative behavior. So, I think it’s interesting to look at tedium from the perspective of removing it as a reward.

    Let’s take long travel times. In WoW, let’s look at the old-school tedious run to Un’goro Crater (UC); originally, there was no flight point there, so you had to go to Gadgetzan, get on your land mount (probably not epic at the time) and run through Tanaris. If you wanted to visit the primary quest hub in north UC, Marshal’s Refuge, you’d have to run across that entire zone, too. What was your reward for this long journey? Originally it was interesting because you got to see the zone and perhaps go into the pretty cave of glowing crystals. But, repeating that journey tedium only granted you access to the same quest hub you went to before. The trip was boring, but it wasn’t so long that everyone else didn’t just do it, too.

    Now let’s look at a “more tedious” game: EQ. As Ferrel’s comment shows, there’s been a lot of nostalgia for EQ; I don’t think it’s simply rose-colored glasses at work here, but that the tedium had rewards in EQ. A steep death penalty meant people had to get good or they didn’t advance. The long travel times were long, because there was no automated travel outside of what other players could (or would) provide. But, what was the reward for travel, the end of the tedious run? Often it was a special area. Maybe it was the really great area that everyone talked about as good for earning experience. Or, you might find a great hunting area that wasn’t already overrun with other adventurers camping the good spots. Maybe you’d see a little out of the way monument that people couldn’t just see from the air. Or, perhaps visiting another racial city meant you’d be one of only a few Halflings in the Barbarian town, for example. The “tedium” had a much greater reward than what you would find in WoW.

    I think this is perhaps a more important lesson for designers. Negative reinforcement doesn’t have to be avoided at all costs, but when you do use it make sure the reward is meaningful. Access to yet another quest hub (especially one people have already visited) just doesn’t cut it. Giving the player something special or a unique experience is good, and can even be heightened because of the “tedious” parts previous to it.

  9. I actually like to travel in my games. I have never played an instant-travel game for more than two weeks.

    I thought EQ Luclin, WoW regular (and Northrend without flying mounts), and Ac2 all did a great job with travel. It makes the game more immersive and fun to me. If you have to go afk on a flightpath after the 50th time – OH NO. Each of those games gave a bit of ease without ruining exploration and encountering the unpredictable.

  10. Ninetytwo says:

    “For example, if you take a look at a Counter-Strike deathmatch server. There is no punishment for dieing, yet you still get better and learn how to play / succeed from doing the action itself (playing).”

    Fundamentally different game. Counterstrike employs forced grouping to create its own negative social reinforcement. If you play suboptimally, your teammates will tell you.

    Modern solo-friendly MMOs do not force grouping and allow players to succeed while still playing poorly. There’s very little in the gameplay before max level of most top-shelf MMOs these days (your WoWs and LOTROs and even EQ2) that compels a player to learn to get better if they don’t group or pvp.

    Don’t get me wrong — i prefer today’s solo-friendly MMOs. But they do shrug of a lot of the responsibility of teaching a player how to play by making the games easy enough that you don’t have to play well.

  11. Ferrel says:

    Brian is pretty dead on here. EQ1 fans are frequently criticized with “oh you just fondly remember your first MMO as the best” line. The problem with that is that EQ1 wasn’t my first MMO and I also remember a lot about that game I hated.

    That said, the degree by which designers have attempted to distance themselves from it is disheartening. It seems like the New Designer Handbook page 1 reads: If it happened like this in EQ1, do it different. There is a lot to learn from that game. We’ve lost so much of what made it great by worrying about what made it horrible.

    The explanation of travel from Brian is just one example. I rarely felt put out by traveling Norrath. If I needed to get somewhere quick I could employ a druid or wizard. If it was close I could get a speed buff. Since quests were largely absent one of the wonders of the game was just touring areas of appropriate level in search of great spots. Finding a neat place to grind was literally a big part of the fun. I miss that and I’m pretty tired of running back and forth to quest hubs.

  12. doubledutch says:

    Ferrel:

    You have a point. Games used to have quests that were actual quests. There might be a handful of gather or kill quests in some form, but they were usually a side thought that didn’t define a game.

    Most recent MMO’s basically require questing and on top of that the quests are pathetic. There is such a huge gap between a normal solo quest and a raid instance that it is ridiculous.

    In my opinion, a large part of the problem is not having “epic” feeling solo quests / small group quests and quest lines while leveling. There is so much emphasis on end game and doing those raids, that there is little creative thought applied to the journey.

    …One thing is the lack of quests that take you on an adventure. There are so many divided up zones that you will go to one and complete a bunch of tasks (won’t even call them quests) and then head to the next zone.

    What happened to quests that had you go throughout a continent and complete multiple steps with dungeons, outdoor locations, a bit of farming, and other interesting elements all rolled into one expansive QUEST.

    …this just doesn’t exist anymore other than *maybe* at end game… though usually it’s pieced together pvp and the promise of sitting in an instance.

    The art of the quest has been lost to being a peasant tediously gathering apples for some npc that you care nothing about.

    I hate to blame WoW here, but really, I see it as the first giant step towards the destruction of the “quest”.

  13. doubledutch says:

    I would also like to note that there rarely are places in MMO’s anymore that don’t have a specific reason for being there. A lot of the adventure used to be exploring the world to find special spots that few had been to. Now, every piece of land has a specific use / reason to be there and is a necessity to go to.

    How often to you feel adventure going somewhere you are forced to go?

    ___________________________________________________________________________________________________

    …another thing to think about when considering questing is:

    When is the last time you have looked at a quest walk-through in order to complete a quest while leveling? …and I don’t mean asking in general chat what the coordinates of a kill quest monster are.
    (assuming the game doesn’t put a giant marker over the location and an arrow pointing towards it on your minimap).

  14. I really think this concept is one of the big ones separating the Asian-based MMOs and the US/European ones. Tedium is perceived much differently by players over here in the US than it seems to be in the Asian (especially Korean) markets.

    One of the great things I’ve found in my travels is how and where you play the game has a lot to do with how you perceive the tedium in the game. In this case, almost all the South Korean MMO players I know play in large cafes. You come over, sit down at the computer, pay for a couple of hours play time, and spend time with friends while playing. Tedium is a lot less tedious when there’s a social setting behind the scenes.

    On the other hand, there’s this closet stigma here in the US about MMO players just grinding away in their basement. Well, that happens overseas too — but the concept is taken a bit differently. Usually, that sort of dedication is revered overseas, whereas here, we get a South Park cartoon making fun of killing 3,835,218 boars in WoW to get some shiny mythical sword … while the mom replaces Cartman’s bedpan ever three hours and brings him Cheezypoofs. And all that’s just so the kids can beat some other guy who never leaves his basement.

    That’s social negative reinforcement of tedium — but it only works in certain cultures. That’s why Lineage is/was so big overseas as compared to here. I think going forward, MMOs will either need to be more global in how they handle tedium, or else be confined to certain segments of the marketplace.

    The same can be said for PvP. Some people love games with negative reinforcement for dying, especially if there’s a corresponding reward for winning. That’s what UO had. Asheron’s Call had much the same mechanic, but it was made easier through a limited loss of items, and a calculated formula for what you would lose on death. That, IMHO, was a good balance. Of course, even that mechanic would likely not fly in any mainstream MMO released today. A large segment of people equate any negative loss of items in PvP as a show-stopping reason NOT to play a certain game. It’s okay to get better equipment, a positive reinforcement, but losing items or XP in PvP is seen as bad. Especially when most MMOs see PvP as a big part of what they want people to do in the end game.

    Anyway, good read all around.

    Later,

    Don!

  15. Tesh says:

    I’m a Bartle Explorer, and I detest “meaningful travel” that in reality is merely a time sink. If I want to explore and wander around, I will (as my screenshot folders will attest to), but my schedule is such that if I have to waste even five minutes getting around from place to place, it bothers me. Guild Wars handled it extremely well; I had to walk somewhere the first time, then any time after that I can just warp to any “hub” destination immediately, from anywhere.

    Ultimately, I don’t need a reason to take the long route and smell the roses. I do that on my own if I blasted well feel like it. If I want to jump in and start playing, especially in those instances when I’m meeting up with friends, travel time is onerous.

    Put another way, if you have quick travel, there’s still the option of walking. You’re catering to both preferences, and players have control over how they get around. If you don’t have quick travel, you’re cutting out those who don’t feel like dealing with yet another time sink, and giving them reasons to log off, perhaps for good.

    Forget immersion. If your world is interestingly designed and fun to play in, players will be plenty “immersed” and keep playing (and paying). Forcing players to take the long route in your game might be an effort to make them “respect the game world” (an argument for permadeath as well), but you’re not respecting your players’ time. That’s a cardinal sin in game design.

  16. Pingback: Game Tycoon»Blog Archive » Articles of Interest

  17. Babs says:

    Yet again, Eric, I agree for the most part =) Nice post.

  18. Pingback: Positive vs. Negative Reinforcement « R&D Department