I must be tired. I tried for analogy and got lethargy instead =/

In my experience, however, user testing outside of the obligatory beta has actually become less prevalent for MMO games. I know AC1 did some basic user testing, largely at Microsoft’s urging — I had the videotapes stored under my desk for several years, even though I wasn’t around to see the original sessions. But I’m not aware of many games since then that have made the effort.

But even if user testing is done relatively late, I think it can really help target the polish in a very useful way. And polish — what I might call getting out of the player’s way — is way more important than we give it credit for even today. For instance, watching Eric play through the newbie area of Greater Faydark in EQII was an education in itself. *grin* That sort of thing can really help smooth out unclear directions, confusing terrain, misplace fight difficulty — all the little bits that can drive players away in a heartbeat.

Babs: “But imagine if an MMO spent parallel amounts of time in both tool-building =and= analyzing Venn diagrams of their target audience. They’d never launch” — oh, I disagree. Considering how absolutely little time most games put into tool-building, we’d only need a day or two of thinking about the target audience to reach parity! *grin*

Unfortunately I think the odds of an MMO being “successful” are currently way below 50%. I have to believe that there’s something we can do to improve the situation. I will concede, however, that given the current megahit-focused investment model, it probably is easier and no less predictive to go with our gut.

Agreed. And yet we do it so often it’s rote at this point, isn’t it? Because we’re constantly seeking that single quality that has universal appeal. One game to rule them all.

But imagine if an MMO spent parallel amounts of time in both tool-building =and= analyzing Venn diagrams of their target audience. They’d never launch =) Which is not to say they shouldn’t be looking for the overlap between person 1 and person 2, it’s just that they end up with the same 50% chance to fail by thoughtful analysis as by having the wrong gut feeling about features. For amount of effort required, it’s cheaper to go with one’s gut.

Personally I couldn’t agree with you more. I work in web design and often find it an uphill battle to explain that even within a single marketing demographic you will see multiple user types and multiple user behaviours. We’ve had some headway with doing ethnographic based persona development workshops where we invite the stakeholders, we often generate 10+ personas and the stakeholders see how difficult it is to narrow that field down :)

The true turning point though is when we can get them into our usability lab to watch (and this is key) ‘more than one’ usability session. When they actually see 2 users who fit their demographic profile behaving completely differently as they interact with the prototype site it suddenly becomes very real for them. After that they are a bit more reasonable when it comes to our suggestions and site designs :)

I know that user testing is become more prevalent in the gaming world but I wonder how often the decision makers, producers, developers etc sit in on these sessions and see how actual users ‘use’ the game. Would this have an impact on the overall game design, or is it too late by this stage for this kind of insight to have an impact?