Who’s In Charge of Quality?

Who gets to say when an MMO patch is ‘done’? Does QA answer to the live producer? Can QA stop the launch of a patch or does the producer alone have that responsibility?

Every live MMO team I’ve been on has had this debate at one time or another, and with some of them it’s an ongoing power struggle that ripples out and affects development in a huge way.

A head QA honcho that I worked with once told me that my job as producer was to produce — to push the product as spec’d out the door on time. And his job as QA was to make sure that the product I pushed out was acceptable and didn’t reflect badly on the company. That meant that the final call on pushing a patch was his responsibility, he said. He could stop the patch if he felt it was unacceptable.

But while that model of control may work perfectly well for a pre-launch MMO or a one-shot boxed game, it doesn’t make any sense for a live MMO – especially not one with a constant (and regular) content update schedule.

The producer of a live MMO is different from the producer of a pre-launch product. We have different responsibilities and goals. We answer much more directly to the customers. Our bottom line rests less on development costs and more on customer retention.

While the live producer is still in charge of scope and schedule, both the scope and schedule of a live MMO are much more flexible in the short term than those of a pre-launch product. Whether you are patching once a month or every three months, there will be another patch. If you have to, you can slip content or push your patch a week without screwing up a multi-million dollar advertising budget.

As producer, I used QA as a resource (one of the my most valuable resources – no argument there!) to tell me the state of the patch. I used that information to decide what was ready to go and what needed to slip to the next patch. And yeah, I pushed without QA sign-off once or twice. After gathering all the information and balancing all the requirements, that was the action I decided was necessary.

The live producer needs to sit down with the people who are invested in the patch – QA, CS, etc. – to work out any difficulties. But giving one of those groups a flat out veto? Not useful. Not in the live MMO world.

This entry was posted in Production. Bookmark the permalink.

3 Responses to Who’s In Charge of Quality?

  1. Jason says:

    To use politics as a comparison, I would say that the producer writes the Bill and QA gets a line item veto. The producers lays out all the things they want into a patch, and QA gets to say if any of those things aren’t ready. The producer either repacks the patch and lets the QA failed items slip to the next patch, or the producer decides to hold the patch until the QA failed items are completed.

    At least that’s what seems logical to me.

  2. As a producer, I can’t live without QA. It’s the department the entire studio relies upon to tell us what the bugs are and reproduce them so they can be fixed. Sending anything live, from game launch to a desperate hotfix, without QA attention is insanely dangerous to the future survival of the game.

    However, the ultimate decision of what bugs you’ll live with and what bugs MUST be fixed in the next release lies with the top decision-maker(s) running the game. Game quality, customer satisfaction, later fixability (and how late you’ll need to wait), cash flow, and long term business survival all factor into this judgment. Many aspects of this decision require information unavailable to a QA manager. Quite a few aspects of the decision are ultimately judgment calls, since you never have enough time to perform QA and fix bugs.

    A QA manager who claims this “final decision maker” level of responsibility is either very inexperienced or a petty bureaucrat desperately seeking authority. A company that endows a QA manager with this level of authority probably has equally inexperienced or petty bureaucratic management.

  3. Sandra says:

    Actually I think both of you did a much better job of summing up what I was trying to say than I did. Thank you!

    (Psst! Would you like to write this week’s post for me? *grin*)