Take Baby Steps

A pet peeve of mine is overblown engines created by teams who haven’t even made a single MMO engine before. The honest truth is that most MMO development teams collapse because they can’t make the technology work. (This is very rarely the reason they admit to, though.) The server is harder to make than they think. The content creation pipeline is vastly more difficult than they expect. So MMO teams fail, more often than not.

In this environment of rampant failure, you would expect to see pragmatic engineers and designers who have modest goals and hope to do just a few cool things with their first game. Make a game with a working engine first, then go crazy with your SECOND game, right? But no, nobody thinks that way. There are lots of reasons why not, but a HUGE part of it is that they don’t think it’s going to be that hard.

“Our engineers are really smart. They’ve read why everybody else failed, and they aren’t going to make those mistakes. We have a brilliant new architecture idea that solves everything.” The sad part is that I never get to say I-told-you-so, because after these guys’ games collapse, the last thing they need is me kicking them when they’re down.

Have you ever looked into the various third-party MMO engines for sale? They are really very primitive, if they work at all. These are companies whose sole reason for existence is to create an MMO engine. And they can’t make it happen in a timely manner. And here your team expects to do all of that, plus make the world’s craziest new features on top of that and add 5,000 hours of content too. In two years. With an estimated cost of just 15 million.

Same old story.

I made the same sort of mistake when I decided to make a casual game. “Sure, everybody says they can’t make real money with indie downloadable games, but they’re doing it wrong. I’m really, really smart. I’m going to knock this out of the park.” My game was basically a failure. It sure didn’t make me rich. But I learned a lot — things I hadn’t even conceived of when I started – and my second casual game could have a real chance of success.

MMO engineers do the same thing. They think they’re going to knock it out of the park on their very first try. But in reality, it’s their second or third try that has a real chance of being amazing. Of course, unlike casual games, an MMO game can often take three, or four, or five years of toiling to get that first game out the door. That’s a big chunk of your lifetime to lose if you’re going to fail in the end.

Engineers are inherently optimists, and the best engineers are incredibly self-confident. I don’t want to change that. Hell, if people honestly assessed the risk involved in creating an MMO, very few teams would try to make one. And that would be sad.

But do me a favor. Start small. Yes, sure, your team is really smart. Way smarter than the team at Perpetual who couldn’t make their server and pipeline work after 4 years of continuous development. Sure, okay. But instead of shooting for the moon right away, could you make a simple version first? Make it a zoned architecture, like an old EQ1 server. Make that work, and if you have time left over, make it zone-free.

Just… take it in steps, okay? Sure, maybe you’ll have time to add flying mounts and realtime terrain deformation. But first just make sure you have path planning and collision detection.

You say your game will be the first to support 50,000 simultaneous players in a non-instanced contiguous landscape? Nice! But before you do that part, can you make sure your server supports 3,000 simultaneous players? (And no, your prototype that can handle 200 connections is not a good enough test. A few hundred are easy. Thousands are surprisingly hard.)

It makes me so sad when people fail after years of blood and sweat. And these failures could often be salvaged into fun games if the team hadn’t shot for the moon right from the get-go. Yes, you’re smart. I sure hope you are, because otherwise you haven’t got a chance of making an MMO. But being smart is just a prerequisite. Those other people who failed… most of them were really smart too.

Realtime vs. Faketime

Developers love new technology. They love it more than players, honestly. This bleeds into developers’ perspectives of their universe, and that’s unfortunate.

Here’s an example: “Wouldn’t it be great if players could beat the ultimate bad guy boss in the story arc, and then a giant rift opens up and a castle appears? That would be so great. Wait, you say that’s been done before?” Back as far as Asheron’s Call 1, developers would pull stunts like this, to great effect. I remember an AC1 event where the ultimate boss dungeon had a big padlock on the door (so to speak) that unlocked itself at a predetermined time. About 12 hours after that, the servers came down. When they came back up, the “aftermath” version of the universe appeared. Players loved it!

“But that’s cheating. It’s not real time, it’s just a hack.” Let’s look at that for a minute, because it’s a pretty common complaint. Sure, it was rigged. The players couldn’t fail, right? Well, in Asheron’s Call 2 we experimented with letting players fail — if a given server hadn’t completed the quest, their world didn’t get updated until later. But it’s still cheating, right? Because it didn’t happen in real time!

Real time wouldn’t really help here, though. Let’s suppose it happened in real time. A group of max-level characters would complete the final quest at 1AM, and everybody who happened to be online at that point would be shown a dazzling real time spectacle. All 50 of ‘em. Everybody else who logged in the next day didn’t get anything out of the “real time” feature you worked so hard on. It’s just not a very good use of development time. And although it’s not as sexy, players enjoy the “fake” version very much. Don’t be afraid to go there just because it isn’t ridiculously over-engineered.

Now don’t get me wrong, some cheap and effective realtime content can go far. In AC1 and AC2, we would have magical portals that appear and disappear in real-time, gating content and quests. Worked great, very easy to implement. Still not sexy (to designers), but it worked great.

Keep it simple! Figure out the easiest way to get the results you want. Do that, not the really hard version. Making and running an MMO is already inconceivably difficult without piling on random features you don’t need.

ACE Library for C++

I’ve been using the ACE C++ library recently and I’m very impressed. It’s an extremely fast portable networking and server-architecture library.

  • Portable primitives for sockets, threads, synchronization, logging, and many other basic components
  • “Plug-and-play” pattern architectures for building high-speed servers
  • A suite of IPC tools for fast portable inter-process communication
  • Extra portability tools such as replacement containers, allocators, strings, etc., in case some of your target platforms don’t have good standard libraries

For me, the magical portability features are huge. I like to develop on Windows and deploy on Linux, but doing this used to be very stressful. The ACE library really takes the nightmare out of it, though. I used to grit my teeth and prepare for serious headaches whenever I copied my Windows code over to the Linux box, but now I trust that it’s going to Just Work.

I think there are two reasons this library hasn’t been a runaway success yet. First, the online tutorials are sub-par. The online references are fine, but for a tutorial you really need to buy the book.

The other problem is that it’s really two libraries in one. It’s both a low-level utility collection and a high-level pattern library in one, and this can cause a bit of a mental disconnect to people who are looking for one half or the other.

Even so, with the book in hand I was up and running in less than a day. I’m very comfortable with C++, so your mileage may vary, but I found it pretty painless. 

The ACE library should be a serious consideration as part of your MMO server architecture, either as the foundation of a full server or as a plug-in replacement for underperforming components written in other languages. This library makes C++ much more appealing as a server platform.

Now if there was only a similar C++ library for portable SQL interfaces…