Wednesday, December 22, 2010

The Real Priorities of Online Game Engineering

I was trying to communicate to management that server developers have different priorities than game developers. As a means to show the importance of laying in administrative infrastructure, and other software engineering "overhead", I put this list together. Hope it helps you to think about making the right investment in making the system sustainable, and make those points to the powers that be.


This is a list of priorities in absolute order of importance. While it is good to address all of them, if we don’t have one of the higher priority requirements solved to a reasonable degree, there is not much point in having the lower ones.

I made this to help us focus on what is important, what order to do things, and what we might cut initially. I’d love to debate this over lunch with anyone. I’m hoping others think of more of these kind of driving requirements.
  1. Don’t get sued. In particular, always deal with child safety. We also need to abide by our IP contract obligations (sometimes including the shipping date). Better to not ship than get sued into oblivion or go to jail.
  2. Protect the game’s reputation. Even if the game is awesome, if the public thinks it isn’t or the service is poor, then we lose. This is especially important early in the lifecycle. This implies not shipping too early.
  3. Be able to collect money. Even if there is no game.
  4. Be able to roll out a new version periodically. Even if the game is broken or not finished, this means we can fix it. This implies:
    1. You can make a build
    2. You can QA it
    3. You can deploy it without destroying what is already there, or at least roll back
  5. Support effort is sustainable. If the game and system are perfect, but it needs so much handholding that our staff burns out, or we don’t have the resources to extend it, we still fail. This implies lots of stuff:
    1. It is stable enough that staff is not working night and day to hold its hand.
    2. There is enough automated maintenance to limit busy work
    3. There is enough automated logging, metrics and alarms to limit time spent hovering
  6. The cost of operating is not excessive. I.e. it sits lightly enough on the hardware that we don’t need massive amounts, or exotic types. (Special warning to engineers: it is all the way down here before we start to care about performance. And the only reason to care about performance is operating cost.)
  7. Enough players can connect. This implies lots of stuff:
    1. The cluster hardware exists at all, the network is set up, etc
    2. There is a web site
    3. Key platform and login features exist
    4. There are enough related server and game features
  8. The server is sufficiently stable that players can remain connected long enough. This implies lots of stuff:
    1. It stays up.
    2. There are no experience-ruining bugs or tuning problems.
    3. Not too much lost progress when things crash.
    4. The load is not allowed to get too high (population caps)
      • This is probably about where we need to get before Closed Beta.
  9. Revenues exceed cost of operation. And eventually, cost of development. This implies not shipping too late. Note that you don't *have* get to this point immediately. And that this is more important than having a fun game.
  10. The game is fun. This implies so much stuff, I won’t write it all down. Note that the requirement to not ruin the game's reputation can move some of this stuff earlier. But don't fool yourself. If you are making money on a game that is not fun, is that bad? I'm sure you can think of some examples of this. Here are some server-specific implications:
    1. You aren’t put in a login queue for too long. You don’t have trouble finding a good time to play.
    2. You aren’t dropped out of the game too often.
    3. The feeling of lag is not that bad.
    4. You can find people to play with. It is an online game, after all.
  11. Players cannot ruin one another’s fun. Note that making the game cheat proof is not the requirement here. The only reason you care about cheating is if other players perceive it badly enough (reputation), or if the players are keeping you from making money.
    1. They cannot grief one another, especially newbs.
    2. They cannot bring down the server
    3. They cannot ruin the gameplay or economy, making swatches of gameplay pointless or boring.
  12. The server can scale to very large numbers of players. This is the profit multiplier.
Be honest with yourself. Are you over engineering? Solving fun technical problems that don't actually address any of these Real Priorities? Doing things in the right order? Remember, as the online engineer, you represent these priorities to management. They may not (yet) understand why this order is important.

Tuesday, December 14, 2010

"Restraint in what you ask for is the key to success" - Arnold's Aphorism

A good friend of mine sent me an email about our current sprint (we use a relatively formal Scrum process), encouraging us to define our stories such that the team could experience a success. He is also a great student of military history. I assumed the quote below was from some past general or philosopher. He claims not. So I've named it after him and am spreading the word.

"Restraint in what you ask for is the key to success" - Arnold's Aphorism

I can picture Napoleon fighting with himself over asking his men to achieve a little too much and fail. Or ask less and wind up with better morale. Applying this to software engineering teams makes a lot of sense. If you always ask for more than can be successfully completed, the team may feel unsuccessful/underachieving, and you may feel disappointed. When in fact, they are doing almost the same work whether you expected more or not.

Here's the thing, though. It's been said before. In all things, moderation. Time management, and interpersonal relationship books talk about expectation management. "Under promise and over deliver". The pessimist is never disappointed and often surprised. Live beneath your means. "Let go". Thou shalt not covet.

This is probably more true of things we ask of ourselves: get this, do that, convince them...

What struck me in the phrasing of this old-new aphorism is how clearly it shows that you have the *choice* in setting the expectations. And yet, that goal is the very thing that defines success. And for many goal-oriented folks like us, that success defines our happiness. Ergo, we choose to be happy. Or not.

But it requires self discipline. Not so much in the exercise of effort, but in the restraining of wanting.

You may have heard analyses of the great American marketing machine. It generates unrequited desires, while offering to fulfill them (for a price). The promise of happiness. Oddly, it is more commonly the restraint of those desires that leads to happiness, not the fulfillment.

I think you are going to see many more instances of this wisdom over then next while. See if you can't identify the aspect that you control.