Tuesday, May 15, 2012

Ultimate Open Source Game Development System

I'm between jobs again. It happens a lot in this industry. One of the frustrating things about that is you leave behind investments you've made in building infrastructure that was supposed to save you effort on future projects. If the company you're leaving fails, that infrastructure investment won't help anyone. So what do you do? Just build another one for the next company? Rinse, repeat. Buy something? Can't influence that much. Build a company that sells infrastructure? Pretty hard to make a profit selling to us picky developers. Open source? Let's think about that...

If I was to build an open source game development system, what would I focus on? I'm not a graphics whiz, but I know about simulation, OO, distributed systems, MMOs, and picky developers. Let's see here...
  • Pick a primary development language. But realize that not all developers will love it. Is there a way to support multiple languages?
  • Rapid iteration is key, so game logic must be able to be written in a scripting language. But it must also be possible to hard code parts of it for performance.
  • Tools are key. In a good game development project the majority of the effort of the team should feed through the content tools, not the compiler. Wouldn't it be ideal if you could build a great game without any programmers?
  • It must be debuggable. In a distributed system, this requires some thought.
  • The world size must be able to scale. A lot of projects are bending their game design to avoid this problem, and that is definitely the least expensive approach. But if your infrastructure supports large scale on day one, what could you do?
  • You want reuse of game logic, realizing different developers/designers have different skills. This means you want a hierachy of game elements that are developed by appropriate experts, and snapped together by others. This game object and level design effort should be efficient and fun.
  • The team size must be able to scale; both up and down. This is a matter of content management. You don't want central locked files everyone contends for, and you don't want burdensome processes if you are a small team.
  • It should be runtime efficient in space and time. This enables use on games that have huge numbers of game object instances.
  • It is easy to ask for dynamic object definition, but that can work against performance. How often is that used? And what other ways are there realize that effect?
  • The framework should be intuitive to as many people as possible. This means being very careful about terminology.
  • Now we enter an area I don't know much about. How do you structure the infrastructure development project itself? You want buy in from lots of people, but you also need a single vision so the result is consistent. You need a means to adjust the vision without blowing up the whole effort, and without encouraging people to fork the effort.
  • What will be the relationship to other open source projects? We can choose to use various utility libraries. But what about using other game projects for graphics? Issues will arise at the boundaries.
  • The system should be useful and get some adoption even without being finished. Because it will never be finished.
  • It should be modular enough. That allows a developer to use the parts they want, and replace the parts they must. It allows broken parts to be replaced.
  • We will need a demo game. How ambitious should it be?
  • We need a name. And a vision statement.
Is this kind of thing really needed by our industry? Why is so much infrastruture rewritten all the time? Is it just the projects I've hit? Do we always push the envelope such that if this existed today we'd spend our entire engineering budget extending it? Or is it a fundamental truth that there can be no Ultimate infrastructure because every game is too different? Lost cause, or an idea whose time has come.

I think the first thing to do is a survey the current state of open source game systems.


  1. Very interesting to read, it really got me going! I've also been thinking about an open source framework for MMOGs (I'm currently developing independently on such a game). Hope you don't mind me sharing some thoughts I have reading your post!

    Development language: Simply choose one to go with for each distinct layer / autonomous component server-side. They must not be the same but it's probably easier if they are for the team working on this. Allow more flexibility client-side.

    "Wouldn't it be ideal if you could build a great game without any programmers?"

    That kind of vision has been around since Cobol was created and I've seen it time and time again how packaged systems claim that non-engineers can configure and extend them. Since the customer doesn't employ any own team of developers for the system they always end up with expensive, properly skilled consultants to maintain and develop them. For years.

    I would say instead: Build it for developers, but of course design to make their work as effective and efficient as possible. "Delegate" any non-logic, and logic rule sets, work to tools as far as possible of course.

    The world size must be able to scale: If I understand you correctly can this be phrased as small initial footprint but linear scalability? Those are good goals and is what I'm shooting for in the MMOG I'm working on.

    What should the principles for modules be? Should they all be truly independent components (not requiring any of the others)?

    I's suspect that any game platform to some extent will be tailored to some fundamental game entity and event model. If so, should such an open source project be tailored for e.g. 60% of conceivable MMOGs well, or e.g. 90% but not that well?

    Some ideas for modules:

    - Simulation engine (possibly both server-side and client-side components)

    - Rule engine (I'm not too familiar with these but imagine it appopriate for RPGs)

    - Contract, mission, quest and in-game transactions

    - Persistency layer

    - Server monitoring and administration

    - Player session handling

    - User authentication and authorization, perhaps also support for payments/subscription handling etc

    Fat clients:

    - Low-level IO (without semantic data knowledge), tailored for MMO requirements

    - High-level IO, with general mechanisms for authentication, MMO-style events, data feeds/state updates, RPCs, bandwidth throttling etc

    - Client UI modules - I'm not familiar with the off-the-shelf UI and 3D softwares out there, but imagine that if some game model is developed for this platform, some adaptor modules might be useful towards e.g. Frostbite or Unreal 3D engines. Requires some expertise to look into.

    Mobile clients: IO

    Web clients: web app APIs

  2. yeah i know people will attack me after saying this, but open source projects mostly sucks, forget all those freedom bla bla bla, usually there is no stable output