U61 (v1.1.0) - Technical notes




Global architecture
===================


  About C++ and Lua
  -----------------

    U61 is a (rather big) monolithic C++ program, which embeds an Lua
    interpreter. This interpreter reads and runs Lua scripts which can be
    tweaked by players to adjust rules to their taste.

    There's much more C++ code than Lua scripts. Indeed Lua is used almost
    nowhere in the game but for rules definition.

    As I get more and more experience in programming I feel that this is not
    especially a very good design. Indeed there's plenty of inelegant and
    poorly written C++ code in U61, which would have been clearly feasible in
    any common scripting language, be it Lua, Python, Perl, Scheme, whatever.
    There are several reasons which led me to chose not to use Lua more often
    in the game:

    * I wanted to clearly separate the C++ game core from the Lua user scripts,
      although I did not really realize that this separation could have been
      done even if I had used Lua in the game core. I would certainly have
      needed to set up different environments for the Lua interpreters, but it
      was possible.

    * I wanted the game core to run as fast as possible, and did not want to
      waste CPU time by using interpreted code in it. Reality shows that ugly
      akward C++ code rarely runs fast and U61 is no exception. It's slow for
      what it does. Mark my words, I did not say it would not run fast on your
      computer, only it could drop down from 30% CPU used to something like
      maybe 3% if it had been written in a smarter way.

    * When I first coded U61, C++ was still a language I estimated, and I
      really thought I'd make great things with it. Experience showed me that
      me and C++ are definitely not made for each other. Now I really hate this
      language, find it ugly, hard to program, consider that its
      object-oriented aspect is very complicated to master, especially since
      I've used scripting languages like Python, where OO programming does make
      sense, and does not look like an ugly pile of hacks and tricks.

    * Lack of experience. When I started U61, I did not really know much about
      scripting languages like Python (http://www.python.org) and ignored cool
      tools like SWIG (http://www.swig.org/). With these tools I would have
      ended up with a much cleaner design than the current monster C++ core. I
      would still have chosen Lua as an extension language for it perfectly
      fits, but for instance all the GUI (menus) would have been better written
      in Python.

  ClanLib
  -------

    U61 clearly depends on ClanLib for most of its tasks, including graphics
    and sounds.

    However, if you look at the code you'll notice there are many features
    present in ClanLib, that I do not use in U61. Worse, I have my own - and
    often limited and buggy - implementation of a bunch of things, for
    instance:

    * network communication

    * menu handling

    There's a good reason for this: U61 is a "long term project" for me, and
    has been arround for already 3 years. In 3 years, ClanLib has moved from
    version 0.2.x to 0.7.x, and a large part of its API has changed.

    ClanLib, as of version 0.2.x, was almost all I needed for a game library.
    My only real problem with it was that it was not completely finished, it
    lacked a few minor features, and some stability, but basically things were
    there.

    Point is that over the years the ClanLib folks have added an impressive
    number of new features, also fixed a bunch of design flaws, but doing this
    they have broken compatibility several times, with honestly very few
    concrete improvements from my egocentric "focused on U61" point of view.

    As I write these lines ClanLib seems to aim at being a complete game
    developpement environment with support of fancy hardware acceleration and
    so on. But I do not need this at all for U61. U61 has very basic graphical
    needs, won't really gain anything with OpenGL, has its GUI already
    hard-coded. All it needs is just plain old ClanLib 0.2.x with a few patch
    applied - I'm a bit exagerating, but not that much.

    So for instance, concerning network code, I've simply moved away from
    ClanLib. When Magnus Norddahl (ClanLib's main developper/project leader)
    came with his ClanNetwork 2 implementation, I did not have the courage to
    make the migration from ClanNetwork 1 to ClanNetwork 2. No matter how many
    fancy options ClanNetwork 2 has, what I needed was to be 100% sure that I
    would never ever need to rewrite my network code when changing to
    ClanNetwork 3, 4, 5 etc. Added to this, I know a bit about POSIX socket
    programming, so changing form ClanNetwork 1 to POSIX sockets was even
    simpler for me than changing to ClanNetwork 2. Now U61 does not need
    ClanNetwork any more, it has its own low-level network API wrapper, and
    even if it's poorly written it compiles and performs good enough for U61
    needs.

    As long as ClanLib 0.6.x is out and that there's a reasonnably easy way to
    compile U61 "as is" with some ClanLib version, U61 will still use ClanLib.
    However, should ClanLib's display and/or sound API change, I might as well
    completely switch away from ClanLib to some more "API-stable" library,
    maybe SDL (http://ww.libsdl.org) or Allegro (http://alleg.sourceforge.net).
    AFAIK ClanLib O.7.x has a different way to handle graphics than ClanLib
    0.6.x. What I will do will depend on how tricky it will be to switch from
    0.6.x to 0.7.x.

    This is sad for ClanLib is a remarkable piece of software, is well written
    and feature rich, but the fact that its API changes so often makes it
    rather complicated for me to develop with it, especially when my projects
    (like U61) take several years to be completed, and are supposed to be also
    avaiblable several years after they've been completed.



Security
========


  Buffer overflows
  ----------------

    Let's be clear: U61 is a game and it's no heavily secured enterprise bullet
    proof application.

    However I tried and did my best to provide a secured enough environment.
    For instance, all string manipulations are done using snprintf-like
    functions, instead of the default sprintf-like functions, which are known
    as being very likely to create vulnerabilities.

    Still, there might be some security hole in U61 - as in any program BTW -
    so use it at your own risks, and do not run it as root or any super user
    account!

  Remote Lua code execution
  -------------------------

    One of the main features of U61 is that its rules are customizable. A
    consequence of this is that in network games, all the clients download the
    game rules from the server, which is an Lua source code, and execute this
    script.

    So U61 does execute remote code within its embedded Lua interpreter, and
    does not perform any advanced checking before executing it. From a
    theorical point of view, this is a very good place to put some sort of
    virus or trojan horse.

    Still, this code is executed in a restricted environment, it should have no
    access to I/O functions which could allow it to retrieve files from the
    hard drive, communicate using the network interface and so on. So in
    practice, I've taken the necessary steps to secure this aspect of the
    program and no trouble should come from it.

    However, you should be warned that U61 gets files from the network,
    executes them and stores them on your hard drive.

    BTW this is exactly what a web browser does when you view a page containing
    scripts. It gets the HTML file from the network, executes the JavaScript
    code in it, and stores it on your hard drive in some cache folder.
    Experience shows that most browsers have had security holes - most of them
    being fixed rapidly - so I'd be very lucky if U61 had no potential security
    hole at all 8-)

  Passwords
  ---------

    The network password is stored as plain text on your hard drive, in the
    "config.lua" file, so anyone capable of reading files on your account can
    technically read this password. So it's wise to use a "weak" password like
    "hello".

    However, passwords are not sent as plain text on the network, instead U61
    sends an MD5 checksum of the password, so it's not possible to use a
    sniffer to capture the password when you join a network game.



Network model
=============


  U61 is different
  ----------------

    Most modern network games use a client/server model, where the server
    maintains a "game state". The clients send informations describing what the
    player did, the server processes these informations, updates the "game
    state" and sends informations back to the client.

    This model has several advantages, but one of its big drawbacks is that
    when a client has a slow connection with the server the game becomes
    "laggy", and is usually no more fun to play.

    U61 does not use this model. In fact, both server and client maintain their
    own "game state", and the only information which is normally sent on the
    network is messages like "player X pressed key Y".

  Network lag
  -----------

    The reason I chose this design in the first place is that it allows each
    client/server to handle its own players in real-time, without waiting for
    informations coming from the server.

    This way, players on a slow-linked client never really "feel" the lag with
    the server. From their point of view everything is done locally, when they
    press a key, the effect of this key is immediate and they do not need to
    wait until the information "I want this block to be dropped" is sent to the
    server, processed, and re-sent back to the client.

    Of course, this is only true for local players, there can still be some lag
    with remote players. U61 solves this problem by "anticipating" the remote
    players map states. Before showing remote players maps, it calculates the
    map state "as it would be now if the player had pressed no key at all".
    Whenever it receives a key stroke event, it performs a sort of "rollback",
    and corrects the map so that it takes the key stroke in account.

    This explains that when you play over a slow link on the network, you might
    see little quirks in remote players maps behaviours, with some impossible
    moves. However my opinion is that this is a lesser problem than true real
    lag which forbids smooth play.

  Synchronization
  ---------------

    Another important issue is synchronization between clients. Indeed, since
    each client maintains its own game state for all players, a bug in the
    implementation could easily lead to a total desynchronization of the game.
    By desynchronization I mean that after some times the maps for a given
    player could be completely different on different computers.

    Since only key strokes are transmitted, this is theoricall possible, the
    implementation bug being "a key stroke does not have the same effect on 2
    different computers".

    This explains why it's impossible to use the builtin system random number
    generator while writing Lua scripts for U61. Indeed the generated random
    number wouldn't be the same on each computer (or one would need to "seed"
    it all the time, and it would not be really random anymore...).

  Cheating
  --------

    Another nice aspect of this network model is that it's practically
    impossible to cheat. Indeed a player can't really "fake" his map, since its
    map state is calculated from key strokes on every networked computer.

    Running a modified server wouldn't even really allow someone to cheat since
    clients would continue to process key strokes in a normal way and give the
    "right" map states to players. The only thing a wicked server could do is
    to mess up key strokes and send wrong informations, this is somewhat
    equivalent to make a bot play instead of a human players. Until someone
    writes an AI capable of playing U61 better than a human - given the fact
    that rules are changeable - this is not likely to be a real problem.

    Additionnaly, U61 clients send and expect "checksums" of maps on a regular
    basis, and if these checksums are wrong are missing, it dumps error
    messages in the log. This is usefull to track down bugs, and also allows
    you to figure if someone's trying to cheat. Again, it's still possible for
    a wicked server to send the right checksum but maintain a wrong map state,
    but what good it be good for, since this server would virtually be playing
    another game.

    Finally, you could technically write a U61 client and/or server that would
    display "Player A has won with 1000000 points", the problem being that all
    other connected clients would see the real pathetic score of 3 points 8-)

  Possible improvements
  ---------------------

    One major improvement I'm thinking about would be to completely remove the
    client/server aspect, and make all clients potential servers. This way node
    A could start a game, node B could join it, then C and D could connect on
    B, A could leave the game and E would still be able to connect on C.

    This could lead to endless - and hopefully fun - games where players would
    connect / disconnect as they wish, with no need for a permanent server.

