[1239] Too many knights

Just dropping in to say that while we're well aware of the lacklustre performance, there are a lot of things standing in the way of "just fixing it", and preventing us from treating it as a higher priority than we already do (which is much higher than some seem to think).

• The main reason KAG is slow is that 80%+ of the game code is in AngelScript. From our measurements, AS is about an order of magnitude slower than C++. The reason for so much of the game being in AS is the community asking for more and more and more modding opportunities, which we gave them by exposing the majority of the game's code as scripts.
  
• On top of this, the way the scripts are run (sequentially for each component of each object) leaves us no room to optimise their performance at an engine level. There is no way to parallelise ("multi-thread") script execution without introducing race conditions (unpredictable behaviour) and crashes without big changes (see below).
• On top of that, the component system behind the scripts is written in a reasonably misinformed way - when everything was C++ (classic), it didn't really matter because the updates were all native code, small in terms of code size and reasonably deterministic. Now, each update of each component could run any number of scripts (which can further change at run-time) and each time needs to pass stuff between the barrier between C++ and AS. This architecture issue is also what locks the way the scripts are run in place, and prevents threading where it might otherwise be possible.
• On top of that the game's networking code is based on a system designed for deathmatch games, with tens, not hundreds of objects; and 8, not 32 players. As many have seen, this system can choke with some extreme loads, particularly high player count when there's a lot going on. This about half the reason server CPUs are strained so much and tend to cap out before clients (tomato bombs while still at 30fps)
  
• To change any of this, we'd have to basically "pull the guts out of" the game's component system, which would take a few months of solid work, and would break all mods and mods of the game scripts. Then we'd have to fix all the scripts that were broken by those changes (which would be most of them), which would largely involve rewriting the game, get modders to re-write their mods, and we'd STILL be fighting an uphill battle against the fact that most of the game is in scripts.

As such, a full re-architecture of the engine is unlikely to happen, and (unfortunately) the "lag issues" are unlikely to ever change in a particularly meaningful, across the board kind of way.

We've learned a lot of lessons through this, and there are things you can do script-side to ensure acceptable performance (which we are applying to the objects in Trench Run), but ultimately we'll need to apply our experience to a new engine in the future.

Some hope: we do have intentions to move some of the more intensive scripts back into "hard coded" engine-side logic blocks, which would run faster. This would require translating various scripts from AS to C++ and giving them reserved names in the configs, which isn't a huge deal - we just need to gather data on which scripts would benefit from this most.

@Rayne @UnnamedPlayer feel free to move this somewhere more appropriate/visible.

@everyone else I'm happy to elaborate more on this if you're interested.

 

Server performance: not much - as I said, servers are usually limited by the netcode strain before they're limited by the actual game code, at least for dedis. Might let you squeeze in another player before you start getting tomato bombs I guess.
Struggling client performance: could be the difference between "OMFG" and "playable". Some people get backbreaking FPS lag with a decent number of players and after a fair bit of time has passed. We need more diagnostic information to find the problem scripts and any other causes though - usually all we get is frustration or vitriol from affected parties though. @Rayne in particular has been helpful identifying a few troublesome scripts but it's an uphill battle as those affected often don't want to play that much, particularly on big servers, and are unaware of the script profiling options available.