How does KAG's networking work?

Other people might use the information that comes out of this thread later.

 

Note: this is my understanding, the networking is more MM's domain but I've had to touch basically every part of it at least once.

KAG's networking is client-authority for player objects, aggressively delta-compressed and lz-compressed before being sent over the wire. Data is sent over ENet, which basically enhances UDP with optional reliability guarantees across separate streams and packet-splitting.

Client authority:

• Clients are "in charge" of their blob. This is the cause of a lot of the lag-related physics weirdness in KAG and allows things like speedhacks, but also means we don't need clientside prediction, don't suffer rubber-banding, and generally simplifies thinking about networking. Non-player objects are owned by the server and otherwise behave the same.

Delta Compression:

• For each net-object, figure out change compared to last frame
• If nothing changed, send nothing
• If something changed, figure out exactly what changed and encode that as small as you can.
  This involves basically having a 1-bit flag for each possible "section" of a delta, which says if it's included. This prevents sending stuff like rotation for stuff that never rotates, at the cost of 1 extra bit in the worst case.
• Some fields (not many) are sent as fixed rather than floating values. This can be (generally isn't) done dynamically (eg a 32 bit float within the 0-1 range might be sent as 8 bit or 16 bit fixed, otherwise as 32 bit float) - this is tagged before the data so you know how much to read. YMMV, this is micro-optimisation :)
  
• For properties, changes are detected during a sync() call vs the last synced value - this means there's no delta cost for non-synced properties. Properties changes are sent as a null-terminated list of name hash, type and data
• Fun Fact: Delta compression is actually the #1 CPU-hog in KAG. Our implementation is super awful, because it stores the previous data in bitstream-encoded form which has to be read out every subsequent frame. Be careful how you implement this.

LZ compression:

• Each frame's net data is lz compressed (I think with fastLZ? one of the realtime-focussed libs that was around last decade) before being handed off to ENet. This is mostly a gain for strings and integers. The ratios aren't great but it's worth doing.

ENet:

• ENet handles getting this synced between the machines - including splitting up too-big frames of data and ensuring it's handled in-order and all that good stuff.
  
• There's a lot more you could do with ENet that we don't do, mainly to do with separating unrelated streams and using unreliable data more. We just use one big stream and almost everything is sent reliably in that stream.
  
• For transient data it's often better to just have the most recent version of something - you've got to be careful about the interaction with delta-compression here though, because if you only send something once, and then it doesn't get there - nothing gets there and you get desync.