1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
  2. Hey Guest, is it this your first time on the forums?

    Visit the Beginner's Box

    Introduce yourself, read some of the ins and outs of the community, access to useful links and information.

    Dismiss Notice

Complete TCPR documentation

Discussion in 'Modding [KAG]' started by Asu, Jul 2, 2017.

  1. Asu

    Asu THD Team THD Team Forum Moderator

    Messages:
    1,580
    Hello,

    As part of the recent KAG community documentation project happening over Github pages, I've written a complete TCPR documentation, including a python example.
    I can create a C++ example (with SFML network or boost::asio). If you are interested, PM me or ping me on Discord, I might start writing it for the docs.
    Link to the documentation: https://kagdocs.github.io/#tcprtcpr
    Link to the repository: https://github.com/kagdocs/kagdocs.github.io/tree/master/_tcpr

    Feel free to pull request if you want to add improvements. Enjoy :thumbs_up:
     
  2. Geti

    Geti Please avoid PMing me (poke a mod instead) THD Team Administrator Global Moderator

    Messages:
    3,730
    Nice to see it getting some attention. I've considered on-and-off a more streamlined "external mod process" system based on stdin/stdout (KAG would start the mod process) but I'm waiting to see any serious adoption through TCPR before I bother. TCPR is hands-down the most powerful modding "level" in KAG as you can do whatever you want, do it asynchronously and leverage external libraries in whatever language you like. Writing the TCPR bridge code into KAG can be a chore but it's not too bad tbh.

    The worst part is quite poor tools for parsing AS-side.


    Fwiw I've kept most of my TCPR stuff in fairly high level languages, node in particular as there's a lot of libraries available to leverage very easily for things like databases, dealing with different file types, etc.

    If you wanted to run on a really low spec server writing something in C++ might make sense but most of the time you wont be doing particularly heavy stuff over TCPR because you can only send so much anyway.

    Some corrections (don't have time to write up a PR today):
    • Exceeding the line length will cause a line to get truncated (at the limit) and the rest of the line will be interpreted as the next "line", both of which may lead to unintended effects. It's recommended to put any extensive logic on the server as a rules script and feed it a few arguments instead of sending huge swathes of script inline each time - it's much faster as not so much code needs to be loaded/compiled each time, and wastes less bandwidth as a bonus. The line length can potentially be lifted to something much higher (lines could be accumulated past the buffer length) but I don't really want to encourage sending a heap of data at once or leave the game open to crashes from sending 4gb of whitespace, no newline. If it's a blocker for someone I'm happy to negotiate though.
    • TCP in general does not have "packets" (at least at the application level) - sockets are stream-based and separate messages might be combined (more on this later).
    • Both \r\n and \n newlines are supported - whatever one you use to auth is the one that will be used for the rest of the session, and this can be different between separate concurrently-connected TCPR clients.
    • Your example should probably accumulate received data, split on newline and only parse/handle complete lines. You've mentioned this in the paragraph above but don't practice it. It doesn't matter in your example because you're just echoing everything that you receive, but I'd expect new interested programmers reading the docs to often copy example code verbatim, so we'll have fewer confused newbies if your example handles multiple lines properly.
    Shit, that's not intended at all, haha! This wasn't reported earlier :D
     
    Asu likes this.
  3. Asu

    Asu THD Team THD Team Forum Moderator

    Messages:
    1,580
    Thanks, I corrected the TCPR protocol part out. I don't know much about networking yet, it's a field I have yet to look up o:
    As for the example code I will edit it out I guess, I'm not a python developer so I spent most of the time looking up documentation and examples. It's quite unfortunate that it looks fairly complicated (compared to what it should be) to read the stream until the newline character. boost::asio seems significantly more convenient about that, but I guess python sockets are meant to be fairly low level. BTW, does the newline used during auth affects the newline sent by the server?
    --- Double Post Merged, Jul 6, 2017, Original Post Date: Jul 6, 2017 ---
    Are you sure there is a 16k bytes limit? I've edited the python program to properly read until \n.
    For the sake of the experiment I've run this:
    Code:
    string s; for (int i = 0; i < 20000; i++) { s += 'a'; } tcpr(s);
    The line doesn't get truncated, no timestamp appears within the data block either. len(line) does output 20011 or so as expected. It works with even higher values.

    Edit: Python code updated; now it instead detects chat messages and broadcasts a /msg when a player sends !help. It now errors properly when failing to connect and when the connection closes, and reads and splits lines properly.
     
    Last edited: Jul 6, 2017
  4. Geti

    Geti Please avoid PMing me (poke a mod instead) THD Team Administrator Global Moderator

    Messages:
    3,730
    Nope there's no 16kb limit from the engine, only to the engine.

    Don't think so. I think the OS you're running it on might affect that actually? Would have to test.

    Code:
    # read the stream
    partial += sock.recv(16*1024).decode()
    # if there's a newline present
    if(partial.find("\n") != -1):
        # split into lines
        lines = partial.splitlines()
        # save (and remove) the remainder
        partial = lines.pop()
        # handle all lines
        for line in lines:
            # might want to trim any \r characters here maybe
            # not sure if they're sent on windows :)
            handle(line)
    Like, it's not lovely but it's nothing terrible; I'm sure you worked out something similar for your example :)
     
    Asu likes this.