Linux 100% cpu load; API call failed

I am just wondering, one of my dedicated servers keeps having problems with servers taking up 100% of the possible CPU power..


Does anyone else have these problems?
This is what the server log says:

Spoiler

Code:

[20:19:34] Player aedanj left the game (players left 1)
[20:19:39] Player (DoC)kingofthebrithins left the game (players left 0)
[20:19:53] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:19:53] sAPIClientStatus.size() is 5
[20:20:03] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:20:03] sAPIClientStatus.size() is 5
[20:22:29] getTokenValidation error: Curl Error: Timeout was reached
[20:22:44] getTokenValidation error: Curl Error: Timeout was reached
[20:24:07] getTokenValidation error: Token not found
[20:24:07] getTokenValidation error: Token not found
[20:24:09] getTokenValidation error: Token not found
[20:24:09] WARNING: API call failed: cURL Error in putStatus(): Couldn't resolve host name
[20:24:09] WARNING: API call failed: cURL Error in putStatus(): Couldn't resolve host name
[20:24:09] WARNING: API call failed: cURL Error in putMinimapPNG(): Couldn't resolve host name
[20:24:09] WARNING: API call failed: cURL Error in putStatus(): SSL connect error
[20:24:39] Restarting map because of inactivity
[20:24:39] *Restarting Map*
[20:24:39] Generating map from seed 545286; size 40000 bytes
[20:24:41] *Match Ended*
[20:25:02] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached
[20:25:38] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached
[20:25:56] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:25:56] sAPIClientStatus.size() is 5
[20:26:05] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:26:05] sAPIClientStatus.size() is 5
[20:26:15] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:26:15] sAPIClientStatus.size() is 5
[20:26:24] WARNING: API call failed: cURL Error in putStatus(): Couldn't resolve host name
[20:26:35] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:26:35] sAPIClientStatus.size() is 5
[20:26:45] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:26:45] sAPIClientStatus.size() is 5
[20:26:48] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached
[20:27:30] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached
[20:27:45] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached
[20:28:14] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:28:14] sAPIClientStatus.size() is 5
[20:28:24] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:28:24] sAPIClientStatus.size() is 5
[20:28:25] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached
[20:28:44] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:28:44] sAPIClientStatus.size() is 5
[20:28:54] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:28:54] sAPIClientStatus.size() is 5
[20:29:04] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:29:04] sAPIClientStatus.size() is 5
[20:29:04] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached
[20:29:12] WARNING: API call failed: cURL Error in putStatus(): Failed binding local connection end
[20:29:33] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:29:33] sAPIClientStatus.size() is 5
[20:29:43] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:29:43] sAPIClientStatus.size() is 5
[20:29:50] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached
[20:31:08] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached
[20:31:52] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:31:52] sAPIClientStatus.size() is 5
[20:31:53] WARNING: API call failed: cURL Error in putStatus(): Couldn't resolve host name
[20:32:12] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:32:12] sAPIClientStatus.size() is 5
[20:32:22] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:32:22] sAPIClientStatus.size() is 5
[20:32:28] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached
[20:32:41] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:32:41] sAPIClientStatus.size() is 5
[20:32:51] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:32:51] sAPIClientStatus.size() is 5
[20:33:01] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:33:01] sAPIClientStatus.size() is 5
[20:33:11] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:33:11] sAPIClientStatus.size() is 5
[20:33:21] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:33:21] sAPIClientStatus.size() is 5
[20:33:31] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:33:31] sAPIClientStatus.size() is 5
[20:33:36] WARNING: API call failed: cURL Error in putStatus(): SSL connect error
[20:33:51] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:33:51] sAPIClientStatus.size() is 5
[20:33:56] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached
[20:34:11] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:34:11] sAPIClientStatus.size() is 5
[20:34:20] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:34:20] sAPIClientStatus.size() is 5
[20:34:30] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:34:30] sAPIClientStatus.size() is 5
[20:34:40] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:34:40] sAPIClientStatus.size() is 5
[20:34:50] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:34:50] sAPIClientStatus.size() is 5
[20:35:00] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:35:00] sAPIClientStatus.size() is 5
[20:35:10] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:35:10] sAPIClientStatus.size() is 5
[20:35:20] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:35:20] sAPIClientStatus.size() is 5
[20:35:30] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:35:30] sAPIClientStatus.size() is 5
[20:35:40] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:35:40] sAPIClientStatus.size() is 5
[20:35:50] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:35:50] sAPIClientStatus.size() is 5
*cut*

other server:

Spoiler

Code:

[20:00:51] Server: Disconnecting player ( User: not found ) ( IP: 184.46.x.x ) reason: Not Premium
[20:11:34] RCON command from 127.0.0.1:48556: /msg This server is using KAG GlobalChat. Add an at-sign '@' before your message to send it across all servers that use KGC.
[20:11:34] /msg This server is using KAG GlobalChat. Add an at-sign '@' before your message to send it across all servers that use KGC.
[20:11:34]
[20:20:22] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached
[20:20:52] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached
[20:21:16] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:21:16] sAPIClientStatus.size() is 5
[20:21:26] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:21:26] sAPIClientStatus.size() is 5
[20:21:36] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:21:36] sAPIClientStatus.size() is 5
[20:21:46] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:21:46] sAPIClientStatus.size() is 5
[20:21:55] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:21:55] sAPIClientStatus.size() is 5
[20:22:05] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:22:05] sAPIClientStatus.size() is 5
[20:22:15] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:22:15] sAPIClientStatus.size() is 5
[20:22:25] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:22:25] sAPIClientStatus.size() is 5
[20:22:35] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:22:35] sAPIClientStatus.size() is 5
[20:22:45] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:22:45] sAPIClientStatus.size() is 5
[20:22:55] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:22:55] sAPIClientStatus.size() is 5
[20:23:05] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:23:05] sAPIClientStatus.size() is 5
[20:23:15] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:23:15] sAPIClientStatus.size() is 5
[20:23:25] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:23:25] sAPIClientStatus.size() is 5
[20:23:34] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call

yet another server:

Spoiler

Code:

[20:11:38] * Star_Black95 connected (admin: 0 guard 0 gold 0)
[20:11:39] Team initially chosen: 0, team disabled: false
[20:11:45] Player PRUNO Star_Black95 left the game (players left 1)
[20:12:01] RCON command from 127.0.0.1:56296: /msg This server is using KAG GlobalChat. Add an at-sign '@' before your message to send it across all servers that use KGC.
[20:12:01] /msg This server is using KAG GlobalChat. Add an at-sign '@' before your message to send it across all servers that use KGC.
[20:12:01]
[20:13:03] * jordan066 connected (admin: 0 guard 0 gold 0)
[20:13:08] Team initially chosen: 0, team disabled: false
[20:18:27] * aaa2 connected (admin: 0 guard 0 gold 0)
[20:18:29] Team initially chosen: 0, team disabled: false
[20:19:30] Player Snake jordan066 left the game (players left 2)
[20:19:31] Player Thorondil left the game (players left 1)
[20:19:45] Player aaa2 left the game (players left 0)
[20:19:54] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached
[20:20:38] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:20:38] sAPIClientStatus.size() is 5
[20:20:38] WARNING: API call failed: cURL Error in putStatus(): Couldn't resolve host name
[20:20:57] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:20:57] sAPIClientStatus.size() is 5
[20:22:58] getTokenValidation error: Curl Error: Couldn't resolve host name
[20:23:29] getTokenValidation error: Curl Error: Timeout was reached
[20:24:08] getTokenValidation error: Token not found
[20:24:09] WARNING: API call failed: cURL Error in putStatus(): Couldn't resolve host name
[20:24:09] WARNING: API call failed: cURL Error in putStatus(): Couldn't resolve host name
[20:24:09] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached
[20:24:09] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached
[20:25:12] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached
[20:25:28] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached
[20:26:06] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:26:06] sAPIClientStatus.size() is 5
[20:26:16] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:26:16] sAPIClientStatus.size() is 5
[20:26:26] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:26:26] sAPIClientStatus.size() is 5

yet another server:

Spoiler

Code:

[10:28:26] Server: Disconnecting player ( User: not found ) ( IP: 78.111.x.x ) reason: Wrong Password
[20:20:05] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached
[20:20:49] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:20:49] sAPIClientStatus.size() is 5
[20:20:59] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:20:59] sAPIClientStatus.size() is 5
[20:21:08] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:21:08] sAPIClientStatus.size() is 5
[20:21:18] WARNING: API call failed: cURL Error in putStatus(): SSL connect error
[20:21:28] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:21:28] sAPIClientStatus.size() is 5
[20:21:38] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:21:38] sAPIClientStatus.size() is 5
[20:21:48] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:21:48] sAPIClientStatus.size() is 5
[20:21:58] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:21:58] sAPIClientStatus.size() is 5
[20:22:02] WARNING: API call failed: cURL Error in putStatus(): Couldn't resolve host name
[20:22:18] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:22:18] sAPIClientStatus.size() is 5
[20:22:28] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:22:28] sAPIClientStatus.size() is 5
[20:22:29] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached
[20:22:47] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:22:47] sAPIClientStatus.size() is 5
[20:22:57] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[20:22:57] sAPIClientStatus.size() is 5
[20:23:07] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call

Only one of my servers didn't crash.
PS, my main server doesn't have these problems at all..

 

Since they did crash eventually, were you running them under GDB? I think that devs could use logs of it to try to fix it.

 

The bad thing is that they don't close themselves, they just stay in some state where no one is able to join and take 100% possible CPU power. So I have to close them manually.
I guess I could run them in GDB and post the log of me closing them manually..

edit: my servers seem to have this error without the extreme usage as well once in a while.

Code:

[21:49:07] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached
[21:49:51] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[21:49:51] sAPIClientStatus.size() is 5
[21:49:51] WARNING: API call failed: cURL Error in putStatus(): Couldn't resolve host name
[21:50:10] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[21:50:10] sAPIClientStatus.size() is 5
[21:50:20] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[21:50:20] sAPIClientStatus.size() is 5
[21:50:30] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[21:50:30] sAPIClientStatus.size() is 5
[21:50:40] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[21:50:40] sAPIClientStatus.size() is 5
[21:50:42] WARNING: API call failed: cURL Error in putStatus(): Couldn't resolve host name
[21:51:00] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[21:51:00] sAPIClientStatus.size() is 5
[21:51:10] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[21:51:10] sAPIClientStatus.size() is 5
[21:51:12] WARNING: API call failed: cURL Error in putStatus(): Timeout was reached

</br>--- merged: Feb 6, 2013 4:25 PM ---</br>

Spoiler: GDB

[17:23:52] sAPIClientStatus.size() is 5
^C #had to kill here
Program received signal SIGINT, Interrupt.
btfull #kill didn't work so I had to kill -9
^CQuit
(gdb) bt full
#0 0xb7fe2424 in __kernel_vsyscall ()
No symbol table info available.
#1 0xb7e0f361 in select () from /lib/i686/cmov/libc.so.6
No symbol table info available.
#2 0xb6dfe618 in CKernel::Execute (this=0x8109b08) at ../Engine/kernel.cpp:134
t = {tv_sec = 0, tv_usec = 604}
mainControl = 135305992
#3 0xb6e932dd in CGame::Run (this=0x80f8ea0, argc=5, argv=0xbffff204)
at ../Game/GameDLL.cpp:457
No locals.
#4 0xb6e58225 in GameDLLInit (ie=..., argc=5, argv=0xbffff204)
at ../Game/GameDLL.cpp:93
profilehandler = {<IProfilerOutputHandler> = {
_vptr.IProfilerOutputHandler = 0xb75eef48}, <No data fields>}
filename = {array = 0x80f6030 "Logs/console-13-01-20-21-30-08.txt",
allocated = 35, used = 35, allocator = {
_vptr.irrAllocator = 0xb75eeb18}}
ex = <value optimized out>
#5 0x0804bdc8 in CApplication::LoadGameDLL (this=0x80c6088,
filename=0x80d1ccc "Base/libJuxta_dedi.so", argc=5, argv=0xbffff204)
at ../App/main.cpp:337
initializer = 0xb6e57ba0
#6 0x0804e038 in CApplication::Run (this=0x80c6088, argc=5, argv=0xbffff204)
---Type <return> to continue, or q <return> to quit---
at ../App/main.cpp:224
dllFile = {static npos = 4294967295,
_M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>},
_M_p = 0x80d1ccc "Base/libJuxta_dedi.so"}}
date = "Jan 8 2013"
buildnum = 135029504
path = <value optimized out>
#7 0x0804c0ce in main (argc=5, argv=0xbffff204) at ../App/main.cpp:597
restart = {
array = 0xb7e8d304 "\210\273\r\b\340\336\350\267t\320\350\267h\320\350\267D\320\350\267\003", allocated = 3085488116, used = 134891584, allocator = {
_vptr.irrAllocator = 0xbffff158}}
dir = {array = 0xb7ff1380 "U\211\345WV1\366S\350\376\210",
allocated = 134963188, used = 3221221720, allocator = {
_vptr.irrAllocator = 0x80a4859}}
(gdb)

I had to kill the entire GDB at the other server I was running with gdb as it was using 100% of my cpu power.
</br>--- merged: Feb 11, 2013 9:04 PM ---</br>
I've got another debug log:

Spoiler

[22:07:21] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[22:07:21] sAPIClientStatus.size() is 5
^C
Program received signal SIGINT, Interrupt.
0xb7fe2424 in __kernel_vsyscall ()
(gdb) bt full
#0 0xb7fe2424 in __kernel_vsyscall ()
No symbol table info available.
#1 0xb7e0f361 in select () from /lib/i686/cmov/libc.so.6
No symbol table info available.
#2 0xb6dfe618 in CKernel::Execute (this=0x8109b08) at ../Engine/kernel.cpp:134
t = {tv_sec = 0, tv_usec = 279}
mainControl = 135305992
#3 0xb6e932dd in CGame::Run (this=0x80f8ea0, argc=5, argv=0xbffff204)
at ../Game/GameDLL.cpp:457
No locals.
#4 0xb6e58225 in GameDLLInit (ie=..., argc=5, argv=0xbffff204)
at ../Game/GameDLL.cpp:93
profilehandler = {<IProfilerOutputHandler> = {
_vptr.IProfilerOutputHandler = 0xb75eef48}, <No data fields>}
filename = {array = 0x80f6030 "Logs/console-13-02-06-17-28-50.txt",
allocated = 35, used = 35, allocator = {
_vptr.irrAllocator = 0xb75eeb18}}
ex = <value optimized out>
#5 0x0804bdc8 in CApplication::LoadGameDLL (this=0x80c6088,
filename=0x80d1ccc "Base/libJuxta_dedi.so", argc=5, argv=0xbffff204)
at ../App/main.cpp:337
initializer = 0xb6e57ba0
#6 0x0804e038 in CApplication::Run (this=0x80c6088, argc=5, argv=0xbffff204)
---Type <return> to continue, or q <return> to quit---
at ../App/main.cpp:224
dllFile = {static npos = 4294967295,
_M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>},
_M_p = 0x80d1ccc "Base/libJuxta_dedi.so"}}
date = "Jan 8 2013"
buildnum = 135029504
path = <value optimized out>
#7 0x0804c0ce in main (argc=5, argv=0xbffff204) at ../App/main.cpp:597
restart = {
array = 0xb7e8d304 "\210\273\r\b\340\336\350\267\250\320\350\267\234\320\350\267D\320\350\267\003", allocated = 3085488116, used = 134891584,
allocator = {_vptr.irrAllocator = 0xbffff158}}
dir = {array = 0xb7ff1380 "U\211\345WV1\366S\350\376\210",
allocated = 134963188, used = 3221221720, allocator = {
_vptr.irrAllocator = 0x80a4859}}
(gdb)

 

yet another debug log:

Spoiler

Code:

^C
Program received signal SIGINT, Interrupt.
^C
Quit
(gdb)
(gdb) bt full
#0  0xb7fe2424 in __kernel_vsyscall ()
No symbol table info available.
#1  0xb7e0f361 in select () from /lib/i686/cmov/libc.so.6
No symbol table info available.
#2  0xb6dfe618 in CKernel::Execute (this=0x8109b08) at ../Engine/kernel.cpp:134
        t = {tv_sec = 0, tv_usec = 68}
        mainControl = 135305992
#3  0xb6e932dd in CGame::Run (this=0x80f8ea0, argc=5, argv=0xbffff204) at ../Gam
e/GameDLL.cpp:457
No locals.
#4  0xb6e58225 in GameDLLInit (ie=..., argc=5, argv=0xbffff204) at ../Game/GameD
LL.cpp:93
        profilehandler = {<IProfilerOutputHandler> = {_vptr.IProfilerOutputHandl
er = 0xb75eef48}, <No data fields>}
        filename = {array = 0x80f6030 "Logs/console-13-02-11-22-16-54.txt", allo
cated = 35, used = 35, allocator = {
            _vptr.irrAllocator = 0xb75eeb18}}
        ex = <value optimized out>
#5  0x0804bdc8 in CApplication::LoadGameDLL (this=0x80c6088, filename=0x80d1ccc
"Base/libJuxta_dedi.so", argc=5,
    argv=0xbffff204) at ../App/main.cpp:337
        initializer = 0xb6e57ba0
#6  0x0804e038 in CApplication::Run (this=0x80c6088, argc=5, argv=0xbffff204) at
../App/main.cpp:224
        dllFile = {static npos = 4294967295,
          _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<cha
r>> = {<No data fields>}, <No data fields>},
            _M_p = 0x80d1ccc "Base/libJuxta_dedi.so"}}
        date = "Jan  8 2013"
        buildnum = 135029504
        path = <value optimized out>
#7  0x0804c0ce in main (argc=5, argv=0xbffff204) at ../App/main.cpp:597
        restart = {array = 0xb7e8d304 "\210\273\r\b\340\336\350\267\274\320\350\
267\260\320\350\267D\320\350\267\003",
          allocated = 3085488116, used = 134891584, allocator = {_vptr.irrAllocator = 0xbffff158}}
        dir = {array = 0xb7ff1380 "U\211\345WV1\366S\350\376\210", allocated = 134963188, used = 3221221720, allocator = {
            _vptr.irrAllocator = 0x80a4859}}
(gdb)

They all seem to be the same.. It just keeps happening.

 

The kag2d.nl hosting server only had this problem once in 2 months now. But my other server has this like once a week.

 

The 100% load problem has been a problem for quite some time, but it was just affecting one server at the time. https://forum.kag2d.com/threads/server-100-cpu-use-people-cant-connect.7377/
All servers are going to 100% at the same time now, it was not like that before. So I think this is a different problem. I have no idea when it was implemented. The first time I can find a big load on my servers right now is 13thof january, but I am pretty sure it happened before. I can see a huge spam of the api call failed in november, but it didn't get the server stuck by then.

 

I keep getting backtraces like this as well when I close a normal (non dbg) server that has this problem

Spoiler

[04:13:33] sAPIClientStatus.size() is 5
^C[04:13:36] Closing console device: Signal 2 received
Saving precached files.
Error: connection lost
*** glibc detected *** ./KAGdedi: double free or corruption (!prev): 0x09be40b8 ***
======= Backtrace: =========
/lib/i686/cmov/libc.so.6(+0x6af71)[0xb756bf71]
/lib/i686/cmov/libc.so.6(+0x6c7c8)[0xb756d7c8]
/lib/i686/cmov/libc.so.6(cfree+0x6d)[0xb75708ad]
Base/libJuxta_dedi.so(+0x7d6237)[0xb6b9a237]
Base/libJuxta_dedi.so(+0x7d6341)[0xb6b9a341]
Base/libJuxta_dedi.so(+0x7d836d)[0xb6b9c36d]
Base/libJuxta_dedi.so(+0x7e17ae)[0xb6ba57ae]
Base/libJuxta_dedi.so(_ZN9APIClient7putJsonESsRN4Json5ValueES2_SsSsSsi+0x3af)[0xb685963f]
Base/libJuxta_dedi.so(_ZN9APIClient9putStatusERSsR9APIServer+0xb99)[0xb685e609]
Base/libJuxta_dedi.so(_Z21putStatus_thread_funcPv+0x2fc)[0xb687dedc]
Base/libJuxta_dedi.so(+0x7f806f)[0xb6bbc06f]
Base/libJuxta_dedi.so(+0x7f8411)[0xb6bbc411]
/lib/i686/cmov/libpthread.so.0(+0x5955)[0xb74ed955]
/lib/i686/cmov/libc.so.6(clone+0x5e)[0xb75cd1de]
======= Memory map: ========
08048000-080b5000 r-xp 00000000 08:01 27933022 /home/pq/creative/KAGdedi
080b5000-080b6000 r--p 0006c000 08:01 27933022 /home/pq/creative/KAGdedi
080b6000-080b7000 rw-p 0006d000 08:01 27933022 /home/pq/creative/KAGdedi
080b7000-080c6000 rw-p 00000000 00:00 0
095ba000-0a224000 rw-p 00000000 00:00 0 [heap]
aea00000-aea21000 rw-p 00000000 00:00 0
aea21000-aeb00000 ---p 00000000 00:00 0
aebff000-aec00000 ---p 00000000 00:00 0
aec00000-af400000 rw-p 00000000 00:00 0
af400000-af500000 rw-p 00000000 00:00 0
af600000-af6ff000 rw-p 00000000 00:00 0
af6ff000-af700000 ---p 00000000 00:00 0
af800000-af8fd000 rw-p 00000000 00:00 0
af8fd000-af900000 ---p 00000000 00:00 0
afa00000-afae7000 rw-p 00000000 00:00 0
afae7000-afb00000 ---p 00000000 00:00 0
afb00000-afc00000 rw-p 00000000 00:00 0
afc00000-afcff000 rw-p 00000000 00:00 0
afcff000-afd00000 ---p 00000000 00:00 0
afdff000-afe00000 ---p 00000000 00:00 0
afe00000-b0600000 rw-p 00000000 00:00 0
b0600000-b06ff000 rw-p 00000000 00:00 0
b06ff000-b0700000 ---p 00000000 00:00 0
b0800000-b0a00000 rw-p 00000000 00:00 0
b0a00000-b0bfd000 rw-p 00000000 00:00 0
b0bfd000-b0c00000 ---p 00000000 00:00 0
b0c00000-b0e00000 rw-p 00000000 00:00 0
b0e00000-b0efd000 rw-p 00000000 00:00 0
b0efd000-b0f00000 ---p 00000000 00:00 0
b0ffe000-b0fff000 ---p 00000000 00:00 0
b0fff000-b17ff000 rw-p 00000000 00:00 0
b1800000-b1900000 rw-p 00000000 00:00 0
b1a00000-b1c00000 rw-p 00000000 00:00 0
b1c00000-b1d00000 rw-p 00000000 00:00 0
b1e00000-b1f00000 rw-p 00000000 00:00 0
b2000000-b2200000 rw-p 00000000 00:00 0
b2200000-b2300000 rw-p 00000000 00:00 0
b2400000-b2500000 rw-p 00000000 00:00 0
b2600000-b2700000 rw-p 00000000 00:00 0
b2800000-b2900000 rw-p 00000000 00:00 0
b2a00000-b2c00000 rw-p 00000000 00:00 0
b2c00000-b2cf8000 rw-p 00000000 00:00 0
b2cf8000-b2d00000 ---p 00000000 00:00 0
b2d00000-b2e00000 rw-p 00000000 00:00 0
b2e00000-b2ef7000 rw-p 00000000 00:00 0
b2ef7000-b2f00000 ---p 00000000 00:00 0
b3000000-b30fa000 rw-p 00000000 00:00 0
b30fa000-b3100000 ---p 00000000 00:00 0
b3100000-b3200000 rw-p 00000000 00:00 0
b3200000-b3400000 rw-p 00000000 00:00 0
b3400000-b35ff000 rw-p 00000000 00:00 0
b35ff000-b3600000 ---p 00000000 00:00 0
b3600000-b3700000 rw-p 00000000 00:00 0
b3800000-b39d6000 rw-p 00000000 00:00 0
b39d6000-b3a00000 ---p 00000000 00:00 0
b3a00000-b3c00000 rw-p 00000000 00:00 0
b3c00000-b3e00000 rw-p 00000000 00:00 0
b3e00000-b4000000 rw-p 00000000 00:00 0
b4000000-b4100000 rw-p 00000000 00:00 0
b4200000-b4300000 rw-p 00000000 00:00 0
b4400000-b4500000 rw-p 00000000 00:00 0
b45fe000-b45ff000 ---p 00000000 00:00 0
b45ff000-b4dff000 rw-p 00000000 00:00 0
b4dff000-b4e00000 ---p 00000000 00:00 0
b4e00000-b5600000 rw-p 00000000 00:00 0
b5600000-b56e8000 rw-p 00000000 00:00 0
b56e8000-b5700000 ---p 00000000 00:00 0
b5700000-b57f7000 rw-p 00000000 00:00 0
b57f7000-b5800000 ---p 00000000 00:00 0
b5800000-b5900000 rw-p 00000000 00:00 0
b5984000-b5985000 ---p 00000000 00:00 0
b5985000-b63c4000 rw-p 00000000 00:00 0
b63c4000-b6d80000 r-xp 00000000 08:01 27927894 /home/pq/creative/Base/libJuxta_dedi.so
b6d80000-b6db1000 r--p 009bc000 08:01 27927894 /home/pq/creative/Base/libJuxta_dedi.so
b6db1000-b6db8000 rw-p 009ed000 08:01 27927894 /home/pq/creative/Base/libJuxta_dedi.so
b6db8000-b74be000 rw-p 00000000 00:00 0
b74be000-b74ce000 r-xp 00000000 08:01 2228264 /lib/i686/cmov/libresolv-2.11.3.so
b74ce000-b74cf000 r--p 00010000 08:01 2228264 /lib/i686/cmov/libresolv-2.11.3.so
b74cf000-b74d0000 rw-p 00011000 08:01 2228264 /lib/i686/cmov/libresolv-2.11.3.so
b74d0000-b74d2000 rw-p 00000000 00:00 0
b74d2000-b74dc000 r-xp 00000000 08:01 2228266 /lib/i686/cmov/libnss_files-2.11.3.so
b74dc000-b74dd000 r--p 00009000 08:01 2228266 /lib/i686/cmov/libnss_files-2.11.3.so
b74dd000-b74de000 rw-p 0000a000 08:01 2228266 /lib/i686/cmov/libnss_files-2.11.3.so
b74e6000-b74e8000 rw-p 00000000 00:00 0
b74e8000-b74fd000 r-xp 00000000 08:01 2228274 /lib/i686/cmov/libpthread-2.11.3.so
b74fd000-b74fe000 r--p 00014000 08:01 2228274 /lib/i686/cmov/libpthread-2.11.3.so
b74fe000-b74ff000 rw-p 00015000 08:01 2228274 /lib/i686/cmov/libpthread-2.11.3.so
b74ff000-b7501000 rw-p 00000000 00:00 0
b7501000-b7641000 r-xp 00000000 08:01 2228304 /lib/i686/cmov/libc-2.11.3.so
b7641000-b7642000 ---p 00140000 08:01 2228304 /lib/i686/cmov/libc-2.11.3.so
b7642000-b7644000 r--p 00140000 08:01 2228304 /lib/i686/cmov/libc-2.11.3.so
b7644000-b7645000 rw-p 00142000 08:01 2228304 /lib/i686/cmov/libc-2.11.3.so
b7645000-b7648000 rw-p 00000000 00:00 0
b7648000-b7665000 r-xp 00000000 08:01 2097193 /lib/libgcc_s.so.1
b7665000-b7666000 rw-p 0001c000 08:01 2097193 /lib/libgcc_s.so.1
b7666000-b768a000 r-xp 00000000 08:01 2228256 /lib/i686/cmov/libm-2.11.3.so
b768a000-b768b000 r--p 00023000 08:01 2228256 /lib/i686/cmov/libm-2.11.3.so
b768b000-b768c000 rw-p 00024000 08:01 2228256 /lib/i686/cmov/libm-2.11.3.so
b768c000-b7775000 r-xp 00000000 08:01 7078863 /usr/lib/libstdc++.so.6.0.13
b7775000-b7779000 r--p 000e9000 08:01 7078863 /usr/lib/libstdc++.so.6.0.13
b7779000-b777a000 rw-p 000ed000 08:01 7078863 /usr/lib/libstdc++.so.6.0.13
b777a000-b7782000 rw-p 00000000 00:00 0
b7782000-b7784000 r-xp 00000000 08:01 2228254 /lib/i686/cmov/libdl-2.11.3.so
b7784000-b7785000 r--p 00001000 08:01 2228254 /lib/i686/cmov/libdl-2.11.3.so
b7785000-b7786000 rw-p 00002000 08:01 2228254 /lib/i686/cmov/libdl-2.11.3.so
b7786000-b778d000 r-xp 00000000 08:01 2228303 /lib/i686/cmov/librt-2.11.3.so
b778d000-b778e000 r--p 00006000 08:01 2228303 /lib/i686/cmov/librt-2.11.3.so
b778e000-b778f000 rw-p 00007000 08:01 2228303 /lib/i686/cmov/librt-2.11.3.so
b7790000-b7794000 r-xp 00000000 08:01 2228305 /lib/i686/cmov/libnss_dns-2.11.3.so
b7794000-b7795000 r--p 00004000 08:01 2228305 /lib/i686/cmov/libnss_dns-2.11.3.so
b7795000-b7796000 rw-p 00005000 08:01 2228305 /lib/i686/cmov/libnss_dns-2.11.3.so
b7796000-b7799000 rw-p 00000000 00:00 0
b7799000-b779a000 r-xp 00000000 00:00 0 [vdso]
b779a000-b77b5000 r-xp 00000000 08:01 2097635 /lib/ld-2.11.3.so
b77b5000-b77b6000 r--p 0001b000 08:01 2097635 /lib/ld-2.11.3.so
b77b6000-b77b7000 rw-p 0001c000 08:01 2097635 /lib/ld-2.11.3.so
bfde4000-bfdf9000 rw-p 00000000 00:00 0 [stack]
[04:13:36] Closing console device: Signal 6 received
Aborted

</br>--- merged: Mar 1, 2013 5:18 PM ---</br>
another gdb log

Spoiler

[18:24:19] WARNING: A call to update the server list API is showing as still in progress when we were ready to make the next call
[18:24:19] sAPIClientStatus.size() is 5
^C[18:24:25] Closing console device: Signal 2 received

Program received signal SIGINT, Interrupt.
0xb7fe2424 in __kernel_vsyscall ()
(gdb) Saving precached files.
(gdb) bt full
#0 0xb7fe2424 in __kernel_vsyscall ()
No symbol table info available.
#1 0xb7de0d93 in waitpid () from /lib/i686/cmov/libc.so.6
No symbol table info available.
#2 0xb7d82e93 in ?? () from /lib/i686/cmov/libc.so.6
No symbol table info available.
#3 0x0804c5d9 in main (argc=5, argv=0xbffff204) at ../App/main.cpp:622
cmd = {
array = 0x80c6088 "/home/pq/gatherservers/two/KAGdedi autostart Scripts/dedicated_autostart.gm autoconfig Scripts/dedicated_autoconfig.gm ",
allocated = 120, used = 120, allocator = {
_vptr.irrAllocator = 0x80a4ca8}}
restart = {
array = 0x80ef778 "Major file update - please restart the game\n",
allocated = 45, used = 45, allocator = {
_vptr.irrAllocator = 0x80a4ca8}}
dir = {array = 0x80d1178 "/home/pq/gatherservers/two/",
allocated = 28, used = 28, allocator = {
_vptr.irrAllocator = 0x80a4ca8}}
(gdb)

this one is different from the other 2. (yes I updated kag while running gdb......... but the server worked fine for quite some time till the errors started)
</br>--- merged: Mar 17, 2013 4:22 PM ---</br>
It happened 3 times today. Here are some packet logs/kbit logs. Blue and purple are incoming traffic. It stops as soon as I shut the damn servers down.