Page MenuHomeSchine - Phabricator
Create Task
Maniphest T2232

Starmade | Odd memory spike, and then server crash...
Open, HighPublic
Actions

Description

Every day, for the past three days, it's the same thing.

Server is running along fine, and then this happens a little over six hours after a scheduled restart:


jstack:

jstack.txt313 KBDownload

Each time this occurs, only one connected player's IP has been consistent:
31.49.176.4

StartMade Names (also matches IGN):
warguy2004
Steviep_04

On the network-side, I see an odd change in traffic patterns at the moment this memory spike occurs:

10:48:55.869880 IP 31.49.176.4.53703 > 199.127.226.177.4242: Flags [.], ack 1225694, win 16652, options [nop,nop,sack 2 {1244518:1245966}{1241622:1243070}], length 0
10:48:55.870080 IP 31.49.176.4.53703 > 199.127.226.177.4242: Flags [.], ack 1225694, win 16652, options [nop,nop,sack 2 {1244518:1247414}{1241622:1243070}], length 0
10:48:55.871443 IP 31.49.176.4.53703 > 199.127.226.177.4242: Flags [.], ack 1225694, win 16652, options [nop,nop,sack 2 {1244518:1248862}{1241622:1243070}], length 0
10:48:55.871454 IP 31.49.176.4.53703 > 199.127.226.177.4242: Flags [.], ack 1225694, win 16652, options [nop,nop,sack 2 {1244518:1250310}{1241622:1243070}], length 0
10:48:55.871683 IP 31.49.176.4.53703 > 199.127.226.177.4242: Flags [.], ack 1225694, win 16652, options [nop,nop,sack 2 {1244518:1251758}{1241622:1243070}], length 0
10:48:55.871964 IP 31.49.176.4.53703 > 199.127.226.177.4242: Flags [.], ack 1225694, win 16652, options [nop,nop,sack 2 {1244518:1254654}{1241622:1243070}], length 0
10:48:55.871975 IP 31.49.176.4.53703 > 199.127.226.177.4242: Flags [.], ack 1225694, win 16652, options [nop,nop,sack 2 {1244518:1253206}{1241622:1243070}], length 0
10:48:55.872409 IP 31.49.176.4.53703 > 199.127.226.177.4242: Flags [.], ack 1225694, win 16652, options [nop,nop,sack 2 {1244518:1256102}{1241622:1243070}], length 0
10:48:55.872947 IP 31.49.176.4.53703 > 199.127.226.177.4242: Flags [.], ack 1225694, win 16652, options [nop,nop,sack 2 {1244518:1257550}{1241622:1243070}], length 0
10:48:55.873161 IP 31.49.176.4.53703 > 199.127.226.177.4242: Flags [.], ack 1225694, win 16652, options [nop,nop,sack 2 {1244518:1258998}{1241622:1243070}], length 0
10:48:55.873322 IP 31.49.176.4.53703 > 199.127.226.177.4242: Flags [.], ack 1225694, win 16652, options [nop,nop,sack 2 {1244518:1260446}{1241622:1243070}], length 0
10:48:55.873832 IP 31.49.176.4.53703 > 199.127.226.177.4242: Flags [.], ack 1225694, win 16652, options [nop,nop,sack 2 {1244518:1261894}{1241622:1243070}], length 0
10:48:55.874002 IP 31.49.176.4.53703 > 199.127.226.177.4242: Flags [.], ack 1225694, win 16652, options [nop,nop,sack 2 {1244518:1263342}{1241622:1243070}], length 0
10:48:55.874551 IP 31.49.176.4.53703 > 199.127.226.177.4242: Flags [.], ack 1225694, win 16652, options [nop,nop,sack 2 {1244518:1264790}{1241622:1243070}], length 0
10:48:55.874708 IP 31.49.176.4.53703 > 199.127.226.177.4242: Flags [.], ack 1225694, win 16652, options [nop,nop,sack 2 {1244518:1266238}{1241622:1243070}], length 0
10:48:55.877754 IP 31.49.176.4.53703 > 199.127.226.177.4242: Flags [.], ack 1225694, win 16652, options [nop,nop,sack 2 {1244518:1267686}{1241622:1243070}], length 0
10:48:55.877984 IP 31.49.176.4.53703 > 199.127.226.177.4242: Flags [.], ack 1225694, win 16652, options [nop,nop,sack 3 {1269134:1270582}{1244518:1267686}{1241622:1243070}], length 0
10:48:55.877994 IP 31.49.176.4.53703 > 199.127.226.177.4242: Flags [.], ack 1225694, win 16652, options [nop,nop,sack 3 {1269134:1272030}{1244518:1267686}{1241622:1243070}], length 0
10:48:55.878235 IP 31.49.176.4.53703 > 199.127.226.177.4242: Flags [.], ack 1225694, win 16652, options [nop,nop,sack 4 {1279270:1282166}{1269134:1272030}{1244518:1267686}{1241622:1243070}], length 0
10:48:55.878246 IP 31.49.176.4.53703 > 199.127.226.177.4242: Flags [.], ack 1225694, win 16652, options [nop,nop,sack 4 {1279270:1280718}{1269134:1272030}{1244518:1267686}{1241622:1243070}], length 0

I'm not finding anything obvious in logs, but here they are anyway:

serverlog.0.log89 KBDownload

logstarmade.0 (2).log3 MBDownload

serverlog.1.log2 MBDownload

logstarmade.1.log4 MBDownload

Details

Task Type
Bug
Testing Results
Affected Gamemode(s)
Multiplayer
Reproducible
uncertain
Category
none/unspecified
Hardware/Software/System
OS-Specific
No
Hardware-Specific
No
Video Card Vendor
uncertain
Serverconfig (server.cfg)
 <replace this block with the file content, 
but do not remove the first and last line. 
Before using the paste function, mark the area from the 
opening angle bracket to the closing angle bracket>
Clientconfig (settings.cfg)
 <replace this block with the file content, 
but do not remove the first and last line. 
Before using the paste function, mark the area from the 
opening angle bracket to the closing angle bracket>

Event Timeline

erthparadine added a comment.Feb 16 2017, 7:34 PM

Better illustration of memory/load:

erthparadine added a comment.EditedFeb 16 2017, 8:23 PM

Server logs are UTC, local (VisualVM) info is US/Pacific.

lancake shifted this object from the S1 Public space to the S3 Starmade space.Mar 10 2017, 2:20 PM
lancake changed the visibility from "Custom Policy" to "Public (No Login Required)".
lancake changed the edit policy from "Task Author" to "Starmade (Project)".
lancake changed Affected Gamemode(s) from none/unspecified to Multiplayer.
lancake claimed this task.
lancake triaged this task as High priority.
Restricted Application added a subscriber: AndyP. · View Herald TranscriptMar 10 2017, 2:20 PM
schema added a subscriber: schema.Mar 10 2017, 2:28 PM

We have to find out what exactly happened at the time. The stack trace shows the server to transmit chunks. So maybe someone was uploading some kind of exploitation ship/station...

schema added a comment.EditedMar 10 2017, 2:29 PM

UTC is 8 hours ahead of PT, which means the event happened at 18:49 in the logs