Starmade | Turrets. Random parts of turrets are disappearing off players stations and ships
Resolved, needs quality check / confirm fix, Release BlockerPublic

Description

For some reason whensome players log back in turrets are missing from some ships and from stations.
It can be just a barrel missing as i found on my station.
The missing entity is not detached and im unable to find them anywhere.


-Confirmed-

It seems to be another shipyard related issue and for those not involving a shipyard there appears to be a problem with create docking

shipyard built turret disconnect
https://youtu.be/onQpMi7OpEs

blueprint station w/ docked ship
station with docked ship .zip427 KB

Details

Task Type
Bug
Testing Results
Affected Gamemode(s)
none/unspecified
Reproducible
uncertain
Last tested (version)
0.19509 dev
Category
Control Block System: Rails/Docking
First occurrence (version)
0.19485
Hardware/Software/System
OS-Specific
No
Hardware-Specific
No
Video Card Vendor
uncertain
Serverconfig (server.cfg)
<replace this line with the file content>
Clientconfig (settings.cfg)
<replace this line with the file content>
There are a very large number of changes, so older changes are hidden. Show Older Changes
lancake set Affected Gamemode(s) to none/unspecified.Nov 9 2015, 11:21 AM
lancake changed First occurrence (version) from v 0.19485 to 0.19485.
lancake set Reproducible to uncertain.
lancake set Last tested (version) to 0.19485.
lancake set OS-Specific to No.
lancake set Hardware-Specific to No.
lancake set Video Card Vendor to uncertain.
lancake moved this task from New / Unconfirmed to Feedback on the Issue Navigation board.
lancake triaged this task as "Urgent" priority.
Restricted Application added a subscriber: AndyP. · View Herald TranscriptNov 9 2015, 11:21 AM
Phocian added a comment.EditedNov 9 2015, 12:03 PM

Individual blocks are not missing.
My base turret was missing the barrel entity, the turret base was still there attached to the homebase.
The design of the turret means that the barrel if detached, would still be trapped in the turret base if it was detached by a player by mistake.
I did search the surrounding sectors and could not find it.
All player stations are centred in their sectors including mine using "teleport_selected_to 0 0 0 as we have had issues with sector borders.

The player ship in question had approx 10 turrets missing. This was found after the player logged back in some hours after completing his ship, which hadn't moved as it was docked at his home base before, during and after.

Hm, it sounds like a certain amount of entities just didn't save, didn't save correctly or got reverted (to their non existent state)...It's most likely to do something with save.

Those entities that disappeared, where they already there for a while? As in: did they get unloaded/load at least once? It may be that the name of the entity is too long which causes it to not save correctly: this should be visible in logs with SQLExceptions on the Name column.

Will check the logs as the ship turret issue was over a period of hours.

The base turret barrel disappearing is strange as the turrets were placed a week prior.

I had the same problem on my ship bt in my case the parts were in the sector center.

Looks like this happens for pretty much every docked entity, more information from T899 and T893:

Given: Ship docked to shipyard.
Bunch of turrets docked to ship.

Action: I left the game.
Result: Upon return i can see cannons from my turrets floating 400 meters aside from place where they're supposed to be.

Action#2: I repaired my ship, using shipyard command, so i can get all missing turrets back in their places.
I left game after the end of construction.
Result#2: Different turrets are now missing, but with their bases too, not just part that supposed to shoot.

Action#3: Same as #2.
Result#3: Few turrets lost their cannons right after construction was finished.
Leaving sector changed nothing.
Leaving the game took away few more turrets.
Building of Anti-Missile-Turrets in a shipyard, after deploying there are 3 Stages that happen. 

First stage: Turret works as it should. 
Second stage: Turret just does nothing, even after recalibrating the ai module or un and redock every parts of it. 
Stage 3: Turret does nothing as in stage 2, after turning the turret out of field of view or after quit and rejoin the "turret-head" is invisible, but still physical.

When i enter the turret-base the base is seen and useable, the head is still invisible, after switching into the head wirh arrowkeys i see the head and the rest is as i woud be in a clear sector without the marks for other sectors, there i´m able to edit the head. 
After switching back to the turret-base, the head is invisible again, but still physical, and the rest of the sector has to be loaded again as i would have newly entered the sector. If i try to undock the head the game freezes. 
Sometimes after ignoring the problem the turret-head and base just dissapear. If i try to undock the head the game freezes.
I also got ships builded in shipyards wich just dissapear after some time and/or restarts of the server. 
In every case there are no traces in the logs, by freezing it looks like i exited the server just like klicking exit... Sometimes the head is found about 5000(nearest) to 41000m(farest)away
lancake removed lancake as the assignee of this task.

Ran out of ideas, I'll keep an eye on it though.

Igel added a comment.Nov 28 2015, 9:29 PM

I mentioned another thing. If I build a ship in a shipyard and place too many blocks at once so I need to restart the game (or mostly my computer) sometimes the builder entity is about 10 km away, not physical. I'm not able to enter it. At the shipyard the design is unloaded. If I try to load the design an error occurs, that the design is not able to be loaded because it is not a correct design ( I'm not able to define the error better at the moment) also I'm not able to select the design over navigation and add it as blueprint via command. All in all the design is completely lost in most cases

Yesterday on Shattered Skies I hit an issue with my new ship (built in a shipyard) where half the turrets just kinda disappeared/teleported a long way away from the base... I haven't had any such issues with blueprint-spawned ships.

spunkie claimed this task.Nov 30 2015, 10:27 AM
Igel added a comment.Nov 30 2015, 10:41 AM

new information for my last post. The ship is everytime at the place i where i left it. If it seemed to be dissapeared, it is back after 1-2 restarts, sometimes the server crashes while loading the structure with the shipyard and design on it. After the crash the server starts up normal and everything is fine. Another thing is interesting: The ship is not away. The ship is at the shipyard anchor, i newly activated an energy generator in astronaut mode where the ship should be, in fact the ship was visualized about 10km away from me, even with the navigation mark

-Confirmed-

It seems to be another shipyard related issue and for those not involving a shipyard there appears to be a problem with create docking

shipyard built turret disconnect
https://youtu.be/onQpMi7OpEs

blueprint station w/ docked ship
station with docked ship .zip427 KB

spunkie changed the task status from "Open" to "In Queue (Game)".Dec 3 2015, 5:54 PM
Restricted Application edited projects, added Starmade, Game Development; removed Issue Navigation. · View Herald TranscriptDec 3 2015, 5:54 PM
spunkie edited the task description. (Show Details)Dec 3 2015, 5:55 PM
spunkie changed Last tested (version) from 0.19485 to 0.19509 dev.

I just had a turret (base + barrel) displaced by about half a sector after relogging. (v 0.19519)
The base was made via "Create Docking" (unlike the barrel).

I have a theory that this has to do with the rotation of the turrets when they're created at the shipyard. I need to do more testing though, but so far it seems that if a turret isn't moved in any way, the shipyard doesn't mess it up.

Igel added a comment.Dec 6 2015, 8:36 AM

Why are ships build in the shipyard then disappearing (or just the visualisation is a distance away) that doesn't fit in your theorie of entities will not dissapear until moving/turning them

That might be a separate issue alltogether.

When doing a test flight i've had docked parts not spawn with the ship, getting a ghost image of the ship on return, which materialized inside the planet core on restart.
When building a design, I've had turrets which acted like turrets but couldn't be undocked. Even when the rail & dock were removed, the turret still acted as a turret. Restarting made these turrets appear in the center of the planet core below. Using the jump drive kicked those turrets out as well.

On the other hand, building a design in the shipyard, no matter how complex, saving it as a blueprint and spawning it in always works.

To what is actually causing it, I can only guess, and test. :/

Igel added a comment.Dec 6 2015, 8:50 AM

Additional, I noticed something else by my turrets, the ones who disappear and appear sometime somewhere, not every one that are build in the shipyard, they are even not working with the ai module...may it help you with testing

I've noticed the same. These are the same turrets that can't be detached, even if the entire docking mechanism is removed.

GRHayes added a subscriber: GRHayes.EditedDec 17 2015, 9:53 PM

I took a ship to the shipyard that I had built outside of the shipyard it was fine before hand. I saved it. My purpose for using the shipyard was to get a more accurate block count by taking it apart and putting it back together. So I saved it to a different blue print when deconstructing. Upon reconstructing it everything looked fine. However I parked the ship and logged off and when I came back on noticed the turrets were floating around.

One thing I can say for certain is the Shipyard is removing the turret status on the AI and setting it to Ship. It also erases any names put on logic and removes items from the hot bar.

I went ahead and repeated the method because I had my ship saved already.

SmilingDemon added a parent task: Unknown Object (Maniphest Task).Dec 23 2015, 6:25 PM
SmilingDemon raised the priority of this task from "Urgent" to "Release Blocker".
SmilingDemon added a subscriber: SmilingDemon.
Phocian added a comment.EditedJan 14 2016, 2:26 PM

Big server ship battle, everyone has huge holes in their ships.

Dock damaged cruiser (310mL) in shipyard, select repair.
Ship disappears nothing happens.

Clear shipyard and build from design.
Everything's looks good and all turrets are there.
Move the ship, 90% of turrets are not attached or only the base section.

Manually have to place every turret, barrel and change bobbie from ship to turret for every turret.

Become suspicious and go back to the first turret I had to dock and its bobbie is back to ship, the target options have remained.
Every single turret is the same, change again but some revert back to ship.....

Advise other players who are about to repair their ships to scrap the blocks, fill a blue print and avoid shipyards for the time being as they are broken.

I've actually done some extensive testing on this. Additionally, this issue and T652 are related.

Any entity that I manually dock is not affected, even if it is a turret/turret base built via shipyard. It is only docked entities built via Create Docking or built as a part of a shipyard design that have this issue.

The principal symptom is that a bugged entity will disappear if the ship leaves the sector. There are a few ways to detect a bugged entity in advance:

  • The entity is not able to undock at all
  • Any weapons on the entity don't register impacts on other structures
  • Bobbies on the entity attempt to revert to default settings
  • Player is unable to remove blocks from the entity in astronaut mode
  • Removing the docking connection on either end does not sever the connection

The only way I have found to repair this situation is to leave the sector and let any bugged entities "disappear". When this happens, they reappear around the center of the sector they were created. The player then needs to relog and fly back to the now-corrected entities and manually re-dock them.

Asypha added a subscriber: Asypha.Mar 7 2016, 3:45 AM

I would like to note that, in contradiction to Kiithnaras's experience, I have had entities that were both spawned as docked, and manually docked come detached without any visible cause. Just today, I came back to a large build and found that I was getting around 5fps. Eventually, I discovered three separate entities un-docked from different parts of my build.

In this case, unlike previous cases, the entire build had recently been admin spawned onto the new server (starside instead of my own computer) and no one had been in that sector for around 24 hours.

AndyP has seen the ship I am talking about and although he wasn't present when anything came undocked, he may be able provide some further insight as we had discussed how long it can take for an element to be processed when first loaded.

Also of note, I cannot verify the third instance, but in the first two, the rails were in the wrong orientation to allow for re-docking, and had to be reversed to allow for it.

schema changed the task status from "In Queue (Game)" to "Resolved".Mar 19 2016, 9:01 PM
schema added a subscriber: schema.

Can I request retesting and clarify the issue. It might be fixed by fxing the docking system in the last 2 weeks

Restricted Application edited projects, added Quality Assurance; removed Game Development. · View Herald TranscriptMar 19 2016, 9:01 PM
lancake changed the task status from "Resolved" to "Open".Mar 19 2016, 10:36 PM
Restricted Application edited projects, added Issue Navigation; removed Quality Assurance. · View Herald TranscriptMar 19 2016, 10:36 PM
lancake claimed this task.Mar 19 2016, 10:36 PM

hey umm, i have found that sometimes, when i relog on a server, i happen to lose my entire fleet, and any fleets around that have 0 ships in it when it had one or seven, so i have no idea if this helps, also a confirmed issue that me and Andy have been working on is that in fleets, any docked entity is undocked after traveling through even 1 unloaded sector

This comment was removed by AndyP.
AyexeM added a subscriber: AyexeM.Jul 18 2016, 10:57 PM

I am seeing this bug in the current version 0.198.477

schema changed the task status from "Open" to "Resolved".Apr 5 2017, 11:52 AM

There have been some changes to how entities are moved within the database (0.199.495+). Needs more testing if error persists (or got less frequent, because if so we would know the root cause and can probably trace it to find the exact error)

Restricted Application edited projects, added Quality Assurance, CBS: Rails; removed Issue Navigation. · View Herald TranscriptApr 5 2017, 11:52 AM

Just a quick question on this. I was considering putting in a potentially related bug issue. But it might not be depends on how your resolved this.
The bug I was going to report is if there is a long chain of entities docked such as I had station modules docked to a station with ships docked to that and other ships docked to that one and the turrets and so on.
Apparently that also will end up loosing entities.
It is different based on which OS you are on. It is related to the directory structure and windows only being able to handle certain length paths to files.
I realized the issue when I was making large chained power supplies. I didn't consider the issue how it might effect other stuff. Or there might be a circumstance someone docked that much together.
So that was a bad assumption on my part and I should have reported it then.
Anyway the question is this issue related and does the fix to this fix what I am talking about or does that need a separate report?

It is possibly fixed with this. But I can't be sure.. The reason this seems OS dependent is because this is a running condition, so it really depends on how fast/slow threads are being handled.

Thanks for the notice though. we will try to break it by making long chains, as that seems to be making the error more probable to happen

GRHayes added a comment.EditedApr 30 2017, 8:02 AM

It still exists just checked in latest versions.
http://i.imgur.com/vXu9N1d.png
The stack you see on the left is what i created in game 128 docked entities.
The right was what came of trying to admin spawn a copy of it. 108
Figured that was faster than building an entire station and entities over again.
The system still thinks there should be docked parts on it and tries to transfer you up to it if you use the arrow keys. This results in a Long pointer null exception error.

While you are correct in that loading could be stopped by the time you allow the thread to apply to it. Of course you also know that can be forced to finish such a task. However, if you have something there to prevent the thread from hanging up to long on something that could interfer.

This might have something to do with it also though. Couldn't say for sure how java handles it. I do know windows itself has issues getting to those far directories. Last time I had to rename everything to a single letter and then delete it.
So understand I knew this was going to cause me some work.
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx

I put it in a suggestion a while back that this should be removed. The driver access and cache hits caused by using directories this way makes for it to really slow down. Far faster if you store links in a file and the entity directories under a single directory. Yes, it will require you to change the naming convention you are using slightly. You can get a much larger performance change if you zip them an unzip it in memory and bring them in that way. That single file pointer search for the beginning of the zipped file is much better.

Moving them into the single directory will probably be enough to help with the speed issue for your threads and if it is the file path length issue it solves that also.

Windows 10 made it easier to delete this time. Still has issues with path being to long though as you can see here.
http://i.imgur.com/B4wIdKS.png

I assume those entities are in a single long chain, right?

MAX_CHAIN_DOCKING = 25 //maximal deepness of docking chains (may cause glitches depending on OS (path and filename length) at high numbers)

We implemented this setting in the server.cfg since a while.
As paths got too long, and weren't properly handled by the operating systems.

However, the other parts of your comment are way over the framework of this bug report.
We had some concepts for fighting those, but they have many impacts and alterations for the userside, so we look into making the current system safe to use.

Yes, they are in a single chain one on top of the other.
It appears the server is completely ignoring the setting. It should have stopped me several times from docking then. I say that because you are correct that is in the config file.
MAX_CHAIN_DOCKING = 25 //maximal deepness of docking chains (may cause glitches depending on OS (path and filename length) at high numbers)

I first ran into when making docked power systems. Ran into it again when making a modular space station. 25 would really mess that up. When I got frustrated deleted that. Was hoping it was fixed.

When you say, "fighting those", are you referring to long docked chains or the bugs?
I can see where some ship saves would need to be fixed to match a new format. I could write a tool in 10 minutes to do that if you used the first part of the method I was talking about. I could do it in python or c, c++.
Preventing long docked chains would be another issue. That would be limiting. Limiting enough at the OS level 25 even more so.
O'well. Thanks for the info.

AndyP added a comment.May 1 2017, 8:56 AM

Fighting was about the issues long directory names can cause, if you go beyond sane limits.
Anyway, if the max chain docking setting does not work it would be a different report to look into.

It does not really matter what type of tool you could write in a different language, and also not considering the large code base around it.
Managing text and file paths is one thing, but being able to serve all use cases it has at the moment completely prevents any regular approach.

So just to prevent any further non-bug-related comments:
This is about documenting the bugs and identification of the narrowest steps required to reproduce it on our side, helping a lot with the process to fix them.
For general chatter, suggestions and other non-focused information, use the starmadedock.

GRHayes removed a subscriber: GRHayes.May 1 2017, 7:00 PM
AndyP moved this task from Backlog / Unclassed to Alpha on the CBS: Rails board.Jul 20 2017, 11:03 PM