Server Crashing on loading World Now

[Hughesbuddy]

So obviously running on a RPI is proving to have lots of bugs. Latest one and not sure if related to this being a linux build, but i've barely done anything in game, used the default world spawner tool, spawned a horse, a banker, and ran around a while. The world saves are working, but closing and relaunching uox3 is now crashing on loading world with this (ran in gdb for debug info):

| o Loading command levels [done]
| Loading World now 100% [done]
terminate called after throwing an instance of 'std::out_of_range'
what(): stof

Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) where
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0xb6bf0804 in __GI_abort () at abort.c:79
#2 0xb6e71f88 in __gnu_cxx::__verbose_terminate_handler() () from /lib/arm-linux-gnueabihf/libstdc++.so.6
#3 0xb6e6facc in ?? () from /lib/arm-linux-gnueabihf/libstdc++.so.6
#4 0xb6e6fb58 in std::terminate() () from /lib/arm-linux-gnueabihf/libstdc++.so.6
#5 0xb6e6ff28 in __cxa_throw () from /lib/arm-linux-gnueabihf/libstdc++.so.6
#6 0xb6e6b550 in std::__throw_out_of_range(char const*) () from /lib/arm-linux-gnueabihf/libstdc++.so.6
#7 0x00076bbc in float __gnu_cxx::__stoa<float, float, char>(float (*)(char const*, char**), char const*, char const*, unsigned int*) ()
#8 0x00071624 in CChar::HandleLine(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) ()
#9 0x00058204 in CBaseObject::Load(std::basic_ifstream<char, std::char_traits<char> >&) ()
#10 0x001e3db4 in LoadChar(std::basic_ifstream<char, std::char_traits<char> >&) ()
#11 0x001e47dc in CMapHandler::LoadFromDisk(std::basic_ifstream<char, std::char_traits<char> >&, int, int, unsigned int) ()
#12 0x001e679c in CMapHandler::Load() ()
#13 0x00026f80 in main ()


I've figured out that the whole shard is essentially all the random files in the "Shared" folder, wiping this out clears everything and it launches and runs fine. Trying to track down the cause of what specifically is wrong in the shard. And not so much "what" is wrong, but how can i troubleshoot it to find it out? I imagine this will not be the last issue i have, so i was wondering how can i troubleshoot these things. Looks like objects are all hex coded objects which data is saved in the *.wsc files, is there a rhyme or reason to their creation and naming convention? They seem to hold the data, and during world load something is throwing an out of range error, how could i track/trace what specific property is out of range for me to try and either fix or delete the save information for that object? I tried perusing around the JS posts and other guides on this site, but i'm not seeing these specific inner workings, maybe they're not documented? Any pointers would be great, thanks!

[Hughesbuddy]

Seemed to be Overflow.wsc, its contained both my Dark Steed i generated, and i think my backpack? What's the Overflow.wsc file used for? Definitely the dark steed, deleted the steed, re-world saved, overflow emptied. Regular horses work fine, i'll look into that later i guess..

[Xuri]

The files in the shared folder each contain a portion of the items and characters that exist in your shard. The world is essentially split into different sections horizontally and vertically, each being 128x32 tiles big, if I remember correctly. Any items or characters that exist within a given section is saved to the specific file for that section. This allows us to save only portions of the world at a time (aside from the first, initial save after a fresh startup) based on which sections contain changes that need to be saved. For example, instead of a world save that potentially takes 30 seconds every time, it instead takes between 0 (if there are no changes to save) to 30 seconds, essentially depending on the amount of players roaming around the world and interacting with NPCs and items.

Overflow.wsc contains items or characters that are not normally found in one of the above mentioned sections for whatever reasons. Do you have any recollection of where you had the Dark Steed spawned in the world, or what you did with it before the crash happened? If the crash is caused by something unrelated to the changes you did for RPI, I might be able to debug, trace and fix the issue if I can reproduce it.

[Hughesbuddy]

very insightful, so it probably threw the dark steed into overflow because i usually don't find Dark steeds at moonglow bank.. Gotcha. When you say "Not normally found" do you mean according to the existing spawn points? or some other file designating where stuff normally is? i created a horse and it went into overflow as well, but i guess i haven't seen any horse spawns in moonglow either..

[Xuri]

Hm. Are you riding these dark steeds/horses when they go into the overflow file? Then it could be because UOX3 moves the horses to a certain point on the map (7000, 7000, 0) whenever they're mounted. At that location they are no longer inside the normal boundaries of the world, so they get saved in the overflow file.

But that doesn't explain why UOX3 would crash when attempting to load a dark steed. So far I've been unable to reproduce this locally on either my Windows machine or my Ubuntu virtual machine.

[Hughesbuddy]

It didn't outright crash on mounting the steed, it was when i closed the server and it reloaded from world save that it throws the out of range error. I'll look into the location info of the steed, maybe the location stuff is why the out of range error.. But i have a lot more questions regarding general GM'ing and trying to do things (Lots of non-OSI stuff to cleanup if i can from memory..), but i'll make a new post for that, or maybe just dig up my discord credentials..

In going back to try and reproduce the issue, recreating a dark steed, mounting it, saving, restarting, etc.. i am also not able to reproduce it.. Maybe i was just tinkering with something else (Horses stats or something)? i dunno.. I wouldn't put any more time into it, if i can reproduce the issue i'll make sure to save the Overflow file and keep it for review later. Thanks!

[Hughesbuddy]

So i let the server sit overnight, came back in and was disconnected, probably timed out, no errors anywhere. Closed the service, relaunched it, and same old out_of_range error. I did the same process of elimination again and found that the 17.1.wsc file was the culprit. Except this time it's massive. Now you said this has to do with what zone things are in essentially, but trolling through 15,000 lines of this file to find the improper setting is near impossible. Is there some type of -debug mode (I'm already using gdb and it doesn't help) or even a verbose mode i can enable for the uox3 loading functions to determine what line of the *.WSC file it fails on? Or could i add a function somewhere to output each [item] or [character] as it reads them to a file, so i can figure out what the last thing that loaded was? Without that, troubleshooting the faulty values seems near impossible? I also checked my dark steed and removed it from the file and that wasn't it this time ^^

Any advice on this?

[Xuri]

Could you send me the .wsc file in question (either on discord or attach it to a post here), so I can see if it happens for me as well? I might be able to do some debugging in Visual Studio if that's the case. You might also be able to get some more details out of gdb if you compile UOX3 with debug flags, but... I don't remember how to do that for Linux ^^;

[Hughesbuddy]

uploaded file to discord. So you're saying you can place that WSC file into a completely unrelated server and load it up (via windows uox3) with debug enabled in visual studio and it should in theory throw the same error? The system doesn't care that the character/account reference isn't there? So the world save information contains all information of the characters/items itself and the Account section is merely a pointer to the characters Hex reference ID? If i have that right?

So really if that's correct, i could just spin up a windows version of uox3, load in the WSC file from my RPI server, and debug through VS to troubleshoot? If so, that'd be awesome, i'm much better with windows and visual studio. .Net is my background, so that would help alot if that's the case..

[Xuri]

That's correct. You can even remove every other .wsc file from the shared folder and throw in the culprit wsc file alone, and UOX3 should load whatever is in that file, and throw the same error. Doing this might make it easier to debug the error as well, since you wouldn't have to output debug information for every single other item/character in the world in the process.