Last week, we held a general load test in IMA Outpost Alpha. Thanks to all who participated! During that test, we had the general avatar activity but we also were simulating rapid user chat activity with ALICE chatbots courtesy of Radegast developers. The simulator did crash during the load test but unfortunately, standard logging did not provide us with meaningful data to isolate the cause of the crash. No viewer crashes were observed. In preparation of a repeat test, logging parameters were adjusted.
Today, we did another test on IMA Outpost Alpha. This time, the same four users with Radegast were logged in to simulate rapid chat activity using the ALICE chatbot plugin enabled. In addition to those, there were two other users (myself and Neovo) observing. Within 25 minutes, we began to see indications of simulator issues (chat lag and viewer rendering interruptions). Five minutes later, the first Radegast simulated user was logged out without warning but also remained a "ghost" for a short time. Subsequent Radegast simulated users experienced log outs and ghosting.
All four simulated users were then relogged and their chat simulations resumed. Watching closer this time, within the same time frame as before, we began to see avatars start and stop typing animations and begin changing rotations randomly just before they crashed. This is of note because all four simulated users were not getting mouse or keyboard inputs to change their rotations. The result by the end of the 30-40 minute cycle was the same as before - and that included not relogging those logged out in sequence. The simulator performance issues were also noted by Neovo who did not have any simulated users hosted on his system.
Chat lag, ghosting, and unpredictable performance in busy sims is a known issue without resolution. Our goal was to see if we could find a way to collect data that can be analyzed and a repeatable test that can be performed on multiple versions of Open Simulator and its derivatives. We hope the data collected is more meaningful this time!
We do plan to repeat this test with fewer variables. Next week, if we could get more human participants, we would like as many people as possible to login their avatars with the latest version of Radegast (one per system) and enable the ALICE chatbot plugin as a simulated user. If you have additional systems please by all means feel free to log in from those with your preferred viewer to add to the load with human user behavior. We want to eliminate the possibility that viewer crashing was related to multiple viewer instances on the same system.
After the results of this test, we hope to develop a test preparation sheet for the Test Management Working Group to use in repeating the test at multiple locations around the Metaverse and on our test grids as we fire them up! The end goal is to have data available and submission of bug reports to development teams if this issue still exists in the latest versions of Open Simulator and its derivatives. Hope to see you next week!
You can find Cinder Roxley's latest release of Radegast at https://bitbucket.org/cinderblocks/radegast