Now that we have had our initial release opening, it's time to take stock of what happened. A lot of things went right - but we also got some things wrong and there are lessons to be learnt from that.
The Good
Performance
The performance of the server was exceptional. At 80 players we had 20 TPS and around 18 MSPT with near zero lag spikes (> 50 MSPT). The feedback we have received from players is that the server was smooth.
The massive MSPT spike you see is when the server crashed. Other TPS spikes is when the server map is switching.
From this data, we can assume that the server can easily support 200 players and likely 300 once optimisations are done.
Osgiliath & Helms Deep
These were our two most successful maps. Osgiliath works well for especially large player counts as it spaces players out well. Both of these maps were also mostly finished and balanced pretty much evenly.
In the future, we are planning to include the ride out in Helms Deep, however currently this is awaiting new terrain to be built.
Bugs
Bugs that were found were quite minor. There were no errors in console, thus the server is quite stable.
If you have any bugs to report, please use the bug reports section of the forums: https://forum.empirewar.org/forums/bug-reports.8/
Tutorial retention & resource pack usage
All new players completed the tutorial and almost 100% of players used the resource pack.
The Bad
Obviously, this first thing is what many of you experienced if you joined early on.
Memory Leak & Crash
The problem
Approximately 30 minutes after v3 started opening, the server crashed. Prior to this, we were experiencing massive lag spikes. The final death blow was this massive Garbage Collector spike:
Analysis
After analysing heap summaries, the issue was determined to be a memory leak in our anticheat. It is likely to be an issue in a library that it depends on, rather than our own code, however analysis is still ongoing.
Update: The possible issue has been identified and a fix is being tested. The results look promising.
It was identified as the cause fairly quickly as the same issue had been seen on v2 a week prior, however I wrongly believed it did not affect v3, as we did not test with as large a player count.
Solution
Thankfully, this is not a critical part for the server, except for catching cheaters. If this had been an issue in a plugin such as our core, the entire release would have had to be scrapped as memory leaks are difficult to solve.
The solution was to remove the anticheat. Of course, this meant we were only able to ban the most blatant of cheaters.
Black Gate (and partly Minas Tirith)
The problem
Black Gate was, unfortunately, quite unplayable. This is entirely a mistake on our part. The map was finished by the builders only in the last few days so setup was rushed and many features not implemented.
The woolmap was not functional, teams did not have their constructor/engineer kits, thus building siege towers was impossible.
Minas Tirith was partly a failure because we did not have Grond implemented and the siege towers did not move. Again, this came down to lack of time.
Analysis
We should have given ourselves more time to prepare the server for release. However after 4 years of developing we felt the need to produce some kind of server this year after a lack of information about it. Additionally, we only saw this problem when it was ~2 days before release, and believed it would be poor of us to delay at such short notice after hyping the server and getting everyone to prepare.
Solution
We are going to take our time to fully implement this map and make sure it is ready for release. Initially, our plans were:
- For the first few minutes of the game, Gondor gathers resources to be able to build siege towers.
- The Black Gate opens in an epic sequence of animations, and Mordor spills out to have a skirmish at the camp.
- The Black Gate closes, and Gondor is now able to build siege towers.
- Past the gate itself, a buildable bridge by constructors between the valley.
- And a set of tunnels that could be excavated by Mordor.
Donator Compensation
For many players, including myself, it was confusing how we were compensating v2 donators. We should have made a forum post about this beforehand.
Compensation has to be done manually. v2 donators are compensated like this:
- 1 EUR donated on v2 = 4 gold on v3.
- We then take the new v3 gold value and divide that by 10 to get the v3 money donated stat, which determines your rank like in v2.
Statistics
Throughout the game, our core plugin collected some basic statistics on maps and kits.
Our most balanced maps were Helms Deep (slightly biased to attackers) and Osgiliath (slightly biased to defenders). Minas Tirith saw mordor have far higher deaths, even with more players, and will need to be rebalanced.
The most powerful kit was Gondor Captain:
Archer is also a fairly powerful kit:
However, Harad Archer fared the opposite:
Elven spearman is the most powerful spear class:
Troll clearly needs a slight buff, as it has a poor KDR, and is meant to be a strong class:
Player peak & partnerships
We reached a peak of 78 players with 148 unique joins (excl. those who previously played on v3 recently).
We would like to thank VeryPotato for their partnership in advertising our event on their Discord, and AragonTelly who streamed the server.
Conclusion
We are going to be working hard on a new update that will fix issues found and improve the quality of the server, taking our time to do so. There is still a lot of work to be done and this is by no means the finished product - it should be seen more as an alpha test.
In the coming days we will post some form of roadmap of our future plans for the server. We hope that everyone enjoyed playing, and look forward to seeing you at the re-launch!
Remember, if you have any suggestions or bugs to report, use the appropriate forum section.
Last edited:
