Update: Server lag optimzation and client improvements
July 21, 2018
With the recent influx of new players to the game, the servers have been struggling to keep up.
Several months ago, I spent a lot of time on server database optimization, which made the servers around seven times more efficient than they were originally. But as maps fill up with player content, there's more and more information to process, and the load generally grows over time. In the past week, Server1, which has the most extensively settled map, had gotten very laggy. It was time to take another look.
This round of profiling revealed a bunch of hot spots that weren't database-related. The optimization process involves running a server with a profiler (I use the amazing Callgrind profiler from the Valgrind project), finding the most obvious hotspot, figuring out if there's a way to speed it up or---even better---skip that operation entirely, and then repeating with a new build to find the next biggest hotspot. I ended up going through this process nine times, fixing nine hotspots along the way. Some of these changes resulted in 7x speedups to certain parts of the server code.
However, even after several days of intensive work that showed huge performance gains in the profiler, when I finally brought Server1 back online for public use and a load of 38 players, the lag returned. There are more issue afoot with Server1 than just slow code. CPU usage jumped up back to 70%, while Server3 sits happily at around 20% with exactly the same player load.
So, Server1 and Server2 will remain "on ice" over the weekend, at the end of the server list where no one will use them by default, until Monday when I will resume diagnosing Server1's lag issues.
You may have also noticed that the connection management features of the client have been greatly improved. You can now specify a custom server from the SETTINGS screen, and copy/paste server addresses to share with friends. Bugs in the twin matching have also been resolved, so joining twin games is reliable. Some issues that caused logins to fail have also been fixed.
|
|
|
