Ahoy there Mateys! My name is Tarun, and I work on Flying Lab’s top-deck as a Software Development Engineer in Test (SDET) which means that, among other things, I work on our automated test system.

Automated testing is a way for us to create controlled and repeatable software tests using tools on specially modified clients. It is an essential part of our QA process.

Recently I have been working on performance testing our new rooms. These rooms are areas that will eventually be added to the game as towns, Port Battles, Skirmishes or mission backdrops. We need to ensure that they can handle a massive swarm of players (in the case of avatar rooms) or a huge armada of ships (in the case of ship combat rooms) and still deliver a decent frame rate.

We also do this automation when testing other graphics changes, such as when we write new shaders or when we add more NPCs to a town.

With the help of a “live-like” game environment that we have setup right here in the FLS office server room, I run automated tests that mimic a fully populated battle by spawning AI ships or avatars to heavily populate rooms. I then collect metrics based on the performance of the game client in that situation.

We analyze these data, compare it with our preferred benchmarks, and then figure out which areas still need more performance improvements. Overall, the goal is to find a balance between detail and performance.

For example, with the help of these performance tests we were able to clear some bottlenecks in the new Vulcan’s Caldera skirmish room. We found out that the polygon count was at twice the normal level for a skirmish room. Our artists were then able to go back and mitigate the situation, which considerably improved performance.

This example also helped us identify some regularly-occurring bottlenecks, which enabled our programmers to write a new room finalizer that will improve the performance of all rooms that we process in future.

As you can see, automated tests are a very important part of our QA process. My goal is to continue to improve our automation and, as a result, your game experience.