Producing Free to Play
We’ve been asked a few times about the process of moving Pirates of the Burning Sea from a subscription model to a free-to-play (F2P) model. Did everything go as planned? Did we encounter hurdles along the way? In this devlog, I’ll give you a look behind the curtain and talk about how the process went—from start to finish.
Let’s start with some history. Not too long after launch, Rusty went to Asia to show PotBS to prospective publishers, and he came back preaching microtransactions as the way of the future. Part of my job as Producer is to keep the visionary CEO’s feet on the ground, so we had lengthy discussions with DevCo and DesignCo about feasibility; what would we monetize in PotBS? How would we implement it? How long would it take? Answer: At that point in time F2P would be crazy complicated and take us too dang long to make the model worthwhile, so the discussion was tabled.
But Rusty isn’t one to let his dreams lie fallow for long. In late 2009, when we were deeply in the process of creating Power & Prestige, he met with a company that specializes in moving games over to a free to play model. We had a few discussions with them about what tools they brought to the table and how we’d tie into them. As a result, DevCo’s earlier work estimate was cut roughly in half. But the end result was that we could either finish P&P (which was already behind schedule), or we could drop that and go F2P, but we couldn’t do both at the same time. So discussion was tabled again.
Awhile after that discussion, SOE gave us a basic overview of some new tools they were working on in preparation for EQ2’s free-to-play version. All along, one of the biggest questions DevCo had been asking about F2P was how would we take payments from players, and what would that mean in relation to SOE (who have always done handled payments for us)?
Would we run both the subscription through SOE and process payments ourselves? Or would we somehow tie a system that was not SOE’s into their system? These options were far from ideal, so when SOE presented an option that would allow us to use their system, things clicked into place, and DevCo’s time estimate dropped by 75%. Now we were talking!
Talking was the easy part.
SOE sent us their API documentation, and DevCo started implementing. Working with another company’s internal tools is always a bit tricky, so not surprisingly, mis-communications ensued. Thankfully DevCo were able to work through every hurdle with SOE’s engineering team, and we ended up with a working payment system.
We had to decide if DevCo should take time away from implementing the API in order to make pets work correctly, or should we delay selling pets until later? Many such questions arose on the way. DesignCo would want X, while DevCo wanted Y. As Producer, I would mediate and find a solution.
There were also other, higher-level decisions to make in conjunction with SOE. What would happen to the various subscription types? What would happen to expired trial accounts? What happens to any boxes still in stores? What would happen when people who got a box ages ago, but never played finally used their key? And so on. It took a lot of meetings to even think of all these, let alone make decisions on them.
And then there was the testing. Internally, most of us have our own individual server cluster that bypasses SOE’s Station authentication. In this way, we can test without disturbing anyone else’s testing, and we do so even during downtimes and outages. But in this case, with our internal clusters we couldn’t connect to SOE’s credit card handling system and test actually buying things, which was obviously rather important. Thankfully, SOE came to our rescue by letting use their staging environment, which allowed us to do fun things like use fake credit card numbers to test the payment flow. Very handy!
We also had to test the new account types. Subscribers were becoming Captain’s Club members, former subscribers were being converted to Premium accounts, and trial accounts became unlimited free-to-play accounts. We treat all of these folks differently in game. We needed to have accounts of each type to test within the staging environment. So we had to ask SOE to create some accounts for us and manually set their type to CC, Premium, or F2P. SOE were of course busy trying to get EQ2e out, so I spent a lot of time trying to come up with ways to work around issues that were blocking QA while we waited.
We had originally planned the deployment so that SOE could make the account changes a bit early on Live without actually changing the experience of players before the free-to-play transition happened. We even hot-fixed the live game so that players wouldn’t notice any change at all, but for various reasons this turned out to be impossible. Because of these issues, we couldn’t give players a preview on the Testbed server, because Testbed uses the same environment as the Live servers.
In other news, we were deciding whether to continue partnering with Akella in Russia. There were a number of reasons why ending the contract was beneficial to both parties, but what to do with our Russian players? Every conversation from then on was “OK, we’ll do X, then Y, then Z. Oh wait the Russians. OK, we’ll do X, and then do X for Caribbean, and then we’ll do Y for all servers simultaneously, and then that won’t work. OK, OK, I got it! We’ll do…” Taking over Caribbean was another ball to juggle.
Meanwhile, for whatever reason, we were suffering hardware failure after hardware failure. We’d like to be able to point our fingers at increased load on the system, but some of the failures happened before we went F2P. Also, some of them were related to Testbed which has a lot less load than, say, Antigua. One of our neighbors in the datacenter overheard our Operations guys cursing and popped around to see what was up. When they replied “hardware failures,” he said, “I’ll bet you have this brand of hardware.” And indeed we did. So Ops have their own suspicions as to the real cause. Personally, I’m thinking sun spots. Or the Mayan calendar.
In any case, between trying to ensure Antigua and Roberts had sufficient hardware for the anticipated load of F2P, setting up character transfers from Caribbean, upgrading Testbed often, all of the server maintenance that the Casual division requires, and all of the server maintenance that running a company requires, our Operations team was getting no sleep.
So I’ve touched on a few of the millions of technical implications of deciding to switch to a F2P model implementation, design changes, hardware setup, etc. But there was also a lot of non-technical stuff to think about.
We had to let the players know we were making this change, and the lawyers had to approve our wording. We also had to give at least 30 days notice before making a change that would affect players’ accounts. If we wanted F2P to go live before Thanksgiving (and we did), then we had to send out our email blast on a specific date, which meant approval from Legal by an earlier date, and so on. Even when everyone is in general agreement on what to say, there’s a bit of back and forth on the drafts to make things more clear, to fix typos, fill in dates, etc. Lawyers can often be a real bottleneck, but SOE’s lawyer was a joy, with fast turn-around and good humor all the way through. I’d love to have her involved in any legal issues I ever have to deal with in my life!
This part of the process wasn’t all that complicated, but we still had to create the email message to US/EU players (and localization thereof), the email message to RU players (and localization… you get the picture), website changes on both our side and SOE’s in preparation for F2P, more website changes when we did go live, and so on. The SOE Producer and I both had lists of lists, daily email exchanges, and weekly phone calls.
Finally the code was ready and tested, notifications had been sent, and the website was ready. It was time for the longest outage ever. It’s quite the risk to do this the week of Thanksgiving, since it’s a short week and fewer folks will be available to deal with any issues we hit, but we really wanted folks to be able to play over the holiday.
And then we hit a major issue on our side and SOE hit a major snag on their side. So, no Thanksgiving F2P. On the one hand, we were really discouraged that we hadn’t met our target. On the other hand, we were incredibly glad we had tackled these issues before we went live.
So then it was time for round two. It was really time for the longest outage ever. I really hate it when players can’t log in for hours and hours. Particularly if they also can’t log in to the forums and they couldn’t during this outage as SOE had to make account changes. So I joined our IRC channel at midnight and stayed on all night and most of the day. And most of the next two days as we tackled the issues that launch days always bring. I wanted to ensure that news was communicated as far and wide as possible. It was fun talking to players, but exhausting to have such a long day.
First, we hit ship loss. This is just about the worst possible problem for our game. So bad that we took the servers back down after 10 minutes to sort through it. Some subscribers showed up as free players. Some Russians showed up as trials. Trials!? There’s no such thing as trials!
Every issue has a number of things that have to happen. We have to figure out: Is it us or SOE? How widespread is it? Is it worth shutting down for? Are we going to reimburse? Each issue needs to be communicated to CS, to SOE, to the players, and to Dev/Ops/Design to find/fix. Then we have to communicate the plan/solution to the players, CS, each other, and maybe SOE. We have some processes in place to try to ensure no one is left out of the loop and all the right data gets gathered, but on launch day when everyone is in a hurry, it’s easy to leave out steps.
It’s my job to make sure nothing falls through the cracks. I start sounding like a broken record: “Don’t forget to log that in the bug database,” “Have we posted this on our website?” and “Hey, don’t assume it’s SOE let’s prove it before we tell them to fix it.” Some things we fixed by tweaking server configuration. Some SOE fixed. Some required code changes which always requires an outage. I’m proud of how my team soldiered on through it all and was really happy with SOE’s responsiveness to us. We tackled each issue as it arose and kept the communication flowing.
Meanwhile, tons of players showed up, and kept showing back up even through all these issues. And they’re still showing up! We ran an advertising campaign starting mid-December through early January. Even though the ads are no longer running, the flow of new players hasn’t slowed much. And they’re buying stuff! At first, the $5 Burning Sea Notes bundle outsold everything. Now larger Notes bundles sell more. My theory is that folks bought the smallest amount they could to see how it worked and now they’re stocking up on stuff.
The society I’m in has recruited a ton of new players and is helping level them up and train them for PvP. Senior players are back online much more than before. I’ve had to decide: Do I join the group helping Fred (yes, a made up name for anonymity) with missions or one of the groups trying to flip Matthew Town or the group doing dailies? It seems like there’s at least one port battle coming or going every time I log in. I can no longer produce all I could sell and I don’t have time to haul anyway because I’m too busy playing.
A wild shoulder monkey appears!
And through getting 2.2 out the door, we’ve been working on 2.3. This whole process has gone from stressful, confusing, and exhausting to really fun and exciting. I can’t wait to release some of the fun stuff in future milestones, including the shoulder monkey which is available in Testbed’s Treasure Aisle right now!