Load testing Archives

Pre-Launch Prep

I’ve been advising a few pre-launch startups that are getting ready to do their first ever product launch. From first-hand experience I know that the first product launch can be nerve-wrecking. You expect the product to be pixel perfect, and all the features to be fully functional and bug-free. But there is such a thing as spending too much time on perfecting the product, and forgetting that you need to be able to support load on the system. What good is a shiny product that people can’t even sign up for because a server has gone down, or their signup process takes on the order of minutes or hours!

Here are a few tips I’ve been giving people to help alleviate their pre-launch preoccupations:

Logging. For good developers this should be a no-brainer, because they would havetested or debugged their code and would have included logging to do so. For Java I use log4J, Ruby has its own class Logger. You also want to make sure you are rotating the logs daily.
Keep an eye on Exception and Errors. Once you’ve got your logging in place create a simple shell script to grep the logs and tally up errors and exceptions. You can set this up in cron to delivery the logs to your email every hour or day.
Fix Showstoppers. These are bugs that prevent a user from signing up, or are glaringly obvious and create a negative and lasting impression on new users.
Monitor Your Servers. Tools like Nagios or Ganglia.
Monitor the drop off points. There are typically three: coming to the site, sign-up, and then performing the first action (depending on your product’s features). To do this you will need Google Analytics.
Load Test. Test the limits of your system so that you know how much load it can handle. JMeter is a good tool to use to do this.
Load Balancer. If you are responsible for your servers then I’d highly recommend doing this. Check out this intro lesson on load balancing.

Hold off on all new feature requests! First impressions count for a lot, and whats more important than being feature rich is being able to have a site up and running. What good are more features if users can’t even get through your front door! Focus on supporting the new and incoming users and giving them a worthwhile and timely experience. Stay tuned for a follow-up on what to do the morning after…

Think About Scale from the Start

If you are thinking about scaling a web application or service, congratulations, because you have users that liked you or were curious enough to sign up and stick around! You will of course be acquiring more users shortly. While the trajectory of user growth is unknown, and depends a lot on your usage model (viral social network vs. word-of-mouth individual user service) there are a few things you need to address:

Capacity – your site will need to handle more concurrent users, signing up users alone can generate a lot of load on the system, even before they get to using the product.
Reliability – users will want to use the service on their own time. The site needs to be up and running 24 hours with limited maintenance windows.
Scalability – if users can generate data on your site you will have more data you need to store/retrieve.

If you’re growing too fast a common way to solve #1 and #3 is to throw hardware at the problem. A startup focuses on creating the MVP (minimum viable product), which means the prototype has just enough functionality to add a significant value to the lives of users that convinces them to sign up and use it for awhile. Putting the product out there initially means you’re testing the product/market fit, and as a result you’re unsure of how many user will signup, and what their usage patterns will be . Let’s say you are a cocky and a cheapskate, you know you’ll have users, but you don’t want to solve problems by buying hardware all the time. If you’re cautious you will do the following:

Start performance tuning and load testing even before releasing the product!
Create a restricted alpha and beta, which allows you to control the growth rate.
Measure the adoption rates and usage patterns for your alpha and beta users.
Use the measured adoption rate to anticipate how many servers you can afford.
Monitor spikes in adoption due to press releases or other news events. And be ready to re-route traffic (failover) in the event of a server failing.

These are the top 5 ways you can initial think about scaling your app without a whole lot of code re-writes. But there will come a time in which you will need to redo a lot of the prototype’s code base. We’ll save that for another post…