The Economics of Efficiency

by | Jun 27, 2016

Sourcetoad develops many different types of applications for many different types of customers. Many of our projects are never intended to be used by vast numbers of users. They are either specialized tools to assist in business functions for clients or are focused on very niche markets with very specific needs. With that said, we sometimes do work on products that the customer hopes might, one day “go viral.” These types of products present quite a few challenges when it comes to efficiency.

When Google’s engineers begin to develop a new product, they know the product will most likely have a very large user-base. Even if the product is a relative flop by Google’s standards, it will have more users than 99% of everything else on the Internet. It has to be efficient at a very large scale right from the beginning. If it is not, the product will put an unreasonable strain on Google’s computing infrastructure. For the environmentalists out there, it will also waste a lot of electricity in the form of inefficient code using a lot of computing power.

Next Big Thing

When our clients come to us with the “next big thing,” their future is much more uncertain. For every viral success, there are hundreds of products that went nowhere. Unfortunately, the cost of building a product that has the potential to handle millions of users is quite high. It would be a shame for a client to spend that amount of resources before they have any indication as to how popular their product will be—not to mention a revenue stream from the product itself. The client would likely balk at the cost, and the next big thing might never even get started.

Clearly, the client should just build something that works, get it out there, and, as it starts growing, invest in making the product more efficient and scalable, right? Well, unfortunately, no, it doesn’t work like that.

Going Viral

The concept of “going viral” is not a linear growth curve. It is more like having no users one day, and more than you could imagine the next. There is no time to refactor the codebase to handle the change. What’s worse is, if your product chokes when the influx of users comes, they will just move on, having completely forgotten about it within minutes. The big shot came and was squandered by unresponsive servers.

This is where the challenging part comes. How can we, as a company, balance the client’s need to not unnecessarily spend money developing a product able to handle a large number of users who may never come against the need to actually handle those users if they do?

The Cloud

Cloud infrastructure can help, to an extent. Products on hybrid or cloud-based services that go viral can be scaled up. They may not yet be optimized enough to efficiently handle the huge influx of users, but that can be compensated for by more computing power.. at least to a point. This additional computing power hopefully will buy enough time to refactor the code to handle the new reality of the “viral” product.

The more important thing to do, however, is to plan ahead. The system should not be built in a way that makes it hard to scale in the future. This seems obvious, but, the vast majority of systems don’t do this. This is because most developers do not take the time to consider the longer-term implications of the initial decisions they make. A small action now, that is only slightly more difficult than the path of least resistance, may have extremely expensive consequences later.

Wrap-Up

This is what we do at Sourcetoad. We will take the extra time to make sure we don’t box ourselves in with the decisions we make during the initial design phase. This is where spending a little extra development time helps. Will it scale to millions of users from day one? Not efficiently, no. That would be prohibitively expensive. Instead, the goal is to design it in such a way that, if and when the time comes to redevelop for that scale, the answer isn’t, “We have to redo everything we did, and spend tons of money redeveloping the data models from the way it is now.”

It is this mindset that allows the client to at least try out the idea without losing a fortune if it doesn’t work out. It’s a balancing act, to be sure, but tradeoffs exist in every aspect of life. Software development is no different.

Recent Posts