The Economics of Efficiency

Jun 27, 2016

Sourcetoad develops many different types of applications for many different types of customers. Many of our projects are never intended to be used by vast numbers of users. They are either specialized tools to assist in business functions for clients or are focused on very niche markets with very specific needs. With that said, we sometimes do work on products that the customer hopes might, one day “go viral.” These types of products present quite a few challenges when it comes to efficiency.

When Google’s engineers begin to develop a new product, they know the product will most likely have a very large user-base. Even if the product is a relative flop by Google’s standards, it will have more users than 99% of everything else on the Internet. It has to be efficient at a very large scale right from the beginning. If it is not, the product will put an unreasonable strain on Google’s computing infrastructure. For the environmentalists out there, it will also waste a lot of electricity in the form of inefficient code using a lot of computing power.

Next Big Thing

When our clients come to us with the “next big thing,” their future is much more uncertain. For every viral success, there are hundreds of products that went nowhere. Unfortunately, the cost of building a product that has the potential to handle millions of users is quite high. It would be a shame for a client to spend that amount of resources before they have any indication as to how popular their product will be—not to mention a revenue stream from the product itself. The client would likely balk at the cost, and the next big thing might never even get started.

Clearly, the client should just build something that works, get it out there, and, as it starts growing, invest in making the product more efficient and scalable, right? Well, unfortunately, no, it doesn’t work like that.

Going Viral

The concept of “going viral” is not a linear growth curve. It is more like having no users one day, and more than you could imagine the next. There is no time to refactor the codebase to handle the change. What’s worse is, if your product chokes when the influx of users comes, they will just move on, having completely forgotten about it within minutes. The big shot came and was squandered by unresponsive servers.

This is where the challenging part comes. How can we, as a company, balance the client’s need to not unnecessarily spend money developing a product able to handle a large number of users who may never come against the need to actually handle those users if they do?

The Cloud

Cloud infrastructure can help, to an extent. Products on hybrid or cloud-based services that go viral can be scaled up. They may not yet be optimized enough to efficiently handle the huge influx of users, but that can be compensated for by more computing power.. at least to a point. This additional computing power hopefully will buy enough time to refactor the code to handle the new reality of the “viral” product.

The more important thing to do, however, is to plan ahead. The system should not be built in a way that makes it hard to scale in the future. This seems obvious, but, the vast majority of systems don’t do this. This is because most developers do not take the time to consider the longer-term implications of the initial decisions they make. A small action now, that is only slightly more difficult than the path of least resistance, may have extremely expensive consequences later.


This is what we do at Sourcetoad. We will take the extra time to make sure we don’t box ourselves in with the decisions we make during the initial design phase. This is where spending a little extra development time helps. Will it scale to millions of users from day one? Not efficiently, no. That would be prohibitively expensive. Instead, the goal is to design it in such a way that, if and when the time comes to redevelop for that scale, the answer isn’t, “We have to redo everything we did, and spend tons of money redeveloping the data models from the way it is now.”

It is this mindset that allows the client to at least try out the idea without losing a fortune if it doesn’t work out. It’s a balancing act, to be sure, but tradeoffs exist in every aspect of life. Software development is no different.


Building an AI Policy for Your Company

Building an AI Policy for Your Company

Artificial Intelligence (AI) is an integral part of our present-day technology landscape. At Sourcetoad, we’ve been assisting our clients for years in leveraging this powerful tool. However, it’s become more important than ever to have an internal policy on how these...

Sourcetoad Presents: Fine-Tuning OpenAI

Sourcetoad Presents: Fine-Tuning OpenAI

Welcome to Sourcetoad’s first video in our Artificial Intelligence series! In this video, Sourcetoad’s own AI subject matter experts discuss how to fine-tune OpenAI. After watching this presentation, you’ll learn: ✅ What fine-tuning means✅ How to use fine-tuning to...

The Agile Manifesto in Practice: Part 2

The Agile Manifesto in Practice: Part 2

The Agile approach to software development first emerged in response to the inflexibility of existing project management methodologies. Unlike traditional approaches that emphasize extensive documentation at every phase of development, Agile instead values the...