The worst trap that people who are pursuing automation fall into is the desire to automate everything. That’s usually a road to disaster. Automation is supposed to save time and money, but it can wind up costing you both if you don’t carefully consider what you automate.

How Automation Goes Wrong

Elon Musk, Chairman and CEO of Tesla

Elon Musk found this out the hard way. His original dream called for the Model 3 to be built almost entirely by robots. He believed that automation would increase the speed and decrease the costs of his production line. However, as GM found out in the 1980s, when an automated line goes wrong, you wind up automating failure instead of success. Apart from the fact that the failure is happening on a bigger scale than formerly, you must freeze the entire production line while specialists determine its cause(s).

Full automation is possible but the time, money, and effort required to get it right usually don’t end up being worthwhile. It tends to work best with very mature production tasks where every possible contingency is well-known and well-accounted for.

In order to determine whether your automation project can succeed, you must step back and ask yourself why you are automating in the first place. Usually, automation is intended to reduce costs. We often assume that automating a task will make it cost less, at least in the long run. But how long is this “long run”? Will the situation you are automating remain the same for long enough to recoup your costs?

How to Choose Your Automation Tasks

Whether you are automating a factory full of robots or just a typical office, choosing which tasks to automate is the key to success. When deciding, be sure that you know how much time is being spent on a given task.

Consider the following case. Max is responsible for data entry. For the most part, he reads data and inputs it into the computer. Every once in a while, though, he has to look up additional data and perform some additional calculations to enter a record. Sounds perfect for automation, right? Well, that depends. In order to determine whether or not the task should be automated, Max needs to ask himself the following questions:

• How much time is he actually spending on these extraneous tasks?
• How similar are they? Is he looking up the same data every time? Different data? Are the calculations the same every time?
• How likely is the process to change in the future?
• How much time will it take to automate the procedure?

For instance, let’s say that Jenny is the programmer. If Max spends one hour each week on these tasks but it will take Jenny a week (40 hours) to program the automation, has Max gained anything? If, next year, the process changes, Max has not gained anything. If the process (and its exceptions) remain constant for the next ten years, it is a huge gain.

That’s why, ironically, more nimble companies are often less automated. They can adapt to changes cheaply. Heavily automated companies require heavy capital investments to make adaptations.

Practical Automation Principles

When implementing automation, I have found that three principles stand out and that these principles surprise many non-programmers:

1. Leave yourself an out.

Tesla Model 3/Smnt  (CC BY-SA 4.0)

It may sound strange, but the most important thing you can do when automating a process is to make sure there is a non-automatic way of performing it. Every system features exceptions that are expensive to code. If you leave yourself a manual component, you can implement the automations that offer the most bang for the buck first. Leave the more complicated components for when you are sure that you will benefit from the automation. Additionally, a manual option makes it easier to adapt when requirements change.

As an example of this principle, think of automated billing systems. For the most part, in an automated billing system, the work your company does for a client is automatically added to the bill which is generated at the end of the month. So, for instance, if your consultancy firm charges by the hour, consultants enter their time in the timesheet, and the billing system takes the time stats and the billing rate and creates an invoice for the customer. This is fine for most processes. However, any sizable business will wind up with exceptions. Large Customer X needs the bills every six months rather than every month. Or the firm may be billed at a fixed, monthly fee. Or perhaps the firm is billed based on specific deliverables.

Given the many possible contingencies, you don’t want to lose the sale of a lifetime just because you over-automated the billing system. Instead, a well-designed billing system will provide a checkbox or similar item on the customer record which says “exception” or “process by hand.” It will then feature a text box with a plain-language description of the billing instructions. Then, every month, someone reviews the customers marked as an “exception” and processes their bills by hand.

Even something as simple as a checkbox that tells the system not to auto-process a record and a text field that tells a human what to do instead can turn an overly-rigid automated process into a system that can more easily adapt to needs.

2. Remember that failure conditions are the most costly to implement.

When we think of automation, we usually picture only the successes of each step. That is, we assume that each step in the process will succeed and we  need only handle the outcome of the successful processing of each previous stage.

However, in reality, nearly every process has failure conditions. Some of them are more obvious and foreseeable than others. What happens if the network connection between two servers is down while processing? What happens if someone entered bad data or data that wasn’t bad but was outside the range of what another component was expecting (and had tested for)? What happens if a machine is jammed? What happens if the printer runs out of ink during the process? What happens if we run out of disk space? What happens if the server reboots during the process?

One really, really strange situation occurs once a year—the “fall back” of daylight saving time. Imagine that you have a long-running process that is best run after-hours. You think, “Hey, let’s run this at 1 AM when no one is on the system!” However, when the clocks “fall back,” 1 AM comes twice in the same day. What happens to your automated system if it winds up running twice?

Automation brings out failure scenarios like this that we don’t even anticipate because we just gloss over the circumstances in ordinary life. In fact, one benefit of automation is that we become better able to recognize all these pieces of the process. There is no better way to understand the interactions you aren’t aware of until you automate what you think you know and wait to see what breaks.

Every failure scenario must be accounted for in some way in an automation project. Coding for the failure scenarios often costs more money than the basic automation itself. Even programmers, especially younger ones, tend to not take these scenarios into account when planning automation projects. It is easy to just estimate for the case presented rather than to pry deeper to find the exceptions and potential failures.

3. Allow yourself to reconceptualize.

The biggest problem we run into when automating is the idea that an automated process should look like the non-automated process, except that machines do the work instead of people. That is often the best way to start, if for no other reason than this: It allows an automated system to be phased in while the kinks are worked out. Replacing individual manual components of a system with automated ones offers immediate benefits with fewer risks. However, the biggest gains usually require reconceptualizing the task. For example, a task can be reconceptualized with the explicit goal of minimizing failure conditions.

Automate with Caution

Don’t get me wrong, I am a fan of automation. Much of my livelihood has come from writing automated systems. However, I have also seen huge amounts of money wasted by companies that were trying to automate the wrong things. I have also seen many companies surprised by the high costs estimated for automation because they fail to see the significance of planning for failure. Finally, I have seen the costs of automation projects skyrocket because the company wanted “full automation” rather than figuring out a way to leave themselves an “out” for the stickier situations. Sometimes, complete automation is possible. However, that usually happens in a field that is mature enough that experienced personnel know very intimately all of their possible failure scenarios, based on a long history or else on reconceptualizing the problem.

Jonathan Bartlett

Jonathan Bartlett is the Research and Education Director of the Blyth Institute.

Also: by Jonathan Bartlett: AI and the Future of Murder If I kill you but upload your mind into an android, did I murder you or just modify you?

Self-driving vehicles are just around the corner On the other side of a vast chasm…

and

Guess what?: You already own a self-driving car. Tech hype hits the stratosphere