Intro

Many of the most significant principles that we rely on in our day to day lives are imprecise. Some, such as "the early bird gets the worm" and "hard work pays off," fall under the umbrella of "wisdom."

If you ask someone where they stand on, "hard work paying off," you'll get a variety of answers. Most will say, "of course it does." Some might say, resentfully, "no it doesn't!" Many others would fall in-between these positions.

I would say that, "hard work pays off" is a self-evidently true statement, regardless of any given exceptions where it's not true. This is an example of a heuristic algorithm. It's imprecise, but yields results consistent with its expectations. The "exceptions to the rule" are not relevant. People intuitively comprehend this nuance, unless they're seeking not to.

Unfortunately, we're living in a society today which has become obsessed with the exceptions to the rule. So much so that it's difficult to declare much of anything without including endless levels of qualification drivel. There's a wannabe "scientist" around every corner ready to squeal, "actually, one of my friends works really hard and never gets ahead."

I spend most of my time, personally and professionally, dealing with problems that have absolutely no provable, sans-exceptions, "correct answers." I suspect the same is true of most people. In order to be productive, again personally and professionally, I've trained myself to build up and iterate on imperfect algorithms to tackle problems efficiently. For example, I'm constantly adjusting my parenting style. The algorithm is radically different for each of my children. Frustratingly for this particular problem, the solutions I had yesterday don't work as well today, and the ones I discover today won't work as well tomorrow.

Many problems are like this. I think it's safe to say that all the most intensely profound problems facing mankind are. They always have been, and they always will be.

Estimation Example

Let's go over a heuristic algorithm employed quite ubiquitously on software engineering teams - estimation.

Anyone whose had experience estimating how long a task will take will immediately have some thoughts rushing through their mind. Maybe you'll think about tasks that are easy to estimate with a high degree of accuracy. In all likelihood, examples of these would be "well known defined processes" (ex. making a peanut butter sandwich). The other end of that spectrum would be tasks that are hard to estimate with any degree of accuracy, quite often being, "poorly known empirical processes" (ex. win the war!).

The further to the right on this spectrum, the more difficult and less accurate any estimation is. Engineers will generally resist providing estimates until the unknowns are at some tolerable level. But even then, estimations are often affectionately known in software as SWAG (software wild ass guess) estimates.

So, what's the heuristic algorithm employed by most software engineering teams? Called, "wideband delphi," most software teams have all engineers "throw out an estimate" simultaneously. Teams then go down one of a few different paths. They may iterate their process until they reach a consensus estimate. Or, they may go with the mode or mean estimate.

There are other, shall we say, less heuristic solutions that yield more accurate results. Lookup COCOMO and COSYSMO if coming up with really accurate estimates does it for you. The problem with these, more accurate methods, is that they're incredibly costly and not that much more accurate than the relatively cheap "wideband delphi."

Extending the Reasoning

So, why do we choose less optimal algorithms? It's somewhat intuitive, but let's say it outright anyway. We do it because it's easier, more practical, and has a better return on investment.

Most people are familiar, and in agreement, with the concept that, "everything is relative." Is "everything" relative? What is "everything" relative to? That nuance matters. People understand the sentiment without qualifying the statement with absolute precision. Communication itself is a highly heuristic endeavor. People empirically iterate on their messaging until they've found a balance of accuracy and effect. The intended audience adds a great deal of complexity as well. You can actually decrease the effectiveness and understandability of a message by increasing the level of detail and accuracy of the content.

Consider writing requirements for some task; something as simple as ordering a pizza would work nicely. How much detail do you include? Even for a simple task there's a ton of room for failure. What if I told you that there is a way to write mathematically verifiable requirements? Those are called, "formal requirements." Any guess as to why almost no organization actually writes formal requirements?

If you guessed, bad return on investment, you're catching on. Only the most overwhelmingly risky endeavors could ever justify formal requirements. Those mathematically verifiable requirements aren't exactly plain English either by the way.

The law of "diminishing marginal returns" is at play. For complex empirical problems, you can trade time and effort for correctness and quality indefinitely. The situation dictates the level of correctness and quality needed. Given the diminishing positive effect on returns, and the inevitable "negative returns" that follow, it pays for people to develop some sense of when they should, "quit while they're ahead."

Nature's Heuristic Algorithm

There is a heuristic algorithm which I believe governs just about everything. It's called, "natural selection."

Generally people will distill natural selection down into the more immediately understandable phrase, "survival of the fittest." These four words encapsulate many concepts at once. The word, "survival" evokes this concept playing out among living things, and "fittest" calls forth the un-enumerable layers of context that may be at play.

Natural selection, or survival of the fittest, doesn't just apply to biology though. It applies to ideas, programs, systems, on and on ad infinitum.

Given enough time, survivors can be presumed to be "fitter," but it's clearly not a perfect process.

How I Embrace Heuristics

The below is my best attempt at formalizing my general strategy in life:

I struggle towards a goal with greater aggression and intensity until the momentum seems to slow, then I dial back just a bit, and maintain until either the goal is reached or it becomes clear to me that whatever I'm pursuing just isn't in the cards.

The aspect of this process I've had to refine most is goal setting. The more reasonable the goal, the more likelihood of success. Even still, I don't shy away from seemingly massive goals with this strategy as long as the payoff seems worth it.

For instance, I decided in 2013 that I would leave the Marine Corps after my second enlistment was up at the end of 2016. I also decided that same year that my next career would be in software development. That was a pretty tall order to set from a bunk in Afghanistan, given my lack of any experience whatsoever in programming.

To me, the primary indicator of success is my own internal sense of momentum. Even if along the way toward a goal I discover that the goalpost is actually further away than I originally anticipated, nothing changes so long as I feel my momentum is good. The distance from the goal is much, much less important to me, than the speed at which I'm approaching it.

When I abandon a goal, it's almost always a result of slowing or stalled momentum.

I now recognize my strategy as a heuristic for targeting the point of diminishing marginal returns. If I reach that point and momentum is good, I continue. If I reach that point and momentum is not so good, I redirect my energies elsewhere.

Conclusion

My life experience has thus far led me to believe that "perfection" is not only unattainable, but not something worthy of pursuit. "Perfect' abstracts an unattainable ideal state. Most hunts fail, but no one has ever truly feasted on "perfection."

"Better" and "stronger" are certainly worthy of continual pursuit (Kaizen). These relative words offer in their context some frame of reference and are therefore self-evidently attainable. Perhaps more importantly, better and stronger are abstractions of what naturally occurs in the world around us. Perfect on the other hand...

As far as I can tell, the only utility which can be derived from "perfect" is the reminder that we, and our creations, can always become stronger and better. Perfect is not something that naturally occurs in the world around us. On some level, I think our first instinct if we were ever presented with something truly perfect might just be to destroy it. It would scare, shame, or otherwise offend us.

Whether we like it or not, we seem to be subjected to a naturally occurring heuristic algorithm. Perhaps with enough context and computational power it would be possible to divine some exacting calculation to determine what is "better" or "stronger," but in keeping with my general theme here, there's very little practical value in putting any eggs in that basket.

No, instead we should use our time, energy, and intellect recognizing the patterns of success we see, experience, or otherwise come to know, and apply those heuristics in our lives. We do ourselves no favors by mocking algorithms that work 90% of the time because of the 10% of the time they don't, particularly when no superior alternative is presented.

You owe your existence to countless generations of your ancestors who succeeded in the most fundamental way life can. Recognize that undeniable success and seek to iterate on it. This is the basic heuristic we should all embrace. Unless of course you'd prefer to be devoured by those who do.