Joel Lehman and Kenneth O. Stanley (2011):
Most ambitious objectives do not illuminate a path to themselves. That is, the gradient of improvement induced by ambitious objectives tends to lead not to the objective itself but instead to dead-end local optima. Indirectly supporting this hypothesis, great discoveries often are not the result of objective-driven search. For example, the major inspiration for both evolutionary computation and genetic programming, natural evolution, innovates through an open-ended process that lacks a final objective. Similarly, large-scale cultural evolutionary processes, such as the evolution of technology, mathematics, and art, lack a unified fixed goal. In addition, direct evidence for this hypothesis is presented from a recently-introduced search algorithm called novelty search. Though ignorant of the ultimate objective of search, in many instances novelty search has counter-intuitively outperformed searching directly for the objective, including a wide variety of randomly-generated problems introduced in an experiment in this chapter. Thus a new understanding is beginning to emerge that suggests that searching for a fixed objective, which is the reigning paradigm in evolutionary computation and even machine learning as a whole, may ultimately limit what can be achieved. Yet the liberating implication of this hypothesis argued in this paper is that by embracing search processes that are not driven by explicit objectives, the breadth and depth of what is reachable through evolutionary methods such as genetic programming may be greatly expanded.
Late in their essay, Lehman and Stanley illustrate their point by describing the navigation of mazes: If you’re going to make your way from the periphery of a maze to the center, you have to be willing to spend a good bit of time moving away from your goal. A determination to go directly towards your goal will “lead not to the objective itself but instead to dead-end local optima.”
(I got to this by following some links from Samuel Arbseman’s newsletter.)
I think this insight has implications far beyond machine learning, and even beyond what Lehman and Stanley call “large-scale cultural evolutionary processes.” It’s true of ordinary human lives as well. When we define our personal goals too narrowly or too rigidly, we render ourselves unable to reach them — or to reach them only to discover that they weren’t our real goals after all.
There’s a wonderful moment in Thomas Merton’s The Seven-Storey Mountain when Merton — a new convert to Catholicism — is whining and vacillating about what he should be: a teacher, a priest, a writer, a monk, something else altogether maybe, a labor activist or a farm laborer. And his friend Robert Lax tells him that what he should want to be is a saint. It’s a marvelous goal not only because all Christians are called to be saints but also because there’s a liberating vagueness to the pursuit of sainthood. In his great essay on “Membership” C. S. Lewis comments that “the worldlings are so monotonously alike compared with the almost fantastic variety of the saints,” and it’s true: there are so many ways to be a saint, and you can never know which of them you’ll be called to take.
I think these thoughts may have some implications for secular vocations as well.
