The Illusion of Understanding Success

In December of 1993, J.K. Rowling was living in poverty, depressed, and at times, contemplating suicide. She resided in a small apartment in Edinburgh, Scotland with her only daughter. A recent divorce made her a single mom. Reflecting on the situation many years later, Rowling described herself as, “the biggest failure I knew.”

By 1995 she finished the first manuscript of Harry Potter and the Philosopher’s Stone, a story about a young wizard she began writing years before. The Christopher Little Literary Agency, a small firm of literary agents based in Fulham, agreed to represent Rowling. The manuscript found its way to the chairman of Bloomsbury, who handed it down to his eight-year-old daughter Alice Newton. She read it and immediately demanded more; like so many children and adults after her, she was hooked. Scholastic Inc., bought the rights to Harry Potter in the United States in the spring of 1997 for $105,000. The rest is history.

Rowling’s story, which includes financial and emotional shortcomings followed by success and popularity, is the rages to riches narrative in a nutshell. It’s the story of an ordinary person, dismissed by the world, who emerges out of adversity onto the center stage. It’s the sword in the stone, it’s the ugly duckling; it’s a story that gets played out time and time again throughout history. Kafka captures it nicely in The Castle: “Though for the moment K. was wretched and looked down on, yet in an almost unimaginable and distant future he would excel everybody.”

The reality of Rowling’s story, however, is just that: it’s a story. It’s a sequence of facts strung together by an artificial narrative. It didn’t necessarily have to have a happy ending and it certainly was not predictable back in 1993. Rowling did not follow a predetermined path. Her life before Harry Potter was complex and convoluted, and, most importantly, luck played a significant role in her eventual success. These variables are always forgotten in hindsight.

Yet, we humans, facing limits of knowledge, to paraphrase one author, resolve the myriad of unknown events that defined Rowling’s life before Harry Potter by squeezing them into crisp commoditized ideas and packaging them to fit a warming narrative. We have, in other words, a limited ability to look at sequences of facts without weaving an explanation into them.

The same problem occurs in science. It’s always the story of invention, the tale of discovery or the history of innovation. These narratives manifest themselves in the form of a quest: A scientist is stuck on a problem, he or she is surrounded by doubt, but after years of hard work an insight prevails that changes the world forever.

In The Seven Basic Plots, Christopher Booker summarizes The Quest, which sounds as much like Darwin on the Beagle, MaGellan aboard the Trinidad or Marco Polo traveling across Asia as it does Frodo traversing Middle Earth. As Booker explains:

Far away, we learn, there is some priceless goal, worth any effort to achieve… From the moment the hero learns of this prize, the need to set out on the long hazardous journey to reach it becomes the most important thing to him in the world. Whatever perils and diversion lie in wait on the way, the story is shaped by that one overriding imperative; and the story remains unresolved until the objective has been finally, triumphantly secured.

Unfortunately, Frodo’s triumph at Mount Doom is more real than natural selection to some. Kahneman is right: “It is easier to construct a coherent story when you know little, when there are fewer pieces to fit into the puzzle.”

Our propensity to story tell is also fueled by the survivorship bias, which describes our tendency to believe that successful people possess a special property. For Steve Jobs it was his assertive leadership and vision, for Bob Dylan it was his poetry and willingness to challenge the norm and for Rowling it was her creativity and imagination. But these attributes are post-hoc explanations; there are plenty of people with Dylan’s musical and lyrical caliber who will never match his success. Likewise, many creative geniuses of Rowling’s stature will never sell tens of millions books. Luck, at the end of the day, might be the best explanation.

When trying to answer the question of what makes people successful the best response might be it’s impossible to know. Indeed, hardwork, intelligence and good genes certainly play a role. But the reality of Rowling’s story is that it is highly unlikely. Twelve out of twelve publishing houses rejected the book. In the years leading up to Harry Potter a number of things could have prevented Scholastic from purchasing the rights to her book. If it weren’t for little Alice Newton, the book may have never seen the light of day.

The true test of an explanation, as Kahneman also says, is whether it would have made the event predictable in advance. No story of Rowling’s unlikely success will meet that test, because no story can include all events that would have caused a different outcome. This being said, we will continue to explain Rowling’s story as if it was inevitable and predictable. We will always be obsessed with happy endings.

The takeaway is twofold: first, be suspicious of narratives, especially if they are charming; second, be humble about what you think it takes to be successful. There is good reason to believe that what you think is an illusion perpetuation by a narrative where everybody lives happily ever after.