Vincent van Gogh lived a short, frantic, and at times tortured life. He apprenticed for an art dealer, pursued a career as a traveling missionary and evangelist, was nearly admitted to a mental asylum, and only became an artist at the tail of his life, a career that lasted for only ten years.

However, despite his short career as a painter, van Gogh had a hot streak between 1888 and 1890, during which he painted some of the most famous works in modern art history, including The Starry Night, Sunflowers, and Bedroom in Arles.

During a visit to the van Gogh Museum in Amsterdam, Dashun Wang, professor of management at the Kellogg School of Management at Northwestern University, was intrigued by this creative fury and wondered what triggered it. He knew of other examples not just in the art world, but also in scientific and cultural fields.

Years later, Wang and colleagues analyzed the career histories of thousands of personalities and uncovered a common pattern: the most successful completed their most cherished works immediately after going through a period of creative experimentation and exploration.

Creative success may have a ‘magic’ formula

Using artificial intelligence, the Northwestern researchers combed through and analyzed a massive dataset comprising over 800,000 paintings collected from museums and galleries, covering the careers of 2,128 artists. They also mined a dataset from the Internet Movie Database (IMDB), which included 79,000 films by 4,337 directors, as well as datasets from the Web of Science and Google Scholar to analyze the careers of 20,400 scientists.

The algorithms were tuned to find hot streaks, measured by auction price of sold paintings, IMDB ratings, and academic paper citations. The timing of these streaks was then analyzed in order to uncover any patterns in the individuals’ career trajectories four years before and after their most creative periods.

Across the board, an overarching pattern emerged whereby the most successful artists, film directors, and scientists engaged in episodes of exploration, straying out from the beaten path, followed by a lucrative period of exploitation. On average, a hot streak lasted only five years.

Those who only experimented and didn’t buckle up to exploit a single creative thread had a significantly lower chance of entering a hot streak. Similarly, those who only stayed in the trenches and solely employed exploitation had lower odds of achieving a hot streak.

“Neither exploration nor exploitation alone in isolation is associated with a hot streak. It’s the sequence of them together,” said Wang, who led the study. “Although exploration is considered a risk because it might not lead anywhere, it increases the likelihood of stumbling upon a great idea. By contrast, exploitation is typically viewed as a conservative strategy. If you exploit the same type of work over and over for a long period of time, it might stifle creativity. But, interestingly, exploration followed by exploitation appears to show consistent associations with the onset of hot streaks.”

Van Gogh, for example, experimented a lot during the years leading up to his 1888 hot streak. Prior to this period, the Dutch painter produced a myriad of drawings, portraits, and still-life paintings that were quite different from one another, and which were very different in particular from his works produced during the hot streak.

“We were able to identify among the first regularities underlying the onset of hot streaks, which appears universal across diverse creative domains,” Wang said. “Our findings suggest that creative strategies that balance experimentation with implementation may be especially powerful.”

“This knowledge can help individuals and organizations understand the different types of activities to engage in — such as exploring new domains or exploiting existing knowledge and competencies — and the optimal sequence to use in order to achieve the most significant impact,” added study co-author Jillian Chown, an assistant professor of management and organizations at Kellogg School.

The findings appeared in the journal Nature Communications.