Stanford Professor Justin Berg has conducted some interesting research on people's ability to evaluate creative ideas. He asked over 300 circus professionals to forecast how popular a series of circus acts would be with audiences. He then tracked their actual popularity with more than 13,000 circus audience members. What did Berg find? The people who actually created the new circus acts are not very good predictors of their success with audiences. Interestingly, managers are not very good judges of their success either. Who is a better judge than either the creators or the managers? It turns out that peers of the creators were more accurate judges. In particular, peers of the creators were much better at identifying the potential success of novel/unconventional circus acts. The research found that, "managers tended to undervalue novel ideas in favor of conventional performances." Interestingly, this finding about the lack of accuracy of managers' predictions holds despite that the fact that many managers were indeed creators themselves at one point in their careers.
This finding reminded me of the value of the "Brain Trust" at Pixar. At the animation studio, the brain trust is a group of people who gather to provide unvarnished feedback to the director of a new film. The brain trust often includes a few higher-level managers at the firm. However, the focus of the brain trust is peer-to-peer feedback. Other directors offer their constructive criticism to the person leading a particular project. The Brain Trust's success at Pixar seems to reinforce the findings of Berg's experimental research.