Criticality Shortages
How bad ideas go unchallenged
One view of social media is of an evolutionary algorithm that creates and refines compelling packaging for digestible ideas1. Much is made of the threat of misinformation, but with tools like community notes becoming more widespread, the perhaps more insidious problem is of true but unsophisticated or incomplete ideas grabbing our imaginations. When we share and have those ideas reinforced by those who share similar feeds to us, we’re left with the impression that only unreasonable people that don’t share our fundamental values could possibly disagree.
Echo-chambers and groupthink have always been with us, but our societies developed sophisticated structures that allowed individuals to point out a fatal flaw in the plan without risking their social standing2. As the world changes ever more rapidly, many of these antibodies have eroded away or become obsolete. Meanwhile bad ideas are only coming at us ever more rapidly. Conscious effort is needed to rebuild criticality within our institutions so they can meet the challenges of the 21st Century without collapsing under the weight of their own contradictions.
We’re social animals, and we pay an extraordinary cost of time and energy to maintain group cohesion and social status. As infants, much of our attention is devoted to learning how to read minute shades of meaning and intention in other people’s faces. The areas of our brain associated with empathy and social cognition are enormously larger and more connected than even our cousins, the chimpanzees. Even into adulthood, some estimates suggest as much as 60% of conversation time is spent discussing relationships, personal experiences, and gossip. We’re hardwired for it; the people who weren’t were outcompeted by those who were. The latter’s groups were better coordinated, more cohesive, and had more individuals willing to make greater sacrifices for the group.
All of this to say that it's perfectly natural that most people, in most settings, will prioritise maintaining relationships with their peers rather than risk social discord by publicly disagreeing with them. Few careers are made by contradicting one’s boss. Contrarians are rarely seen as ‘team players’. Being argumentative is often seen, rightly, as being ego-driven and unhelpful.
Of course, groupthink is not a universal law. Effective organisations manage to chart a middle course between the cohesive and the critical. The right conditions can enable relationships and shared purpose to be maintained alongside challenges to ideas and direction. Four key factors - trust, culture, professionalism, and structural incentives - do most of the heavy lifting.
Trust
Most criticality is enabled by trust built on long-standing personal relationships. If you can trust that my criticism is constructive and not a personal attack, and I can trust that what I say won’t be used against me, then speaking up is easy.
One unfortunate consequence of today's unprecedented labour mobility is significantly shorter tenures within an institution. As I've discussed previously, this brings advantages for knowledge transmission and learning skills, but also makes systems reliant on trust built through long association much more vulnerable to bad ideas.
Professionalism
Professional institutions can help give a sense of a higher calling than the immediate social dynamics of their institution, and the social license to express it. ‘As an Engineer/Lawyer, I must advise you…’ converts dissent into a duty. The critique becomes a discharge of an obligation, rather than anything personal. Admittedly, the system works best for longer-established professions where this obligation is broadly understood and expected by the general public.
In some ways, professional systems only pass on the problem. Professional bodies themselves can become uncritical and promote ideas misaligned with the public interest, or even make membership contingent on adherence to political ideologies unrelated to the core purpose of the institution. No system is better than the people running it.
Culture
Occasionally, an organisation can develop a culture that rewards and reinforces criticality. Typically, this comes about from a strong selection for unusually disagreeable and truth-seeking individuals. Some of the most famous research institutions, like ITRI, PARC, or the Royal Society in its heyday, were full of these sorts of people when they produced world-changing research and technologies. As you might expect, these groups of these sorts of people tend to be highly volatile. Further, they’re vulnerable to takeover by more agreeable individuals better able to climb corporate ladders. Institutions like these are rare, and for each there are dozens with huge endowments that failed to deliver.
Operating like a high-pressure research organisation is probably not a realistic (or desirable) goal for most organisations3. However, a culture that expects ideas to be challenged can be curated. Leadership needs to set the tone by deliberately making themselves vulnerable to criticism and rewarding those who do so constructively. Unfortunately, current HR practices tend towards discouraging such a culture. Usually, the stated objective will be a variation of ‘helping everybody feel safe to speak up and contribute’. However, without recognising that criticality and comfort are in tension, cultural policies invariably reinforce the latter at the expense of the former.
Incentives
‘Show me the incentives, and I’ll show you the outcome’
-Charlie Munger
Providing employees with company stock theoretically aligns the employees’ interests with the long-term financial success of the company. In practice, below the level of senior executives, the benefits of speaking critically about suboptimal decisions are far outweighed by the social and financial risks of antagonising the boss. In large mature companies, future salary progression will always be significantly more valuable to mid-career employees than any bump to net worth their actions could reasonably bring about4. Prioritising social relationships over minor improvements to company performance is not only natural but also financially rational.
People tend to be much more motivated by downside risk than upside reward. As a result, employees owning stock are much less likely to become whistleblowers after discovering serious fraud, as doing so would significantly affect their net worth. We’ve seen this dynamic play out dramatically with the failures of the 737 MAX at Boeing. Employees would gossip about mounting problems internally, but neither took their concerns public, nor risked their career progression by ‘rocking the boat’ and taking the extraordinary action necessary to rectify systemic issues.
Frankly, I’m not sure how to robustly deal with this problem. Individuals may be able to take local action in their domain, but the tide is flowing out. Society is increasingly Balkanised into distinct and self-reinforcing information environments. Modernity is too complex. The algorithm helpfully provides us with facts and ideas that conform to our preconceived ideas and interests5. Constructive criticism and compromise are impossible without a common frame of reference.
After decades of relative peace and stability, we’ve come to the end of the end of history. Change can be embraced, or it will be forced upon us either by the electorate or shifting geopolitics. My concern is that reform is not only done well, but also in such a way that affirms and reinforces the unfashionable and increasingly forgotten principles that have brought us this far - liberty, pluralism, and the rule of law. Groupthink simply won’t get us there.
The best ideas and packaged and repackaged a hundred times by different users on different platforms. Sophisticated users A/B test different templates, captions, thumbnails, and conceptual framings and observe which score the most views. The posts that go viral are not wild left-field ideas that challenge our preconceptions, but highly refined recombinations of our prior beliefs, views, and algorithmically identified interests.
Examples from the sycophantic Trump Administration that is undermining its own support at a record rate due to poor internal critique of bad judgment are left as an exercise for the reader.
Xerox PARC invented many of the technologies of the modern age, but the high pressure and anti-social environment also had its consequences. According to Michael Hiltzik’s Dealers of Lightning: Xerox PARC and the Dawn of the Computer Age, in its 70s heyday PARC had a 25% annual staff turnover rate.
Consider an employee on $150,000/year at a mid-sized public company of 1000 employees. Playing the game might result in an average 5% pay rise per year, leaving them on $191,000 after five years. If the same employee also had $40,000 in restricted stock with the company, then even extraordinary efforts that result in an additional 25% growth [i.e. essentially impossible for an individual employee to achieve in a diversified mature company] would only result in a one-time ‘bonus’ of $10,000. Saving the company hundreds of millions at the expense of career growth is against self-interest.
One of the key advantages startups have is a reversal of this reward structure. For early employees, the success or failure of the company can far outweigh the impact of any future salary gain. Further, the impact of any given decision on the future value of the company is potentially enormous. This creates a large incentive for people to speak up and be critical of decisions they think are wrong, and equally large incentives for decision-makers to put their egos aside and listen.