Abandon Underperforming Projects

It seems likely that the world would be a marginally better place if people and organisations abandoned underperforming projects more often. Underperforming projects are a time suck, consuming resources which could be better allocated to other projects. If you feel uncertain about the value of a personal project or are finding it taking longer than expected, it might be worth abandoning it. If you notice a work project is struggling to reach completion or is delivering relatively poor returns, it might be worth speaking up and asking whether the project is worth it. At the very least, it might provoke some discussions about the project’s future.

Slouching towards Obsolescence before being born

Projects are not like animals. They aren’t born into the world, mature, and die by either brutal unpredicted demise or accelerating death ratchet. Instead, they are more like ecosystems, cities, or organs. A liver cell has an average age of three years yet an individual’s liver (as a system) will generally last a human lifetime. Mature ecosystems can remain relatively stable for significant periods of time (over thousands of years) despite the complete recycling of individual organisms. The oldest cities are thousands of years old, despite the average lifespan of a city occupant being less than 50 for much of that city’s existence. Sufficiently large projects are distributed across multiple humans, digital artifacts stored in forgotten codebases waiting to be revived by a project manager hoping to cut costs. Like a grain of pollen, a project can remain dormant for an extended period of time, waiting for spring.

Many underperforming projects start with a purpose, by the time they have failed their purpose fully forgotten. Some engineer proposes a system that would provide service X. The project is in the engineer’s head. Service X sounds like X’ to some excited project manager who tells some other stakeholder that we are building an X’. This important stakeholder doesn’t know what an X’ is, but something makes him think it’s a Y. He is deeply excited about Y, the project might become a priority. The project is distributed across three individuals. Soon it will be distributed across ten. The original engineer leaves or swaps to another project, the excited stakeholder is replaced by some other stakeholder. He is not particularly excited by a Y, but he is excited by the idea of working on important things. It is now a Z, or perhaps it’s an A. Nobody is really sure s what it is, they’re just sure it’s important. The project is distributed across the entire organisation.

There is, of course, a great deal of ruin in an organisation ¹, especially larger ones. An organisation can harbor many underperforming projects in hopes that they can, in time, be made seaworthy. The problem isn’t the underperforming projects themselves. It’s the emergent consensus that these projects are serving some purpose. This consensus can persist long after the project’s effective return falls below zero.

Challenging the Emergent Consensus

The conventional explanation for why organisations keep investing in underperforming projects is the sunk cost fallacy ². This might be part of the story, but it’s not clear it’s the important part of the story. Groupthink is closer to the important part of the story, but is also unsatisfactory as an explanation as it fails to tease out why groupthink persists.

One answer, hinted at above, is that we fail to kill underperforming projects is two-fold:

1. The project is distributed across N individuals within an organisation.
2. Abandoning the project requires consensus among K out of N individuals, where K is dependent on the influence of the K individuals.

The first point doesn’t explain why it’s hard to abandon a project, it explains why a project is surprisingly resilient to such attempts. In a weak sense, an individual trying to abandon a project is like a liver cell trying to opt out of being part of a liver. The fate of such a liver cell is almost certain death, but humans are more malleable and can slot into many of the different organs within a large organisation. If the project is deemed sufficiently important, the organisation can simply pay a modest switching cost and, if needed, one can pay the switching costs many times (whenever in doubt, repeat to yourself: “there is a great deal of ruin in a large organisation”). If switching fails, an organisation can turn to hiring or, worse comes to worst, let the project muddle along ³.

The second point isn’t, on its own, an explanation of why it’s hard to abandon a project, so let’s unpack it a bit. The basic problem is that an organisation is not coherent to the degree that an individual or a liver is coherent. An organisation is built out of many different parts, with each part having weakly shared objectives and representations. The poorly shared representations would not necessarily be a problem. Many neurons in my brain are specialized in detecting edges and this prevents them from effectively communicating with neurons specialized in detecting sound frequencies, but this not an impediment to my brain as a whole because these neurons are all relatively unified towards a common objective. What is a problem is that the many parts of an organisation only weakly share objectives and the lack of shared representations make it difficult to work out what the objectives should be. When the organisations needs an X, will it recognize that it needs an X? When it recognizes that it’s instead getting a Y, will it recognize that it’s not getting an X?

If only an organisation was coherent enough to recognize when its getting a Y but needing an X. To abandon a project, an organisation needs to achieve consensus that a given project is struggling. Relative to the brain or the body, organisations are large and have relatively low communication bandwidths. Once a project is sufficiently distributed, coming to the conclusion that it needs to be abandoned is difficult. Coordination is distributed across space and time. Pushes to abandon the project are derailed. Someone has a clever idea how the project might be saved. Abandoning a project often requires many things to go in the right direction and the time until having many things go in the right direction can be surprisingly long.

What might be done?

Organisations, despite all their problems, do manage to course correct. How do they do this? Some subset of individuals, those who are most effectively networked, can exert an outsized influence on the organisation’s future. These individuals can reduce the “effective size” of the organisation due to their interconnectedness and use their limited communication bandwidth to drive the organisation towards a single goal: abandoning the underperforming project.

Alternatively, organisations can be designed such that decision making is more effectively localized. This can be seen as an attempt to move from power-law connectivity to small-world connectivity.