Reporting By Survivorship Bias

#MidYearReport  #SurvivorshipBias #SelectionBias #ReportByException

#ReportWhatMatters

As the mid-year report review for your project’s approaches, consider the type of project reports you are reviewing. Do they include any Survivorship Bias understanding and reporting?

Survivorship bias or survival bias is the logical error of concentrating on entities that passed a selection process while overlooking those that did not. This can lead to incorrect conclusions because of incomplete data.

These biases involve a reversed understanding of information, leading to incorrect decisions due to how information is perceived.

Here’s an important story related to this concept: During World War II, the British noticed that planes returning from combat missions over German territory suffered heavy damage from enemy fire. Few planes returned unscathed, while most had bullet holes in various areas.

Aeronautical engineers aimed to reduce these losses. Initially, they considered reinforcing the entire plane with heavy steel armour to protect against bullets and shells. However, the weight of full armour would hinder the plane’s agility.

Given their limited capabilities and lack of precise scientific tools to identify weak areas needing reinforcement, they had to rely on logical reasoning.

They observed that the planes returning were heavily hit in the wings and fuselage, but the cockpit and tail were rarely damaged. Therefore, they concluded that reinforcing the wings and fuselage was necessary because those areas seemed vulnerable.

New planes were reinforced accordingly, and engineers eagerly awaited the results. But here came the surprise: the overall survival rate didn’t improve significantly. Most planes still crashed, and only a few returned safely.

So, what went wrong?

The military decided to consult experts outside their field, and they chose the mathematician scientist Abraham Wald who turned the tables on them!

He explained that they had inadvertently used reverse logic. The planes they examined—those returning with bullet holes in the wings and fuselage—were precisely the ones that survived. These planes had absorbed the damage and made it back. Conversely, no surviving plane had bullet holes in the cockpit or tail. This indicated that those areas were the true vulnerabilities and needed reinforcement.

When they tested reinforcing the undamaged areas, the survival rate significantly increased. The principle was then applied to other weapons in the military.

This story became a model for what we now call “Selection Bias” and specifically “Survivorship Bias.”

Now, which type of report are you reading? The change is straightforward. Report By Exception.

For project exclusive summary, focus on the problems, solutions, and decisions required from all project stakeholders. Follow this with a section breaking down tasks for solving issues and implementing changes. Finally, present the overall project results as we know them usually presented.

This approach ensures that essential information is discussed without wasting time on reading results that no longer impact the ground reality.