Project Reporting Without Survivorship Bias

ZALBASIREPPM · Decision-Ready Reporting

Project Reporting Without Survivorship Bias

The projects visible on a dashboard may not represent the projects management needs to understand.

Survivorship bias occurs when analysis focuses on the cases that remain visible after a selection process and overlooks those that disappeared. A portfolio report can look successful because cancelled, rejected, delayed, failed or unreported work has silently left the denominator.

01 · The Statistical Lesson

Study What Is Missing, Not Only What Returned

During the Second World War, Abraham Wald’s work with the Statistical Research Group examined aircraft vulnerability using damage observed on surviving aircraft. The essential lesson is that the observed sample excluded aircraft that did not return. Areas with little damage among survivors could therefore indicate vulnerability, not safety.

The popular aircraft story is often simplified. The reporting principle remains powerful: evidence from survivors cannot describe the full population unless the selection mechanism and missing cases are understood.

02 · Hidden Project Populations

What Conventional Dashboards Commonly Exclude

Rejected Proposals

Ideas screened out before approval, including their cost, rationale and recurring weaknesses.

Cancelled Projects

Work stopped before completion, often removed from current performance reporting.

Paused Or Deferred Work

Initiatives waiting for funding, capacity, approval or changed strategic conditions.

Failed Bids

Opportunities not won, abandoned or priced out, with lessons relevant to selection and capacity.

Incomplete Reporting

Projects that submit late, weak or optimistic data and therefore appear less severe than transparent projects.

Post-Completion Outcomes

Benefits, defects, claims and operational consequences that occur after the project leaves the active dashboard.

03 · Reporting By Exception

Necessary, But Not Sufficient

Reporting by exception helps executives focus on deviations, risks, decisions and actions. It does not automatically correct selection bias. If thresholds, populations or data-quality rules exclude the most important missing cases, an exception dashboard can become a highly efficient presentation of incomplete evidence.

Define the complete population and why each item enters or leaves it.
Preserve rejected, cancelled, paused and completed records with their final outcomes.
Show the denominator behind every percentage and trend.
Report missing, late and low-confidence data as exceptions.
Keep threshold changes and classification changes traceable.
Compare like-for-like cohorts by approval date, stage, risk, size or delivery model.

04 · Executive Questions

Challenge The Selection Mechanism

Which projects or opportunities are absent from this view—and why?

Did the denominator change since the last report?

Are poor performers being cancelled, reclassified or rebaselined out of the trend?

Which teams have not reported, and is their risk higher?

Do completed projects still deliver the benefits used to justify them?

What evidence would contradict the current conclusion?

05 · Anti-Bias Measures

Make Selection And Attrition Visible

Pipeline Conversion

Ideas, proposals, approvals, starts, completions, cancellations and realized benefits by cohort.

Attrition Reasons

Why work was rejected, deferred, cancelled or stopped—and when the decision occurred.

Data Completeness

Coverage, timeliness, validation status and confidence by project and reporting period.

Baseline Churn

How often cost, schedule or scope references change and the effect on reported success.

Post-Completion Value

Benefits, operating performance, defects and unresolved obligations after delivery.

Decision Outcomes

Whether interventions were implemented and produced the expected effect.

06 · Decision-Ready Structure

Lead With Exceptions, Preserve The Full Evidence

An executive report can begin with material problems, proposed responses and decisions required. It should then show accountable actions, forecast effects, confidence and the complete portfolio context. Summary does not mean deletion: readers must be able to trace every conclusion to the included population, source data and excluded cases.

Balanced reporting: exceptions deserve attention, but achievements and stable performance also matter when they reveal reusable practices. Avoid replacing survivorship bias with failure-only bias.

Management Conclusion

The Missing Projects May Hold The Most Important Lesson

Good reporting does not merely summarize what remains. It preserves the history of selection, attrition and outcomes; shows missing data; keeps denominators stable; and makes exceptions actionable. Decision credibility depends as much on what the report reveals about absence as on what it displays.