Wednesday, 30 May 2012

Statistics written by the winners

Survivorship bias is the tendency to draw incorrect conclusions from survivors in a group, because those that didn't survive are no longer in it. Studies of the performance of investment funds often ignore funds that have closed but a common reason for closing a fund is that it has performed badly, so the performance reported by the studies of funds in general is inflated.

Apparently, the term was coined when the stakes were even higher. During WW2, engineers studied British bombers that returned from missions to determine why so many weren't making it home. They decided to reinforce the areas most ridden with bullet holes but that didn't work, so they assumed the weight of the new armour had affected the handling of the planes and negated the advantage.

But mathematician Abraham Wald suggested the engineers armour the areas of the bombers that had not been holed, after all the bullet holes they had examined had not brought the planes down so the critical areas would be where there were no holes.