Home > Articles

Bayesian Methods for Hackers: Would You Rather Lose an Arm or a Leg?

  • Print
  • + Share This
Loss functions are one of the most interesting parts of statistics, but they can be a bad thing, too. In this chapter from Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference, Cameron Davidson-Pilon discusses why “It is better to be roughly right than precisely wrong."
This chapter is from the book

5.1 Introduction

Statisticians can be a sour bunch. Instead of considering their winnings, they only measure how much they have lost. In fact, they consider their wins to be negative losses. But what’s interesting is how they measure their losses.

For example, consider the following:

  • A meteorologist is predicting the probability of a hurricane striking his city. He estimates, with 95% confidence, that the probability of it not striking is between 99% and 100%. He is very happy with his precision and advises the city that a major evacuation is unnecessary. Unfortunately, the hurricane does strike and the city is flooded.

This stylized example shows the flaw in using a pure accuracy metric to measure outcomes. Using a measure that emphasizes estimation accuracy, while an appealing and objective thing to do, misses the point of why you are even performing the statistical inference in the first place: results of inference. Furthermore, we’d like a method that stresses the importance of payoffs of decisions, not the accuracy of the estimation alone. Read puts this succinctly: “It is better to be roughly right than precisely wrong.”1

  • + Share This
  • 🔖 Save To Your Account