It is surprising to most people that there could be anything remotely controversial about statistical analysis. Nevertheless appearances can be deceptive, and a fundamental disagreement exists at the very heart of the subject between so-called Classical (also known as Frequentist) and Bayesian statisticians.
Classical statistics uses techniques such as Ordinary Least Squares and Maximum Likelihood – this is the conventional type of statistics that you see in most textbooks covering estimation, regression, hypothesis testing, confidence intervals, etc. In contrast Bayesian statistics looks quite different, and this is because it is fundamentally all about modifying conditional probabilities – it uses prior distributions for unknown quantities which it then updates to posterior distributions using the laws of probability. In fact Bayesian statistics is all about probability calculations!
In essence the disagreement between Classical and Bayesian statisticians is about the answer to one simple question:
“Can a parameter (e.g. the mean of a distribution such as the mean life of a component) which is fixed but unknown be represented by a random variable?”
In other words can a quantity that has a fixed but unknown value be represented by a quantity that has a random value? In particular, if we initially have no information at all about the fixed parameter, is there a way of representing this state of knowledge (or lack of it) by assuming that the parameter instead has a “vague” (or flat) probability distribution over all of the values that it could possibly take?
What is your view on this question? Your answer determines whether you are a member of the Classical school or the Bayesian school of statistics. You may even change your mind after reading what we’ve written below!
The Classical school considers that the status of a quantity is either fixed or random (but not both) – just because we don’t know what the fixed value actually is doesn’t mean that we can “blur things” by treating a fixed value as if it were random. On the other hand the Bayesian school takes the opposite view, i.e. that it is valid to represent an unknown fixed parameter value as a random variable. Bayesian statisticians specify what is called a “prior distribution” to represent this “blurring” and then update this distribution in such a way that it remains completely consistent with the observed data (using what’s known as Bayes’ Rule).
So why are Bayesians so insistent on this point, and why don’t they just accept that “fixed” means “fixed” and not “random”? For some Bayesians it is because they interpret probabilities in terms of their own subjective “degrees of belief” – in particular their initial degrees of belief about the value of an unknown fixed parameter are represented by the prior distribution. For others it is because the type of analysis that can be done using the Bayesian approach is much more powerful. Bayesian statistics deals exclusively with probabilities, so you can do things like cost-benefit studies and use the rules of probability to answer the specific questions you are asking – you can even use it to determine the optimum decision to take in the face of the uncertainties. Classical statistics on the other hand gives you something rather short of this.
So why are Classical statisticians so insistent that the benefits of the Bayesian approach should be rejected and not simply accepted? There is no single or simple answer to this question, but an essential requirement of the Bayesian approach is the need to specify a prior distribution for the unknown parameter before analysing any data. It is for the purpose of specifying this prior distribution that subjective judgement is applied. However the Classical school points out that this subjectivism does not sit well with “the scientific method” which must be as objective as possible, and in particular must not depend on who does the experiment or who analyses the results.
If it works, why not be pragmatic and use the Bayesian approach anyway? Being realistic, some problems cannot begin to be tackled without making the sort of subjective judgements required for the Bayesian approach. Clearly the Bayesian approach is an appropriate choice in such cases. However, the greater power of the Bayesian approach comes at the high price of subjectivism. In a situation where the Classical approach is satisfactory, there is therefore little reason to adopt a subjective Bayesian approach instead of the usual, objective and conventional Classical approach.
What approach should be used in safety and reliability work – Classical or Bayesian? Our own view is as follows:
- Generally we would recommend that the Classical approach is used where possible as this is by far the more conventional and widely accepted approach. Though Classical statistics can be somewhat “clunky” in answering real questions, it is objective and therefore dependable.
- The Bayesian approach may have a role where the Classical approach could not provide adequate answers to the questions being asked. However we would warn against using the Bayesian approach (or exercise great caution in using it) in analyses which are critical to health or safety for the following reasons:
- the Classical approach is objective and inherently errs on the side of caution
- subjective judgements made in the Bayesian approach may implicitly include optimistic assumptions
In recent years the Bayesian approach has gained favour as the advantages of its greater power are recognised in many applications. Nevertheless the Achilles’ Heel of Bayesian statistics is ever-present because this weakness is created right at the outset of any analysis – i.e. the subjective prior distribution. This aspect of Bayesian statistics certainly can’t be ignored. In practice it may be easier to consider in any given situation whether this subjectivism can be validly ignored or whether subjective judgement may even be a valuable input into the analysis when the uncertainties are otherwise too large. However we would certainly recommend that you always use Bayesian statistics with caution.
We hope this comparison has thrown at least some light on the fundamental difference between the Classical and Bayesian approaches to statistical analysis, a difference that continues to divide the statistical community and provides a continuing source of controversy, debate and interest in the field of statistics. If you have a statistical analysis problem and are considering using the Bayesian approach instead of the usual Classical one, we hope that some of the points raised here will help you decide on the best way forward. If you need advice before deciding, please feel free to contact us and we will be very pleased to help if we can.
