L’Aquila Earthquake: Science collides with the law…

22 10 2012

Today we heard that a group of seismologists have been jailed over the advice they gave prior to the 2009 quake in L’Aquila in Italy which killed over 300 people.

Is this good and right?

Well let’s start by imagining you lived in l’Aquila and a series of tremors had the town worried. A group of experts comes to town, and after a long meeting, come out and declare the risk is miminal. So rather than evacuating, you go home, rest assured, only to lose your family two days later.

I would want blood. I would want to lash out. My family could have been saved!

============

So at first glance this judgement may seem like a last sad ripple from a tragic event, however, it isn’t. Upon further reflection, this judgement has serious implications for the relationship between science and the law.

I have always been troubled by how a guilty verdict means ‘you did it’ even if you didn’t, turning ‘beyond a reasonable doubt’ into no doubt at all. I have previously blogged about pragmatism in law, and how you cannot be 80% guilty and thus serve 80% of a sentence.

With such pressures on the law to come to a clear conclusion, it was only a matter of time before a court was asked to decide on whether scientific advice was correct…

============

So today, when a court decided that scientists were ‘too reassuring’ to the public, my alarm bells were set ringing and still haven’t stopped.

Armed with the hindsight that an earthquake in fact did occur, it is natural that the families of the victims are appalled by the advice they got. However, in this case it is critical to also put yourself into the shoes of the scientist – in the days before the quake.

They must have honestly doubted a quake would happen. If you had expected a quake you would surely have said so!

In their statements they confirmed a quake was possible but unlikely. So what we have to ask is this: based on the data they had, was that conclusion faulty?

To answer that you only need to ask – do all tremors lead to quakes? Well no, most don’t. So the quake was by no means likely, let alone inevitable. They were not covering up. None in the group was suppressed or censored. I can only conclude that they, after years of chasing  tremors, had come to discount the value of tremors as indicators.

=============

And today, a judge decided that their advice was not only wrong, but criminally wrong.

The problem here is that science is not a PR exercise. If scientists put spin on data, they lose credibility. If they cry wolf, they lose credibility. The only safe way to do science is to stick to the facts. The facts did not indicate an imminent earthquake. The judge does not seem to realise: scientists cannot, and do not claim to be able to, predict earthquakes.

Perhaps the judge should read Taleb’s treatise on rare events, and he’d seen that just because something is unlikely (as the scientists said) does not protect us from the possibility it may still happen.

=============

So what we have seen today is this: scientists, giving their edified analysis, have been thrown in jail. The mob are satisfied, but you should be distinctly worried.

=============

See also: Letter from the AAAS to the Italian president

Advertisements




The ref’s word is the law: pragmatism in science and the legal system

16 05 2010

The Silent Contract

If you have ever played a sport competitively, there is good chance you have experienced a blatant error by the referee. If you are anything like me, and if the game is balanced on a knife edge, there is a good chance your blood pressure shot right up and you had some kind words to say about the ref’s eyesight and quite possibly the ref’s mother too.

And if you are like me you would look back later and realise you were somewhat out of order. It was, after all, at worst an error, and no ref is perfect.

However, in those moments of blind frustration,  that rule: “the ref’s word is final” seems so wrong. In big-brother/1984 style, once the ref says it was a foul, it was a foul and unless he changes his mind within a few seconds it will be forever recorded in history as a foul. The real true events become an irrelevant fantasy.

This policy clearly does not serve the ‘truth’, so why has it developed in so many sports?

Most footballers understand why – and most judges understand why.

Let us consider what would happen if the ref’s word were not final.

Imagine a system of democratic debate and judgement over all close-calls at a sports event. Recall that all contenders and most of the audience are highly partisan. It would not be football, it would be a debate. It is obvious you need someone impartial, and you need something quick.

So what actually happens is that all stakeholders enter a silent contract to accept the ref’s errors in order to keep the game lively.

Implications in Law

So we have an example where we in society are willing to accept that the truth is not the priority. I am OK with this on the football field, but this type of pragmatism is actually used every day in the legal system, which is rather scary to me.

There is a lot in common between a ref and a judge, or indeed a jury. It falls on them to decide the official version of truth based upon the available evidence.

However, because the consequences of legal judgement are routinely more extreme than a free-kick, a more considered system usually develops. In most western courts, there is now a ‘threshold’ (called ‘reasonable doubt’ in the US) beyond which ‘the probable” becomes ‘the fact’. If this sounds rather crude, its because it is – they are saying that after they look at the evidence, and place the judgement on a scale of probability (0-100%) that they will set a number, above which the person is guilty and below which they are not – giving our grey world a little more contrast.

If they set the threshold at 100%, to ensure that no innocent man is ever convicted, you would convict no one. If they lower it to get more practical conviction rates, they will soon start to convict the innocent. Thus a compromise needs to be reached – the exact level will be a moving feast, but will generally reflect the culture and the Zeitgeist.

The Alternative…

Who else has has to make a call about truth?

Scientists are also trying to discern ‘truth’ – about magnetism, about cellular function, about  black holes, well about most everything really.

Yes, they do use pragmatism – we, for example, continue to use Newton’s Laws in most calculations because we know they work well enough for most practical purposes. We also continue to build on theories that have inherent contradictions (meaning that are probably flawed), because history has shown that such pragmatism still  moves us forward and is better than getting ‘stuck’. Yes, it does lead to some waste, some researcher’s entire lives are built on earlier mistakes that were ‘pragmatically’ ignored (think of homeopathy).

However, in the end, the scientist has to keep record of all pragmatism and return to it and root it out, because in science, the truth is the target, and no compromise can be left on the books.

This raises an interesting tension when a scientist is brought into legal proceedings to provide ‘opinion’. Judges (and politicians) mistakenly think that science deals in facts (not probabilities), and would also ask a scientist to make a pragmatic call ‘yes, this is the suspect’s DNA’. A good scientist would leave judgement to the judge and sprinkle their statements liberally with the word ‘probability’.

But can a legal system make judgements with probabilities? Without thresholds and verdicts? Could we dole out punishment in proportion to the probability of guilt? Could we punish several people for one crime if we know one of them did it? I don’t know if we could.

Has such a system cropped up anywhere? I don’t know if it has.

Open Verdict

So science can leave matters undecided on the basis that more evidence may come, but science too, from time to time, may benefit from a little pragmatism – because just as in the law-courts, we have the power to set potential killers loose with our inability to be decisive. Think of  two words: ‘global’ and ‘warming’.