Essay on chance and determinism

      David Steinsaltz

An acquaintance who grew up in the Soviet Union said, "We had a joke 
then: the future is laid out in the five-year plan, but with the party 
perpetually rewriting history, the past is always unpredictable."  Why is 
this funny?  If this were George Lakoff's talk, he might tell you about how 
most languages, such as English, spatialize time with the future in front 
of us and the past behind, but a few do it the other way around, for a blindingly 
simple reason: the future is behind us because we can't see into it.  The 
past lies open to our eyes.  But I'm here to talk about chance and 
determinism.

Science has become associated in the popular imagination with a kind of 
na•ve physical determinism, which arose in the first flush of enthusiasm 
for Newton's celestial mechanics, and collapsed (within science) soon 
after.  From the point of view of modern (or even nineteenth-century) 
physics, as I will explain, the future is as much a mystery as intuition 
tells us it should be.  From the point of view of paleontology, 
history, or thermodynamics, the world looks a bit like in that Soviet 
joke.  If we have a bar of warm iron, the heat equation tells us exactly 
what the heat distribution will be at any time in the future, but 
information about the past is constantly being lost.  The earth's crust and 
the government archives are churning constantly, making the reconstruction 
of the past ever more an imaginative endeavor, and ever less a matter of 
calculable fact.

In my perusal of the philosophical literature, I have come upon six different 
notions of determinism.  The first four I will mention only to exclude 
them from my comments, though someone else may wish to take them up.  
"Logical determinism" traces its roots to Aristotle.  This is usually 
summarized by the Latin phrase *tertium non datur* --- there is no third 
thing.  Something either happens, or it doesn't.  If George W. Bush had 
gotten drunk at his father's inauguration and said, "I betcha I'm gonna be president 
too, someday," then he was right, and the smart alecks who laughed were 
wrong.  A true statement is true for all time, even if no one can know, and 
so the world cannot be other than it is.

A slight variant, more acceptable to the medieval scholastics, is termed 
"theological determinism".  According to this line of thought, it doesn't 
matter what George W. asserted while in his cups, but what God said, or 
thought.  And, since God is thinking all the time, and he thinks about the 
future all the time, and can't be wrong, well, somehow the future must 
already be determined now.  God certainly knew that GWB would become 
president, so it must have been decided before Adam and Eve were expelled 
from paradise.

"Ethical determinism" traces its pedigree to Socrates and Plato.  This is 
a much more limited theory.  It's not saying that everything in the world, 
from the fall of Rome to the fall of a maple leaf, is precisely determined 
beforehand.  This is a theory of psychological determinism.  Simply put, 
the argument is, that no one can intentionally choose what is bad over what 
is good, so people are compelled to do what is good.  Of course, this 
theory does allow for errors.  And what one person calls an error, another 
might call a disagreement.

"Historical determinism" may best be illustrated by Tolstoy's great novel, 
whose title, Voyna i Mir, it has been pointed out, is conventionally 
mistranslated into English as "War and Peace".  (Voyna is war, no 
question, and there is a good deal of that in the book.  The problem is, 
there is no peace.  But "Mir" is a slippery word in Russian, with meanings 
that range from peace to world to society.  Quite likely, Tolstoy intended 
with his title something more like "War and Society".)  Vast armies range 
over the continent, apparently directed by the dictator Napoleon.  
Nevertheless, Tolstoy assures us, Napoleon is himself only a feeble pawn, 
pushed about by forces irresistible and impersonal.

Finally, the two topics I want to discuss at greater length: physical and
statistical determinism.  Physical determinism, of the Newtonian
persuasion, received its canonical statement in the writings of Pierre
Simon de Laplace: "An intellect which at any given moment knew all the
forces that animate Nature and the mutual positions of the beings that
comprise it, if this intellect were vast enough to submit its data to
analysis, could condense into a single formula the movement of the greatest
bodies of the universe and that of the lightest atom: for such an intellect
nothing could be uncertain; and the future just like the past would be
present before our eyes."

Before I go down that road, let me address what is probably the most vexed 
issue connected with physical determinism, namely, the so-called "paradox of 
free will".  I say "so-called" because, while it may once have been a 
genuine paradox, I think that philosophical progress has blunted the 
apparent contradictions, to the point where it is now a pretend paradox, 
kept alive solely for the purpose of protecting some lazy habits of 
thought.

Here is the argument in a nutshell: little Kenny stole candy from the supermarket. 
His lawyer points out to the court that the boy's body (and brain) are a system 
of Newtonian particles which were
set on a course which inevitably would have reacted to certain stimuli with 
a certain sly pinching motion.  Thus, the law should treat him as 
someone acting under a compulsion, not responsible for is actions, hence 
not subject to punishment.  You might want to lay the blame on his parents, 
but they are not responsible either.  So Kenny gets off scot free, and the 
next thing you know he's stealing cars, robbing banks, and finally becomes 
the CEO of a major energy-trading company.  Not a pretty story.  But what 
could we have done?

A weaker version, which might be called "psychological determinism", says 
that perhaps a future psychology would be able to analyze the boy and say, 
he is of psychological type NB097/Q.  That type, when sent into a store just like this one, with 
no money in its pocket, invariably steals candy.  But it's not his fault 
that he has psychological type NBO97/Q.  This is a consequence of genes and 
environment.

Now, most people are inclined at this point get a sour look on their faces,
and may be inclined to quote Dickens: "If the law supposes that, the law is
a ass."  Or, since the law never does suppose anything quite so
impractical, they may wish to cite Berkeley, to the effect that "The
philosophers throw up a dust and complain that they cannot see."  But the
philosophers are rarely just making a fuss to hear themselves talk.  To
properly lay this paradox to rest would force us to think more clearly than
we are ordinarily wont to do about such matters as justice, moral
responsibility, child rearing, and the purpose of punishment.  We could 
spend many hours investigating these questions, and many more reviewing the 
fascinating history of the concept of "law", as it has been extended from 
the king's writ to principles of nature.

To give just one example, the law is generally a practical instrument, 
concerned with consequences, rather than with abstract virtues.  One of the 
acknowledged functions of punishment is deterrence
if I am hurled from a second-story window, and land on someone, who dies,
it would not make sense (from the perspective of deterrence) to prosecute
me for murder.  My action was compelled by Newtonian laws, and I could not
have deflected the fall, no matter what motivations I may have had.  This 
is, more or less, what the law means by "compulsion".  The proper issue in 
court, or in the legislature, becomes, which other cases are sufficiently 
similar in principle to 
While threat of punishment could not have restrained me from my actions he
situation is otherwise for the physical or psychological laws governing
Ken's behavior.  Surely, one of the determinants of NBO97/Q filching
tendencies is the expected sanction.  Armed guards stationed in every aisle
with orders to shoot pilferers would, one suppose, deflect the atoms in the
lad's brain in a more honest direction.  

The philosopher Daniel Dennett has 
compared the free decision-making self in psychology to the 
center-of-gravity in physics.  Both are fictions, but they are useful 
fictions, because they tell you where to apply force to achieve desired 
effects.  Recognizing that they are fictions does not force us to abandon 
them, but it does suggest to us that we may need to put them aside when 
the problem at hand depends on the fine structure.

Rather than spend hours exploring the ramifications, trying to understand 
why most people most of the time are content to make decisions on the basis 
of such a sloppy notion of compulsion and of self, I will attack this 
problem from the other end.  Laplace's notion of physical determinism came 
out of a sense of amazing power inspired by Newton's success
in bringing order to the planets.  It's a bit reminiscent of Harry 
Houdini's trick, when confronted with a particularly uncrackable vault, to 
offer instead to be placed inside, and break out.  Amazing, people 
thought, not considering that safes are not designed to be secure from the 
inside.  Similarly, it seemed that if Newton's laws explain the motions of 
the heavens, the motions of mere terrestrial bodies must be child's play.  
Simply, as Laplace pointed out, a matter of data and calculation.

Those who quote these lines rarely remark that they appear in the 
introduction to Laplace's treatise on probability theory, and that he goes 
on to explain that this divine perspective is not especially propitious 
for us mortal beings.  This he takes as a lead-in for what he considers an 
appropriate technology for coping with the complexity of the world, which 
is the calculus of probabilities.

Much has been made of late, in popular culture of "chaos theory", as a 
challenge to determinism.  It 
shows you how important a good name can be for selling a theory.  The 
butterflies-generated tornadoes are purely speculative.  The real 
point, and one that was well known to Laplace, is simply that knowing that 
you can make a complete prediction with complete information does not imply 
that you can make a partial prediction with partial information.  In fact, 
partial predictions from partial information are a difficult art.  Many 
were misled by the triumph of Newton's equations in matching reality for 
a system of one sun and one planet.  To whatever precision you could 
measure the current motions of Jupiter and the sun, for instance, you could 
compute their orbits to the same precision.

Laplace knew that this was not true in general, because he and his 
contemporaries had foundered at the very next step: the "three-body" 
problem.  It turns out that Newton's equations do not offer a closed 
solution when you increase the number of bodies to three.  The laws are not 
wrong.  Laplace's divine intellect could turn perfect information about 
current positions and momenta into a perfect prediction of the future and 
the past.  But an initial uncertainty of one part in a billion would 
compound itself in short order, until soon we would not know whether the 
bodies have collided or not.  Several generations later, Henri PoincarŽ 
initiated what has become known as chaos theory, when he showed that we 
cannot predict with certainty whether our solar system, assuming it were 
to obey Newtonian laws, would remain stable indefinitely.

One of the lazy habits of thought that I referred to earlier is what I like 
to call the "in-principle" fallacy.  One of the great gifts that Albert 
Einstein gave to physics, and to philosophy, is the understanding that our 
models of the world do not need to resolve every describable 
contradiction.  They only need to address the ones we can imagine.  Thus, 
since perfect measurement, and perfect computation are not possible, we do 
not need to trouble ourselves about what the consequences would be for our 
laws and society.  We are free to consider them nonetheless, but then we 
are really just talking about theological determinism in scientific dress.

This is especially true when we speak of human psychology.  Why do we have 
these enormous brains?  Brains acquire and integrate information, but they 
wouldn't be worth their metabolic salt if they couldn't turn that 
experience into action.  If the brain's actions could be predicted by a 
computer, on the basis of a few gigabytes of coarse information, then 
surely evolution would have dispensed with these overly complicated 
appendages.  This is not to say that we cannot find -- or create -- 
circumstances in which the human brain will be predictable, or that we 
cannot find levers of power in the brain which can compel it one way or 
another.  Thrown out the window, the human being will reliably accelerate 
at 32 feet per second squared.  But this is a case of simplifying the 
system, not of comprehending it.  We still won't be able to predict (in a 
precise, scientific way) what will occur to him on the way down.

The last form of determinism, and one that I think is a particularly modern
concern, is statistical.  Consider the following quote from the Oakland
Tribune: Friday, December 06, 2002 - "Most major Bay Area cities saw crime
rise in the first half of 2002 compared to the first half of 2001 -- with
crime in Oakland up by 28.1 percent -- and the increases were above the
statewide average, according to statistics released Thursday. " On the 
radio, I heard a few weeks ago the prediction that this Christmas shopping 
season might be the worst in a decade.  Total sales, they said, might be 
up only two percent over last year.

Setting aside the question of what economic assumptions go into claiming 
that 2 percent better than a good year makes a bad year, what kind of 
belief system makes sense of comparing crimes this year with crimes last 
year?  Or sales this year with sales last year?  The people committing 
murders this year are, by and large, different people than the ones who 
did so last year.  Many crimes are crimes of opportunity, and yet these 
fortuitous opportunities seem to present themselves at about the same rate 
from year to year.  We have grown so accustomed to this, that we seek an 
explanation only for significant changes from one year to the next.

And yet, in the late 
eighteenth and early nineteenth centuries, the regularities shook up the 
incipient social sciences in Europe.  The first observations were in the 
domain of so-called vital statistics: births, deaths, and marriages.  
Immanuel Kant looked at the actuarial tables, and wrote, "Since the free 
will of man has obvious influence on marriages, births, and deaths, they 
seem to be subject to no rule by which the number of them could be 
reckoned in advance.  Yet the annual tables of them in the major countries 
prove that they occur according to laws as stable as those of the unstable 
weather, which we likewise cannot determine in advance, but which, in the 
large, maintain the growth of plants, the flow of rivers, and other natural 
events in an unbroken, uniform course.  Individuals and even whole peoples 
think little on this.  Each, according to his own inclination follows his 
own purpose, often in opposition to others; yet each individual and people, 
as if following some guiding thread, go toward a natural but to each of 
them unknown goal; all work toward furthering it, even if they would set 
little store by it if they did know it."  John Arbuthnot, among others, was particularly 
enchanted by the discovery of a regular proportion of male to female 
births, with the male excess being exactly balanced out by higher 
mortality, so as to create a perfect balance of the sexes when they 
attained marriageable age.  Surely, it was thought, we see here the action 
of divine providence shaping the world.

But then the French republic started collecting statistics on all kinds of 
things.  Was divine providence responsible for the fact, as Laplace 
pointed out, that the Paris post office had about the same number of dead 
letters every fear?  And what about murders, suicides, and industrial 
accidents?  Henry Thomas Buckle wrote that suicide "is merely the product 
of the general condition of society...  The individual felon only carries 
into effect what is a necessary consequence of preceding circumstances.  In 
a given state of society, a certain number of persons must put an end to 
their own life.  This is the general law; and the special question as to 
who shall commit the crime depends of course upon special laws; which, 
however, in the total action, must obey the large social law to which they 
are all subordinate...  The offences of men are the result not so much of 
the vices of the individual offender as of the state of society into which 
that individual is thrown."

What does it mean?  If there seems to be a quota of suicides, filled year 
after year, to what extent is the individual suicide a product of free 
will?  There is no complete answer.  The best we can do is to give an 
analogy.  Consider flipping a coin.  Most of us would agree that the coin 
is free to come up heads or tails.  If we flip it 100 times, it is free to 
come up tails every time.  And yet, we would be justifiably astonished if 
this were to happen.  It's a strange fact, in a way, since that's just as 
likely as any other sequence of outcomes.  If someone had said, the coin 
will come up heads tails tails heads tails, and so on, and stated the 
sequence of 100 flips exactly right, that would have been, if anything, 
even more astonishing.  The point is, of the 2^100 possible outcomes, we 
have picked out a vanishingly small proportion as special.  We are correct 
to assume that none of these will occur.  It is not impossible, only 
vanishingly improbable.

Unfortunately, we have no clear idea of why this model should be applied 
to these problems at all.  Consider, for example, the following facts: In 
1998, there were 30,575 suicides in the US.  In 1999 it was 29,199.  Should 
we suppose, then, that each American has a probability about 11 in 100,000 
of doing him- or herself in, in any given year?  That can't be right.  The 
rate for men is several times higher than that for women: about 18 in 
100,000 men, but only 4 in 100,000 women.  So is there a separate 
probability for the sexes?  What about races?  White men's suicide rate 
was twice as high as that of black men.  Suicide rates increase with age, 
as well.  It seems that each individual must have his or her own 
probability.  But then we are back where we started.  What guarantees the 
near constancy of the distribution of probabilities.

Another problem is, probable and improbable have no place in Newtonian 
determinism.  Imagine that you have a box with 10000 red balls and 10000 
black.   If you pick one, there's nothing I can tell you about which one 
you'll get.  If you pick 100, though, I can tell you 
that you won't get more than 70 red.  Another way of saying this is, 
I could imagine making up a new box, a vast box, where each ball in the 
new box corresponds to a possible sequence of 100 picks from the old one.  
Let's paint them red now only if more than 70 of the picks are red. Then 
we can compute that only about one in 100 thousand in the new box will be 
red.  We reach in and take one at random.  Intuitively, we are sure to get 
a black ball.  And yet, from the point of view of physics, either the one 
or the other is certain.  The probability is in our state of knowledge.

What probability theory does for us is to give us a systematic way 
of reasoning from the little boxes to the big boxes, and to compare the 
proportions in different boxes.  Ultimately, the only prediction is that if 
you make one pick from a box which is overwhelmingly black, you're going 
to get a black ball.

This is as certain as anything in everyday life, with one caveat.  Physics 
tells us that when we drop a ball out the window, it falls down.  If we 
drop it a thousand times, it falls every time.  But when you pick from 
your giant box a thousand times, one of those times you may very well 
happen to get a red ball.  Repeating the experiment puts you in the 
situation of a bigger box.  This is a serious problem in statistics.

For example, Bob Burton told us a couple of months ago about a study of 
surgery for early-stage prostate cancer.  He pointed out then that the 
study was being proclaimed a success for surgery, because the number of 
prostate-cancer deaths was lower in the surgery group than in the control --
16 against 31 -- even though the difference was nearly balanced by a 
higher rate of death due to other causes in the surgery group: 37 against 
31.  In total, there were 53  deaths among the patients receiving surgery, 
as against 62 deaths among the controls.

The argument which the researchers appeal to implicitly runs as follows: If 
the surgery were useless, a patient dying of prostate cancer should have 
an equal chance of being in either the surgery or the control group.  The 
result is thus equivalent to flipping 47 coins, and getting, instead of the 
expected 23 or 24, only 16 tails.  How likely is that?  This is what they 
call the "p-value", and it is about 2%.  The other deaths, on the other 
hand were like 68 coin flips yielding 31 tails, which is, in fact, not at 
all unlikely.  They can then say, if the surgery was no good, then what we 
did was the equivalent of reaching blind into a box with 98 black balls and 2 
red, and pulling out a red ball.

Intuitively, we feel fairly confident that this will not happen.  And yet, 
it is far from the same thing as physical determinism.  If 100 researchers 
all happened to be doing the same experiment, and the surgery really were 
useless, we would expect a couple of them to come up with a result at least 
this convincing.  If they chose to publish, and the others did not (and 
many journals refuse to publish negative results), we would be sorely 
misled.

This may be what happened in a famous experiment by Elizabeth Targ an the 
healing power of intercessory prayer.  She divided AIDS patients into two 
groups.  The patients in one group received -- without their knowledge -- 
prayers from a rotating panel of priests, rabbis, shamans, and other 
prayer professionals.  Those in the other group were ignored.  In her 
published paper, she reported significantly lower rates of hospitalization 
and a long list of possible complications among the patients in the prayer 
group.  The problem, as it came out later, was that she and her coauthors 
had originally planned to compare the number of deaths in the two groups.  
Only after protease inhibitors intervened, reducing the death rate to near 
zero, did they cast about for other measures of success.  This is as 
though they picked ten or twenty times from the box, before getting the 
one-in-a-hundred red ball.  What is even more troubling, they diagnosed the 
patients' illnesses after the fact, and after knowing who had received the 
prayers.

This is a fundamental difference between statistical and physical 
determinism.  When CERN finds a single trace of a Higgs boson it deserves a 
Nobel prize; the meaning of the result is not tainted by the fact that 
millions of traces had to be discarded before finding this one.