When Science Attacks!

Two scandals rocked the sci-tech world last week. Not to put too fine a point on it, they reminded us that bad research and implementation can kill.

In South Africa, a widely used antiaircraft cannon called the Oerlikon GDF-005 suffered from what many observers believe was a computer malfunction, which killed nine soldiers and maimed 15 in a training exercise. Its computer-controlled sighting mechanism went haywire, and the gun automatically turned its barrel to face the trainees next to it, spraying bullets from magazines that it automatically reloaded until it was out of ammunition. Many compared the incident to science fiction fare like Robocop or Terminator, in which military bots turn on their masters.

In the United States, James Watson, who won the Nobel Prize for helping to discover the double-helix shape of DNA, was suspended from his administrative duties at Cold Spring Harbor Laboratory over comments he made to the London Times about how blacks are genetically hardwired with lower intelligence than that of other races. Watson has made comments like this about blacks (and women) throughout his career, but apparently this was the last straw. Reporter Charlotte Hunt-Grabbe, who says she has Watson's comments on tape, quoted him saying he's "inherently gloomy about the prospect of Africa" because "all our social policies are based on the fact that their intelligence is the same as ours -- whereas all the testing says not really." He told Hunt-Grabbe his "hope is that everyone is equal" but that "people who have to deal with black employees find this not true."

Nobody compared Watson's racism to science fiction, though one could bring up Gattaca, Brave New World, or any other genetic dystopia where DNA warlords like Watson -- whose employer controls millions in research money -- have created a world where genes are destiny.

These two very different incidents demonstrate the fallibility of science and, more importantly, how the arrogance of scientists can be horrifically destructive. The tragedy in South Africa could have been avoided if the engineers who designed that cannon had simply refused to computerize its sight. With a big gun, computer error can be far worse than human error. Any decent engineer would have known that failure in computer systems is inevitable and come to the conclusion that weapons should not be programmed to function autonomously.

Watson's remarks are another form of scientific arrogance that leads to gross and fatal mistakes. After all, Watson is hardly the first person to use genetics as a way to create false hierarchies of human beings based on "evidence" that some races and sexes are "naturally" superior to others. The history of biology as a discipline is riddled with racism and sexism. Eighteenth-century scientist Carolus Linnaeus, who invented the taxonomy of species we still use today, originally divided the species Homo sapiens into four racial subclasses: Americanus, Asiaticus, Africanus, and Europeanus. While Europeanus was "inventive," Africanus was "negligent." Even in the 20th century many geneticists endorsed the eugenics movement as a way to keep the species strong by preventing "dysgenic," racially mixed babies from being born.

Today leaders in the field of evolutionary biology like Steven Pinker and E.O. Wilson routinely say that people are hardwired to behave in certain ways based on their genetic heritage, which is often linked to their racial background or sex. "Scientific" studies on the genetic inferiority of female intelligence are what motivated former Harvard president Lawrence Summers to claim that there are so few women in science because they just aren't smart enough.

So should a computerized gun run amok and a racist geneticist undermine our faith in science? Yes. People who build autonomous weapons systems know their work might kill people, but they do it anyway. And people like Watson derail brilliant research by bringing sex and race bias into the lab. Science is nothing more than the sum of what scientists do. Without ethics, science is no better than Christianity during the Crusades, a dogma that kills out of arrogance and prejudice.