One day in 1966, in an incident that has since passed into computer science legend, Weizenbaum’s secretary sat down to use ELIZA. The secretary soon asked Weizenbaum to leave the room so that she could have privacy. The thought of needing to be alone to talk to a piece of software—much less one that she had seen her boss create—was astonishing, and Weizenbaum would retell the anecdote both in his book and to interviewers for the rest of his life. It reflected something novel and, Weizenbaum thought, disturbing about the relationships people might develop with machines, particularly computers. In attributing to them feelings, thoughts, and identities, people would form harmful attachments to computers, prioritizing their decision-making above human intuition and human needs. In the process, they would surrender to the larger political systems and capitalistic forces that directed technological innovation.
A refugee from Nazi Germany who found himself, as an MIT professor, uncomfortably situated at the center of the military-university-industrial establishment during the height of the Vietnam War, Weizenbaum knew of what he preached. Born in Germany in 1923, Weizenbaum fled with his parents in 1936, settling in Detroit. He studied at Wayne State University, where he helped assemble a computer in an era when such machines took up entire floors of university buildings. After college, he worked on early banking software for General Electric. In the 1960s, MIT called. Growing up first in Nazi Germany and then Detroit, Weizenbaum had developed a sense of racial politics and structural discrimination. Working for large corporations and then a Pentagon-funded university department during the Civil Rights Movement and the Vietnam War, he developed a sense that his own work was out of step with his professed beliefs. While he and his colleagues imagined profound technological possibilities, in practice, many of them did incrementalist research that greased the American war machine. The world had changed, but not for the better.
Joseph Weizenbaum in Hamburg, Germany in 1980
Wolfgang Kunz/ullstein bild/Getty Images
Nominally a computer scientist with tenure, Weizenbaum has been more accurately described as a social critic who believed that what mattered was less what computers could do—or might one day be capable of doing—than what we make them do. “If the triumph of a revolution is to be measured in terms of the profundity of the social revisions it entrained, then there has been no computer revolution,” Weizenbaum argues in Computer Power and Human Reason. Rather than dismantling the old order, computers arrived in time “to entrench and stabilize social and political structures that otherwise might have been either radically renovated or allowed to totter under the demands that were sure to be made on them. The computer, then, was used to conserve America’s social and political institutions.”
