The Coming Technological Singularity by Vernor Vinge (novels for teenagers .TXT) 📕

would benefit from a closer connection with biological life. Instead of simply trying to model and understand biological life with computers, research could be directed toward the creation of composite systems that rely on biological life for guidance or for the providing features we don’t understand well enough yet to implement in hardware. A long-time dream of science-fiction has been direct brain to computer interfaces [2] [29]. In fact, there is concrete work that can be done (and is being done) in this area:

o Limb prosthetics is a topic of direct commercial applicability.

Nerve to silicon transducers can be made [14]. This is an

exciting, near-term step toward direct communication.

o Direct links into brains seem feasible, if the bit rate is

low: given human learning flexibility, the actual brain neuron

targets might not have to be precisely selected. Even 100 bits

per second would be of great use to stroke victims who would

otherwise be confined to menu-driven interfaces.

o Plugging in to the optic trunk has the potential for bandwidths

of 1 Mbit/second or so. But for this, we need to know the

fine-scale architecture of vision, and we need to place an

enormous web of electrodes with exquisite precision. If we want

our high bandwidth connection to be in addition to what paths

are already present in the brain, the problem becomes vastly more

intractable. Just sticking a grid of high-bandwidth receivers

into a brain certainly won’t do it. But suppose that the

high-bandwidth grid were present while the brain structure was

actually setting up, as the embryo develops. That suggests:

o Animal embryo experiments. I wouldn’t expect any IA success

in the first years of such research, but giving developing brains

access to complex simulated neural structures might be very

interesting to the people who study how the embryonic brain

develops. In the long run, such experiments might produce

animals with additional sense paths and interesting intellectual

abilities.

Originally, I had hoped that this discussion of IA would yield some clearly safer approaches to the Singularity. (After all, IA allows our participation in a kind of transcendance.) Alas, looking back over these IA proposals, about all I am sure of is that they should be considered, that they may give us more options. But as for safety … well, some of the suggestions are a little scarey on their face. One of my informal reviewers pointed out that IA for individual humans creates a rather sinister elite. We humans have millions of years of evolutionary baggage that makes us regard competition in a deadly light. Much of that deadliness may not be necessary in today’s world, one where losers take on the winners’ tricks and are coopted into the winners’ enterprises. A creature that was built de novo might possibly be a much more benign entity than one with a kernel based on fang and talon. And even the egalitarian view of an Internet that wakes up along with all mankind can be viewed as a nightmare [26].

 

The problem is not simply that the Singularity represents the passing of humankind from center stage, but that it contradicts our most deeply held notions of being. I think a closer look at the notion of strong superhumanity can show why that is.

 

Strong Superhumanity and the Best We Can Ask for

 

Suppose we could tailor the Singularity. Suppose we could attain our most extravagant hopes. What then would we ask for: That humans themselves would become their own successors, that whatever injustice occurs would be tempered by our knowledge of our roots. For those who remained unaltered, the goal would be benign treatment (perhaps even giving the stay-behinds the appearance of being masters of godlike slaves). It could be a golden age that also involved progress (overleaping Stent’s barrier). Immortality (or at least a lifetime as long as we can make the universe survive [10] [4]) would be achievable.

 

But in this brightest and kindest world, the philosophical problems themselves become intimidating. A mind that stays at the same capacity cannot live forever; after a few thousand years it would look more like a repeating tape loop than a person. (The most chilling picture I have seen of this is in [18].) To live indefinitely long, the mind itself must grow … and when it becomes great enough, and looks back … what fellow-feeling can it have with the soul that it was originally? Certainly the later being would be everything the original was, but so much vastly more. And so even for the individual, the Cairns-Smith or Lynn Margulis notion of new life growing incrementally out of the old must still be valid.

 

This “problem” about immortality comes up in much more direct ways. The notion of ego and selfawareness has been the bedrock of the hardheaded rationalism of the last few centuries. Yet now the notion of selfawareness is under attack from the Artificial Intelligence people (“selfawareness and other delusions”). Intelligence Amplification undercuts our concept of ego from another direction. The post-Singularity world will involve extremely high-bandwidth networking. A central feature of strongly superhuman entities will likely be their ability to communicate at variable bandwidths, including ones far higher than speech or written messages. What happens when pieces of ego can be copied and merged, when the size of a selfawareness can grow or shrink to fit the nature of the problems under consideration? These are essential features of strong superhumanity and the Singularity. Thinking about them, one begins to feel how essentially strange and different the Post-Human era will be — no matter how cleverly and benignly it is brought to be.

 

From one angle, the vision fits many of our happiest dreams: a time unending, where we can truly know one another and understand the deepest mysteries. From another angle, it’s a lot like the worst-case scenario I imagined earlier in this paper.

 

Which is the valid viewpoint? In fact, I think the new era is simply too different to fit into the classical frame of good and evil. That frame is based on the idea of isolated, immutable minds connected by tenuous, low-bandwith links. But the post-Singularity world does fit with the larger tradition of change and cooperation that started long ago (perhaps even before the rise of biological life). I think there are notions of ethics that would apply in such an era. Research into IA and high-bandwidth communications should improve this understanding. I see just the glimmerings of this now [32]. There is Good’s Meta-Golden Rule; perhaps there are rules for distinguishing self from others on the basis of bandwidth of connection. And while mind and self will be vastly more labile than in the past, much of what we value (knowledge, memory, thought) need never be lost. I think Freeman Dyson has it right when he says [9]: “God is what mind becomes when it has passed beyond the scale of our comprehension.”

 

[I wish to thank John Carroll of San Diego State University and Howard Davidson of Sun Microsystems for discussing the draft version of this paper with me.]

Annotated Sources [and an occasional plea for bibliographical help]

[1] Alfve’n, Hannes, writing as Olof Johanneson, The End of Man?, Award Books, 1969 earlier published as “The Tale of the Big Computer”, Coward-McCann, translated from a book copyright 1966 Albert Bonniers Forlag AB with English translation copyright 1966 by Victor Gollanz, Ltd.

[2] Anderson, Poul, “Kings Who Die”, If, March 1962, p8-36. Reprinted in Seven Conquests, Poul Anderson, MacMillan Co., 1969.

[3] Asimov, Isaac, “Runaround”, Astounding Science Fiction, March 1942, p94. Reprinted in Robot Visions, Isaac Asimov, ROC, 1990. Asimov describes the development of his robotics stories in this book.

[4] Barrow, John D. and Frank J. Tipler, The Anthropic Cosmological Principle, Oxford University Press, 1986.

[5] Bear, Greg, “Blood Music”, Analog Science Fiction-Science Fact, June, 1983. Expanded into the novel Blood Music, Morrow, 1985.

[6] Cairns-Smith, A. G., Seven Clues to the Origin of Life, Cambridge University Press, 1985.

[7] Conrad, Michael et al., “Towards an Artificial Brain”, BioSystems, vol 23, pp175-218, 1989.

[8] Drexler, K. Eric, Engines of Creation, Anchor Press/Doubleday, 1986.

[9] Dyson, Freeman, Infinite in All Directions, Harper && Row, 1988.

[10] Dyson, Freeman, “Physics and Biology in an Open Universe”, Review of Modern Physics, vol 51, pp447-460, 1979.

[11] Good, I. J., “Speculations Concerning the First Ultraintelligent Machine”, in Advances in Computers, vol 6, Franz L. Alt and Morris Rubinoff, eds, pp31-88, 1965, Academic Press.

[12] Good, I. J., [Help! I can’t find the source of Good’s Meta-Golden Rule, though I have the clear recollection of hearing about it sometime in the 1960s. Through the help of the net, I have found pointers to a number of related items. G. Harry Stine and Andrew Haley have written about metalaw as it might relate to extraterrestrials: G. Harry Stine, “How to Get along with Extraterrestrials … or Your Neighbor”, Analog Science Fact-Science Fiction, February, 1980, p39-47.]

[13] Herbert, Frank, Dune, Berkley Books, 1985. However, this novel was serialized in Analog Science Fiction-Science Fact in the 1960s.

[14] Kovacs, G. T. A. et al., “Regeneration Microelectrode Array for Peripheral Nerve Recording and Stimulation”, IEEE Transactions on Biomedical Engineering, v 39, n 9, pp 893-902.

[15] Margulis, Lynn and Dorion Sagan, Microcosmos, Four Billion Years of Evolution from Our Microbial Ancestors, Summit Books, 1986.

[16] Minsky, Marvin, Society of Mind, Simon and Schuster, 1985.

[17] Moravec, Hans, Mind Children, Harvard University Press, 1988.

[18] Niven, Larry, “The Ethics of Madness”, If, April 1967, pp82-108. Reprinted in Neutron Star, Larry Niven, Ballantine Books, 1968.

[19] Penrose, Roger, The Emperor’s New Mind, Oxford University Press,

1989.

[20] Platt, Charles, Private Communication.

[21] Rasmussen, S. et al., “Computational Connectionism within Neurons: a Model of Cytoskeletal Automata Subserving Neural Networks”, in Emergent Computation, Stephanie Forrest, ed., pp428-449, MIT Press, 1991.

[22] Searle, John R., “Minds, Brains, and Programs”, in The Behavioral and Brain Sciences, vol 3, Cambridge University Press, 1980. The essay is reprinted in The Mind’s I, edited by Douglas R. Hofstadter and Daniel C. Dennett, Basic Books, 1981 (my source for this reference). This reprinting contains an excellent critique of the Searle essay.

[23] Sims, Karl, “Interactive Evolution of Dynamical Systems”, Thinking Machines Corporation, Technical Report Series (published in Toward a Practice of Autonomous Systems: Proceedings of the First European Conference on Artificial Life, Paris, MIT Press, December 1991.

[24] Stapledon, Olaf, The Starmaker, Berkley Books, 1961 (but from the date on forward, probably written before 1937).

[25] Stent, Gunther S., The Coming of the Golden Age: A View of the End of Progress, The Natural History Press, 1969.

[26] Swanwick Michael, Vacuum Flowers, serialized in Isaac Asimov’s Science Fiction Magazine, December(?) 1986 - February 1987. Republished by Ace Books, 1988.

[27] Thearling, Kurt, “How We Will Build a Machine that Thinks”, a workshop at Thinking Machines Corporation, August 24-26, 1992. Personal Communication.

[28] Ulam, S., Tribute to John von Neumann, Bulletin of the American Mathematical Society, vol 64, nr 3, part 2, May 1958, pp1-49.

[29] Vinge, Vernor, “Bookworm, Run!”, Analog, March 1966, pp8-40. Reprinted in True Names and Other Dangers, Vernor Vinge, Baen Books, 1987.

[30] Vinge, Vernor, “True Names”, Binary Star Number 5, Dell, 1981. Reprinted in True Names and Other Dangers, Vernor Vinge, Baen Books, 1987.

[31] Vinge, Vernor, First Word, Omni, January 1983, p10.

[32] Vinge, Vernor, To Appear [ :-) ].