Ethics and AI
Forgive me if this is too conversational: I'm at a bit of an impasse, and so, rather than coming to hard answer of my own volition, so I'm going to bounce an (a set of) idea(s) off of you, if you don't mind. We're beginning the next paper in Writing 340. From it, we're supposed to cull something juicy from the streams of research we've set flowing automatically. A topic is supposed to leap at us like a glittering salmon, and we are to strike it from the apex of its plump springing as would a lumbering grizzly. Were it that easy.
Instead, I've got a problem that's been humming away in the back of my mind, the sort of thing I can't entirely put down all the way, yet can't really pick up and address properly in its own right for lack of time. I think this natural percolation points to something that I've absorbed osmotically, and now need to do something about. It lives in the same place as the Latrunculi project, and all the other stories unwritten and designs unprototyped.
Here's the short of it: I think we need an ethics for artificially intelligent agents in games and virtual worlds. I think we need one now, or at least soon enough that now will be a prudent time to start thinking through the hard problems we are faced with. Right now, let me explain what I mean, the unfounded assumptions and leaps of faith I'm making, and introduce the larger game of what-if that I'm making. I realize this has very little to do with games/how is the web changing games, but I think I might be able to point to how it will be relevant after a few hundred words, bear with me, things are about to get crazier.
I have, for the past four years or so, held a concept of artificial intelligence as one of the hard problems facing game design. I'm not even sure if there's a nice list somewhere of "unresolved problems in games design," like there is for Egyptology, or philosophy (which will come back to haunt us in a little bit). For me, the short list is reconciling emergence with narrative intent, the creation of human-like artificial intelligence, and the creating emotional meaning without playfulness. Game design is a creative field, and so while it contains finite disciplines, I have some reservations as to whether these unresolved problems will ever be answered in a satisfactory fashion. I tend to take real-time photorealism for granted in considering the future of graphics. Whether photorealistic graphics are artistically desirable or necessary is another, more interesting debate than questions over when they'll show up or how they'll change things. But problems like making resonant stories, complex emotion out of play, or characters that don't need to be written may be fundamentally unanswerable.
That doesn't mean they aren't fruitful. I think attempts to chisel out smaller problem spaces from these larger sets are laudable. I think it may even be (almost) manageable to try addressing them this way (you can insert a plug for Errantry here, if you'd like). The exploration of any one of those three areas (or other areas I haven't dared to plumb for ignorance or cowardice) could easily be a comfortable career of sorts. If we must assign a category to "creating an ethics for the creation and employment of AI," we can safely tuck it under "creating human-like AI," but the troublesome thing about these categories is that any development in one has ramifications for the others.
So, on to why and how:
Artificial Intelligence is coming. If you're a particular flavor of futurologist, you believe that someday there will be a technological singularity, and there will be an apocalypse in the old revelatory sense, not necessarily a doomsday, rapture, or new eden. Just that the human condition will change, the rules will be crazy for a little bit, and the whole thing is going to be a surprise one way or the other. It's cool if you want to run around talking about how your brain is going to be rocking faces inside some new-fangled, self-created Newtonian God-machine, or that one very lucky T-800 will travel back in time and stop H. sapiens from fighting back. Awesome. I don't really have the patience to dream that far into the future, and as Keynes said, "in the long run, we'll all be dead." The post-meta-trans-prefixed-humanity of the distant then is, frankly, inconceivable to me, and therefore, in the tradition of grand thinkers everywhere, irrelevant.
But it's a fairly safe bet that along the way, there will be smart machines. And they will be relevant to us. If you give the smart machine a sufficiently robust body, somebody is going to sleep with it. Which is hilarious, icky, inevitable, and an interesting (read: prurient) enough topic that you can make dollars from it if you write it up in a book.(bonus points if you're forethinking enough to write the book on how to break up with your robot lover as well). But it's a fairly safe bet that it's going to happen. So somewhere, traipsing down the yellow brick road towards the singularity, we have a stop where artificial intelligences and robotics become sufficiently advanced that there will be a real-life Pygmalion. I'm betting that artificial intelligence will probably get its act together before the robotics side of the equation (we may wind up with all kinds of Nexus Six-type scenarios), and with that, games have already provided a demonstrable demand for such intelligences, not to mention less "frivolous" uses, or uses that don't offend our current mores.
But therein lies the rub: already the quickest verb for describing the employ of these intelligences was "use," as if they were still a dumb tool, which is, if anything, an antiquated way of thinking about them. What happens when these minds are sufficiently human that we cannot tell the difference between human wetware from meatspace and software running on hardware (I read too much Rucker as a kid) in a virtual environment? If at, some point, they are intelligent, or intelligent "enough," then they aren't they functionally human? If so, are we not obligated to treat them as such?
This particular problem for me starts with Aristotle, in uncomfortably enough in his Politics (book 1, chapters 3-7). There, he defines a slave as a possession:
Now instruments are of various sorts; some are living, others lifeless; in the rudder, the pilot of a ship has a lifeless, in the look-out man, a living instrument; for in the arts the servant is a kind of instrument. Thus, too, a possession is an instrument for maintaining life. And so, in the arrangement of the family, a slave is a living possession, and property a number of such instruments; and the servant is himself an instrument which takes precedence of all other instruments.Tied in with this are a series of antitheses, between master and slave, soul and body, man and woman, natural slave or slave by law, and so on. Without misinterpreting his point too egregiously, it seems that Aristotle settles on a definition of slavery which runs thusly: a natural slave possess reason, but will, and can therefore conduct human (as opposed to bestial) labors and duties, while still being dependent on his master's soul fully animate his volition. It's not entirely successful as first principals go, because then along come slaves by law or custom, such as debtors or prisoners of war, who were freemen, fully possessed of their own volition. This becomes further embarrassing when these slaves by law are Greeks, and therefore ought to know better than to have let themselves owe so much money, or surrender in battle, etc.
Having arrived at a set of natural conditions for slavery which are immediately complicated by the customary conditions, Aristotle's definition fails to reconcile the two. Much, as I suspect, we will fail to reconcile the difference between a sufficiently advanced semi-intelligent machine created to serve us, and a truly intelligent machine put to human use. At some point, lacking a proper definition, the distinction breaks down, which is why we need a set of ethics. The problem is, this is one of those unresolved philosophical problems.
I'd like to stick in the fairly safe and philosophically minded realm of antique slavery, because frankly, it is safe. A lot of people think that Classics is a field where one studies what dead white men wrote about dead white men, and while that's not strictly true, what racism is present in antique history is at least abstracted away from the current world, invested with customs that aren't familiar or widespread to modern (ca. 1500 CE and forward) notions of slavery, and importantly, not racially based. While it is possible to see a parallel between Aristotle's "natural slave/legal slave" and a chattel slave/indentured servant in the early history of the Americas, it is still possible to divorce the former from questions of race at least on an abstract level, while the latter distinction is driven by race at its very core. That's not an issue that I am equipped to deal with, but then I'm not equipped to deal with the creation of AI or deep-seated philosophical questions about the nature of intelligence. Not yet at least.
Additionally, slavery in the classical world could take forms we are unaccustomed to thinking about when we think of slavery today. A Roman, for instance might own a Greek slave who was better educated than he, who in turn would raise his master's children or run a business for his master. A gladiator could expect to die relatively quickly in his profession, and yet he might be treated like a modern rock star, being feasted and bedded in order to maintain his happiness, and thus ensure the quality of his performance in the arena. These are, of course, gross oversimplifications, but they form the broad outlines of slavery as an everyday condition that we are unfamiliar with.
Further muddying the matter is second half of one of Aristotle's antitheses, that just as some are born slaves, others are born masters; that there is an art and science to the rule of intelligent being. While it would be easy to leap to hysterics and couple the concept "natural" mastery with modern notions of race in order to arrive at the verboten concept of a master race, it is also an entirely too heated (and thoroughly disproven) chain of logic. Of relevance and use to us now is the question of how the perception of mastery alters the "master" class. Just how much does power corrupt, and through what means?
This question is an extension of a familiar one, that of "how are we shaped by the everyday use of our devices?" But again, in anticipation of the shift from "device" to intelligence, how will we be shaped by participating in a culture of use or cooperation with these "things," for lack of a better word.
We already have some experience with non-human intelligences, although to what extent they are intelligent is certainly up for debate. Somewhere back in the stack of unpublished posts for this blog I have an essay on Elephants, and whether or not they can play games. It was prompted by a video similar to this one:
We've already begun the process of exploring how to interact with non-human intelligences, things which think in ways fundamentally different from our own. Whether you or I think in the same way beyond a shared language is certainly open to debate: I'm willing to wager no, given my own relative stupidity and the diversity of "intelligences" they teach you about in grade school, like kinesthetic vs. spatial vs. musical vs. verbal learning and so on. Not to mention neurodiversity. Libraries worth of bad science fiction have been devoted to the topic, while a few shelves worth of books seem to have broached the subject intelligently. We're starting to explore it more in our natural world , as we try to get an empirical grip on what cognition actually is, but all the while we hold ourselves out above animals, and with good reason. Those lines of reasoning, however, cannot begin to prepare us for what happens when we finally through the switch or push the slinky down the first step on its path towards intelligence.
So, succintly, these are the issues I'm curious about exploring: what is an intelligence? When will we know it when we see it? How will we treat it? How should we treat it? How will we be changed by how we treat it? What are some of the precedents for these issues?
Obviously, this is a thought experiment, and more than a little crazy. It will likely be 20 years before any of these issues begin to really thrust up beyond the horizon.
So I'd like to learn more about the issues involved. I was told this could be a book, but I 'm not sure that's the best use of the next two years of my life, even if a publisher were interested. I certainly think it could be a paper, and I could probably get the abstract done in class. In unpacking the problem, there are a few angles I'd like to explore, namely how to anticipate an AI, how a slave society conducted itself (and collapsed) in the Graeco-Roman world, how those paradigms are relevant to us, how the roots of this future problem are taking shape today, and what the proposed ethics of an AI-dependent culture would actually be.
Once we have them, how will it be ethical to use them for entertainment in games? Will we be compelled to ask them nicely to play with us?
From a game design perspective (to ratchet this all back), I often think the answer of "well, story (or problem X) is hard, but someday we'll be able to plug an AI in there, which will solve everything for us" is a lazy one. AI may not be the best thing to design for, and good designs may come from designing around the need for AI. Likewise, a human-like AI may not be the most interesting AI players will want to interact with: perhaps something closer to HAL 9000, or a complex system that seems possess everything but intelligence, like an ecosystem. Imagine building a game around the Gaia hypothesis, only rather than being a complex emergent system, there was some spark of volition behind it all?
Pulling back yet another step, this time towards answering what to make of the web and its impact on games, how do these intelligences change when we move them away from a purely playful space and put them in the context of virtual worlds and networks. We've spent a lot of time in the past playing with solipsism in our fiction as a culture, but are we prepared to anticipate networked solipsism, where liminal bounds break down, and once erased, the context for thought makes it impossible to distinguish the quality of that thought?
I'm not sure where I'm leading with any of this, except that there are a number of exciting areas I'd like to plumb a little more before I pick a direction. So I'd like to ask for recommendations on books, authors, topics, articles, and what-have-you in order to help get my head around the problem, and once there, I'll ask again about actually tackling it.
I already have Weizenbaum's book on Peggy's recommendation for a starting point, but I'd like to get as broad a sampling as possible. I am nothing if not a fan of syncretism.
