I think of the Tom Hanks character in Saving Private Ryan on the opening Omaha beach sequence. Some soldiers mistakenly believed it was better to hide behind the steel crosses on the beach or to not creatively attack the pill boxes that had them pinned down. I think of the difficult goals of getting space colonized in a major way or conquering diseases and making significant progress against age deterioration as pill boxes that have us pinned down on a dangerous beach. Just because the time has been stretched out to decades, centuries, millenia does not mean that we are not collectively on a dangerous beach. We can and should do a lot over the next 50 years and beyond.

Read the whole thing.

What would work better would be transferring over the moral complexity that you used to make up these goals in the first place.

Also, as you point out, these goals are vague. More specific and useful from a programmer’s perspective would be some kind of algorithm that takes human preferences as inputs and outputs actions that practically everyone sees as reasonable and benevolent. Hard to do, obviously, but CEV (http://www.singinst.org/upload/CEV.html) is one attempt.

That’s really the crux. Moral complexity does exist in algorithmic form…within our brains. And that goes to the difference between laws and goals. My goals are what I’m trying to do, both morally and in other areas. There are some sophisticated software programs running in my brain made up of things that I’ve been taught, things I’ve figured out for myself, and things that are built in. All of these add up to provide me the tendency to act a certain way in a certain situation. The strategies that drive that software are my moral goals.

Laws, on the other hand, exist outside of myself. I am not specifically programmed to do unto others as I would have them do unto me. I have some tendencies in that direction, but there’s nothing stopping me from acting otherwise, and — let’s face it — I often do. I have tendencies to be nice, fair, just, etc., but I also have tendencies to try to get what I want, to get even with those who have wronged me, to try to be a bigshot, and so on. These tendencies compete with each other, and my behavior overall is some rough compromise.

An artificial general intelligence (AGI) built as a reverse-engineered human intelligence would be in the same position. It would have the “moral complexity” Michael mentioned, but also the baggage of competing tendencies. You could no more guarantee such an intelligence’s compliance with a rule or set of rules than you could a human being’s.

A law like the Golden Rule is a high-level abstraction of certain strategies (algorithms) that produce a desired set of results. On a conscious level, I can use that abstraction to determine whether my behavior is where I want it to be:

Wife complained of being chilly when I got up at 5:00 AM to work out. Covered her with blanket. Good.

Sped up on highway in attempt to keep a guy trying to merge from going ahead of me. Not so good.

Commenter on blog revealed that he doesn’t really understand the subject at hand. Ripped him to shreds. Bad.

Through discipline and practice, I can “program myself” with it to try to move my tendencies in that direction. But I can’t write it into my moral source code and set it as an unbreakable behavioral rule. That’s partly because it’s too vague and partly because I simply lack that capability.

Presumably, I could be externally constrained always to follow the Golden Rule, no matter what. If my actions were being constantly monitored, and I was told that the I would be killed immediately upon violating the rule…I’d certainly do my best, now wouldn’t I?

Still, I’d have a hard time believing that anyone holding me in such a position was much of a practitioner of that rule him or herself. If the people trying to enforce the rule on me in this manner told me that it was for my own good — that they were trying to make me a better person — I don’t know that I’d buy it. And if I figured out that they were only doing this to protect themselves from harm I might to do to them, I think I would pretty annoyed with them (to say the least.)

I would expect a reverse-engineered human intelligence to feel the same way, so I don’t think attempting to constrain an AGI in such a manner would be a particularly good idea, especially not if we have a reasonable expectation that it will eventually be smarter and more powerful than us. On the other hand, letting it use the process I described above — evaluating its own behavior against a defined standard — an AGI might achieve far better results than I have, if only because it can think faster and would have much more subjective time in which to act. This is the notion of recursive self-improvement that matoko kusanagi referred to. The trouble with recursive self-improvement on its own, as Eliezer Yudkowsky and others have pointed out, is that if the AI starts “improving” in a direction that’s bad for humanity, things could get out of hand pretty quickly.

If the artificial intelligence is a modified version of human intelligence, or new intelligence built from scratch, we raise the possibility of building a moral structure into the intelligence, rather than trying to enforce it from outside. That’s the idea behind the the Three Laws and my Three Goals — that they would somehow be built in. But they certainly can’t be built in in anything like their current form. Michael Sargent (and others) pointed out the weakness of that approach, the less important goals have to take the back seat to the more important ones:

Each Goal must have a clear and unbreakable priority over the others that follow it and thus, in the order stated, collective continuity trumps individual safety (“The needs of the many outweigh the needs of the few, or the one.”), individual safety (broadly construed, ‘stasis’) trumps individual liberty (‘free will’), and happiness (‘utility’, a notoriously slippery concept for economists and philosophers to get a firm intellectual grip on) trumps both individual liberty and individual well-being (allowing potentially self-destructive behavior on the individual level insofar as that behavior doesn’t exceed the standard established for ‘safety’ in Goal 2).

I see the reasoning here, but I’m not 100% convinced. Consider the goals that drive a much simpler AI, system — the autopilot system found on any jet airliner. The number one unbreakable goal has got to be don’t crash the plane. But there are many other goals that might drive such a system:

Don’t move in such a way as to make the passengers sick.

Don’t waste fuel.

In landing, don’t go past the end of the runway.

Above all, the system will seek to ensure that first goal. But within the context of ensuring that first goal, it also has to do everything it can to ensure the others. And, yes, it can and must sacrifice the others from time to time in service of the first. So the plane might temporarily move in a nauseating way, or it might waste fuel, or it might even slide past the end of the runway if doing any of those things help ensure the first goal.

Reader TJIC suggested that an AI programmed to meet the Three Goals as I defined them…

1. Ensure the survival of life and intelligence.

2. Ensure the safety of individual sentient beings.

3. Maximize the happiness, freedom, and well-being of individual sentient beings.

…would end up creating a nanny state wherein human freedom is always sacrificed to individual safety. And he may well have a point, but I would argue that just as an autopilot can be calibrated to allow whatever what we deem the appropriate relationship between having the flight not crash and not make us sick, so could these three goals be calibrated in such a way so as to maximize human freedom within an acceptable level of individual risk — whatever that might be.

Getting back to the vagueness problem, it’s hard to calibrate the goals as stated, seeing as they are written in an awkward pseudo-code that we call human language. If we want to improve on the algorithms that are built into human intelligence, or develop entirely new ones — in other words, if we’re going to come up with algorithms that will provide us the ends stated in the goals — we’re going to have to do it mathematically.

But that isn’t necessarily going to be an easy thing to do. Eliezer Yudkowsky argues that developing an AI and setting it to work on doing some good thing are relatively easy compared to the third crucial step, making sure that that friendly, well-intentioned AI doesn’t accidentally wipe us out of existence while trying to achieve those good ends:

If you find a genie bottle that gives you three wishes, it’s probably a good idea to seal the genie bottle in a locked safety box under your bed, unless the genie pays attention to your volition, not just your decision.

Again, I think this goes to the issue of calibration of the system. Eliezer wants to calibrate what the AGI does with the coherent, extrapolated volition of humanity. Volition is an extremely important concept. Earlier, I mentioned the golden rule. If I decide that I’m going to do unto others as I would have them do unto me, I might start handing out big wedges of blueberry pie to everybody I see. After all, I like pie and I would love it if people gave me pie. But if I give my diabetic or overweight or blueberry-allergic friends a wedge of that pie, I wouldn’t be doing them any favors. Nor would I be doing what I wanted to do in the deepest sense.

Eliezer describes the concept of extrapolated volition as meaning not just what we want, but what we would want if we knew more, understood better, could see farther. Coming up with a coherent extrapolated volition for all of humanity is a tall order, especially if we’re doing it not just for the sake of conversation, but in order to enable a system which will try to realize that which is within our volition.

I like to think that humanity’s CEV would look a lot like the three goals that I’ve written. And I honestly believe that the algorithms that power human progress do work, in a rough and general way, towards those goals, which is why people are generally freer, safer, and happier than they have been in the past — though obviously not without many, many, appalling and horrific exceptions. So perhaps our calibration efforts involves feeding the AGI algorithms that will enable it to speed our progress towards those goals while cutting the exceptions way down. Or eliminating them, if that’s somehow possible.

So to finally come around to it, what will those algorithms look like?

Maybe we can take hint from the study of Game Theory. Robert Axelrod held two tournaments in the early 1980′s in which computer programs competed against each other in an attempt to identify the optimal winning strategy for playing the iterative version of the the famous Prisoner’s Dilemma. In the one-off version of the game, the optimal strategy is to screw the other guy. (This is not the sort of thing we want to go teaching the AGI, at least not in isolation!) However, when multiple rounds of the game are played, something else begins to emerge:

By analysing the top-scoring strategies, Axelrod stated several conditions necessary for a strategy to be successful.

Nice
The most important condition is that the strategy must be “nice”, that is, it will not defect before its opponent does. Almost all of the top-scoring strategies were nice. Therefore a purely selfish strategy for purely selfish reasons will never hit its opponent first.

Retaliating
However, Axelrod contended, the successful strategy must not be a blind optimist. It must always retaliate. An example of a non-retaliating strategy is Always Cooperate. This is a very bad choice, as “nasty” strategies will ruthlessly exploit such softies.

Forgiving
Another quality of successful strategies is that they must be forgiving. Though they will retaliate, they will once again fall back to cooperating if the opponent does not continue to play defects. This stops long runs of revenge and counter-revenge, maximizing points.

Non-envious
The last quality is being non-envious, that is not striving to score more than the opponent (impossible for a ‘nice’ strategy, i.e., a ‘nice’ strategy can never score more than the opponent).

Therefore, Axelrod reached the Utopian-sounding conclusion that selfish individuals for their own selfish good will tend to be nice and forgiving and non-envious. One of the most important conclusions of Axelrod’s study of IPDs is that Nice guys can finish first.

Bill Whittle has written recently that the qualities listed above underpin western civilization, and help to explain why the West has out-competed other civilizations, who operate using different strategies:

Now, this is where my own analysis kicks in, because frankly, nice, retaliating, forgiving and non-envious pretty much sums up how I feel about the West in general and the United States in particular. The web of trust and commerce in Western societies is unthinkable in the Third World because the prosperity they produce are fat juicy targets for people raised on Screw the Other Guy. Crime and corruption are stealing, and stealing is Screwing the Other Guy. It’s short-term win, long-term loss.

I would add that if we look at the three goals as goals for humanity rather than for artificial intelligence, we see better progress towards them in western societies than elsewhere. In the tournament, the winning strategy, embodying all of the above characteristics, was called tit-for-tat. Interestingly, the computer program driving that strategy consisted of only four lines of BASIC code. That’s very interesting, and it suggests a startling possibility — like a simple recursive formula producing a complex Mandelbrot image, the moral complexity we’re looking for might just be packed into a very simple set of mathematical relationships.

So in order to develop and calibrate an Artificial General Intelligence that carries out our three top goals (or that helps us to achieve our coherent extrapolated volition) one of the important parameters to explore is how the AI relates to us and to other AIs. The secret might ultimately lie in playing nice with the AI, and teaching it to play nice with us and with other AIs. Not just because we want it to be nice, but because nice turns out to be — at a mathematical level — the best way to play.

UPDATE: This entry has been republished at the website of the Institute for Ethics and Emerging Technologies.

Court to rule if chimp has human rights

He recognises himself in the mirror, plays hide-and-seek and breaks into fits of giggles when tickled. He is also our closest evolutionary cousin.

A group of world leading primatologists argue that this is proof enough that Hiasl, a 26-year-old chimpanzee, deserves to be treated like a human. In a test case in Austria, campaigners are seeking to ditch the ‘species barrier’ and have taken Hiasl’s case to court. If Hiasl is granted human status – and the rights that go with it – it will signal a victory for other primate species and unleash a wave of similar cases.

Hiasl’s story is not a happy one. If he were human, everyone would agree that his rights have been repeatedly, horribly violated: stolen from his family while still a baby; sent to a distant country where he was to be sent to a lab as the subject of whatever experiments were paying that month; snatched once again from the sanctuary where he eventually ended up only to be threatened once again to be sent to a different lab.

The folks bringing this case are attempting to get human status for Hiasl so that he can be appointed a legal guardian, and be protected from any further threat of being shipped off to a lab. Presumably, his being granted human status would save all chimps in Austria (and possibly the EU) from the same fate. I’m no lawyer, but I can’t help but think that if one chimp is human, they’re all pretty much human.

It probably wouldn’t take long for gorillas and orangutans to get similar legal status. Would all the big primate houses in all of Europe’s zoos be shut down? Or would the zookeepers manage to get legal guardian status for “their” apes — even if they did, viewing of these animals by the public would presumably be illegal. Plus, how long before less human-like, yet intelligent and beloved animals such as dogs and horses are also granted human status? And once horses, beef industry notwithstanding, wouldn’t cows get the same rights? Pigs are more intelligent than horses — surely they would be granted human status as well. And of course, cats.

Before long, we’ve got the PETA dream of all animals having recognition under the law equal to that of human beings.

Would that be a bad thing? Getting past the idea that it sounds kind of crazy — and the lifestyle change it would involve for someone like me who enjoys fishing and who loves sitting down to a big medium rare porterhouse — I can at least understand the impulse behind wanting to create that world. A world in which animals no longer suffer pain at the hands of capricious humanity doesn’t strike me as a particularly bad idea, nor as a totally unreasonable expectation. I have argued in the past that technology will soon get us to the point where we can enjoy eating real meat and wearing real leather without killing animals, and that those who like to hunt and fish will be able to experience those things virtually, again with no animals being harmed.

At that point, I think laws will be passed protecting animals from being killed; these will be a natural extension of our current laws preventing cruelty and mistreatment of animals. Animals don’t have to be human in order to enjoy such protections under the law.

Around the same time, we may see a new set of pretenders to human legal status — artificial intelligences. If chimps are granted human status by this Austrian court, it could prove significant in a subsequent legal battle over the rights of a computer with chimp-level (or greater) intelligence. After all, if Hiasl is deemed human under the law, how could a creature with even greater intelligence, greater sensitivity, and greater self-awareness be found not to be human? If there is no precedent from Hiasl’s case, lawyers arguing for the AI’s rights will have to rely on a different strategy; the way forward may prove more difficult if courts tend to rally around protecting the line between humanity and other entities.

Which leads to another thought — how long before Transhuman Rights becomes a recognized specialty within the practice of law?

UPDATE: Yikes. The linked story is dated April 1. I’m having a bad week, here. Either this business of passing off April Fool’s jokes as real news has gotten totally out of hand, real life has become indistinguishable from an April Fool’s joke, or I’m just a big dope. No, I’m not asking for anyone to weigh in on the relative likelihood of those scenarios. Anyway, I’m not the only one being suckered if it is a prank.

UPDATE II: Plus everyone on Slashdot is fooled, should my paranoid suspicions pan out. Currently there are almost 1800 comments on this story, including at least one individual arguing that his/her cat should, indeed, be considered human under the law.

UPDATE III: Okay, here’s coverage from March 28. If it’s an April Fool’s joke, they’re breaking the rules. Now back to our serious discussion about animal and transhuman rights. The absurdity of trying to distinguish “reality” on the web we’ll save for another day.