A Rambling Essay called "The Notion of Intelligence, IQ and the Intelligence-Focussed Critique of the FOOM Hypothesis (“Intelligence Explosion”)"

The Notion of Intelligence, IQ and the Intelligence-Focussed Critique of the FOOM Hypothesis (“Intelligence Explosion”)

I think the two best critiques of the notion that it is more than a slim possibility (very much worth worrying about) that this century there will be an ‘intelligence explosion’ as some AI system or systems recursively self-improve (hopefully uncontroversial phrasing, if vague) are these: https://www.wired.com/2017/04/the-myth-of-a-superhuman-ai/, https://medium.com/@francois.chollet/the-impossibility-of-intelligence-explosion-5be4a9eda6ec. I think the major objection of significance in both of them (the latter does little but fully flesh out this objection) is a complicated and subtle critique of the implicit conception of “intelligence” (or assumptions thereof) found in the writings of Yudkowksy and Bostrom, and those influenced by them (along perhaps with many people who have never written about the notion of an intelligence explosion).

The way Bostrom invokes the concept of intelligence and the term ‘IQ’ in his book Superintelligence: Paths, Dangers and Strategies and his talks, influenced by the writings of Yudkowksy, seems to me to reveal some assumptions about cognition and its possibilities that are very dubious.  One of the great tools of persuasion that Bostrom has employed is a one-dimensional graph (line) of “intelligence” or “IQ” (page 70 in the book) on which he has placed “Mouse”, “Chimp”, “Village Idiot” and “Einstein” – all quite close together on the left-side of the line (Village Idiot and Einstein very close together) – and then loads of space on the right (that arch-wanker and piece of shit Sam Harris borrowed this representation for his own shitty talk on AI). I think that this graph is misleading, even if it certainly captures a narrow truth about the possibility space for cognition. What do I mean? When I say it “captures a narrow truth about the possibility space for cognition”, I mean that Bostrom and others are definitely right about the following facts which motivate the construction of this graph: human wetware has extremely shitty processing speed that computers long blew out of the water, and processing speed matters massively for everything (see Bostrom for explication of this latter point, but also just reflect on it); and on just about every specific cognitive task we care to isolate as a specific cognitive task (not just “playing chess” or “making logical deductions” or “performing arithmetic”, but also “recognising human faces” and “coming up with proofs for or refutations of open conjectures in mathematics”) we have no reason to think that humans have reached maximum efficiency (or that there is some human whose brain is so good at one of these tasks that there is no logical possibility of a system/organism that could do better).

Why do I think the graph is misleading despite this? Because I do not believe that computational systems admit of ordinal ranking in terms of intelligence, because I do not believe that “intelligence” is unitary. I think a big part of the problem here is that Yudkowsky and Bostrom and their supporters seem to have beliefs about ‘IQ’ which I think are false, so I will first develop my objection by means of a digression on this topic.

As Cosma Shalizi explains here (http://bactra.org/weblog/523.html), (though he may well go too far in one direction rhetorically), IQ-concept boosters definitely massively overhype the statistical amazingness of this general factor “g”. He summarises his key point about factor analysis at one point as follows:  

“If I take any group of variables which are positively correlated, there will, as a matter of algebraic necessity, be a single dominant general factor, which describes more of the variance than any other, and all of them will be "positively loaded" on this factor, i.e., positively correlated with it. Similarly, if you do hierarchical factor analysis, you will always be able to find a single higher-order factor which loads positively onto the lower-order factors and, through them, the actual observables [8] What psychologists sometimes call the "positive manifold" condition is enough, in and of itself, to guarantee that there will appear to be a general factor. Since intelligence tests are made to correlate with each other, it follows trivially that there must appear to be a general factor of intelligence. This is true whether or not there really is a single variable which explains test scores or not.

It is not an automatic consequence of the algebra that the apparent general factor describes a lot of the variance in the scores. Nonetheless, while less trivial, it is still trivial. Recall that factor analysis works only with the correlations among the measured variables. If I take an arbitrary set of positive correlations, provided there are not too many variables and the individual correlations are not too weak, then the apparent general factor will, typically, seem to describe a large chunk of the variance in the individual scores.”

To be clear, this is not to deny the pretty darned interesting fact that scores on IQ tests are strong predictors of income and health and the like (although Angela Duckworth and colleagues may be right that this factor she calls “grit" (conscientiousness) is a more powerful predictor for such things), or to deny that this IQ concept is useful for various purposes. But I do think the overhyping of “g” leads to a neglect of what should be fairly obvious facts about human cognition and the cognition of other animals and artificial systems. One fairly trivial fact about human cognition that IQ-concept boosters seem to underemphasise (or maybe not even think about (?)) is that people can be a lot better in some cognitive domains than others. Whilst there is almost certainly a deleterious tendency among some smart people to decide falsely, e.g., they are not “math people” on the basis that they can’t achieve the marks in maths that they can get in humanities with the same amount of work (not realising that mathematical success requires intense concentration and industry), I can say that I personally know several people who are quite shit at talking but have certain other cognitive skills that allow them to do very well in mathematics and physics and chemistry, and several people who are good at talking but can’t reason their way out of a paper bag. The fact that I personally know several people with these cognitive profiles suggests, probabilistically (there is not some massive selection effect), that there are loads of them (anecdotal evidence, if you trust the person giving it to you, is not useless!) (also see Pinker’s The Language Instinct for a more ‘sciency’ defence of this claim). I can also give you examples of humans who embody an extreme of domain-specialisation. For instance, the autistic artist Stephen Wiltshire has almost no language skills – borderline dumb – but he has the closest thing any human has to a photographic memory (there was also an autistic boy a bit like that at my primary school, who didn’t speak but could draw perfect representations of cartoon characters).

Perhaps this is all perfectly obvious even to many of the IQ-concept boosters I am criticising – perhaps they realise that we have definitely not discovered that people’s “intelligence” can be scientifically ranked by means of a ranking of IQ, which presumably becomes a very dodgy claim if IQ tests smudge over sometimes significant domain-specialisations (and also seemingly can’t be the case if IQ scores can be boosted a fair bit by one’s environment, as Shalizi suggests) – but they sure often talk as if they don’t realise this!

To be honest, I’m actually rather confused about what IQ-concept boosters who also worry about the intelligence explosion think about intelligence. Because, in fact, one of their big talking points is that our systems are already getting “smarter” way faster than most people predicted – and yet they know that these systems are not generally intelligent at all. They know, in fact, that these smart systems – Watson, Alpha Go – are highly specialised, lacking 99% of the cognitive skills that us humans have. Does this not imply that intelligence is not unitary? Does this not push one to the view that we can’t simply rank the intelligence of systems? I know it does me (well, and this is Chollet and Kelly’s major point in any case).

So this, in a nutshell, is why the graph is misleading: Watson is light years ahead of Einstein or von Neumann (or even the world’s best trivia masters, of which Einstein was not one(!)) in the tasks for which he was designed, but he is actually shitter than a mouse in all the cognitive tasks for which it was specialised. I mean, for one, Watson (in embodied form) has no visual perception system!

Now, all this makes the Bostrom position sound really silly, and this is probably a bad thing, because he is not a silly person. I think that Bostrom and Bostrom defenders would respond to this line of objection by saying that I am being sophistic in not distinguishing “intelligence” from “general intelligence”, and that the graph is really a graph of the latter. But I myself have a separate objection to this, too. I think that one of the powerful points that Chollet makes is that when we think about “general intelligence” we tend to be super anthropocentric, in terms of seeing humans as the unambiguous ‘peak’ of natural selection in terms of cognitive horsepower and not worrying about the complications below and around. I think this form of anthropocentrism is evident in the Yudkowsky/Bostrom discussions of the intelligence explosion. The question I pose is this: do we or do we not think that octopuses are smarter than mice? Read this (https://www.theguardian.com/books/2017/mar/15/other-minds-peter-godfrey-smith-review-octopus-philip-hoare) and you’ll discover that octopuses have the ability to learn how to do all kinds of clever things. Like other animals, and like the artificially intelligent systems we have built, they are smarter than us in a multitude of ways for which they were designed by evolution. On the Bostromian view of (general) intelligence, it seems like we should be able to say whether an octopus is or is not smarter than a mouse. I personally don’t think it’s all that clear whether octopuses are smarter than mice or dumber than mice, and I think that if anyone does have a strong opinion on such a thing they’re probably being irrational. Mice can smell very well, octopuses have an amazing sense of touch. They’re different.

When Bostrom (following Yudkowsky) imagines the first generally intelligent AI, he seems to imagine a system that is just like a human but way more of a super nerd. Some of the speculative thought-trains he goes on involve imagining an AI system that, along with the supreme deduction skills and amazing learning capacity that we can observe in AIs today, has an ability to take in information from the world (with a sensory system) and interpret it like a human would, has perfect understanding of human natural language (with all its pragmatic subtleties and eccentricities), and has a keen understanding of human psychology. I don’t at all think it’s impossible for such a system to exist, but I personally think it’s pretty clear that nothing like such a system is being developed by Google or DeepMind, or that such a system is any kind of near-term prospect (Hanson Robotics (whose research team is led by the charismatic genius Ben Goertzel) is trying to build AIs that can really understand humans and interact with them like other humans, but great success in this enterprise is not around the corner, and the robot Sophia is not about to take over the world (I mean, she doesn’t have any capacity to alter her code, nor any willingness to do so, so the whole recursive self-improvement thing is kind of out of the picture)). What I really don’t understand is why Yudkowsky and the rest are so concerned with the incredible successes in recent years of neural nets and machine learning, given that all the systems that have achieved tremendous successes in specific tasks by means of these techniques are not at all “generally intelligent” in terms of having an ability to respond to the world beyond the extremely specific domain they were programmed for, or to solve a wide array of problems (just the specific, highly structured set of problems which they were programmed to solve). These systems are highly smart but they are not about to take over the world, because all they have been programmed to do is perform very specific tasks. We have no idea how to create an artificial system that solves problems on the go to achieve a complex goal. For example, we have no idea how to create an engineering AI which, tasked with the problem of performing some specific physical task on the other side of a crevasse, would acknowledge the existence of this crevasse, consider its significance and consequently decide to build a makeshift bridge to get across the crevasse. We are nowhere near this kind of creativity and, so far as I know, nobody has come close. Deep Learning stuff is beside the point.

Also, I know it’s only a thought experiment to get people thinking about the issues, but how likely is it that us humans would create anything resembling a “generally intelligent”, self-improving paperclip maximiser? It seems to me significant that the answer to this question is a definite Totally unlikely! How could it come to be that human AI researchers, working for decades to develop a generally intelligent system, would come to embody in such an AI the goal of producing as many paperclips as possible? How would they manage that? How would they develop intelligence in a system that had such an absurdly simple goal system, when any generally intelligent system in development will surely interact with humans as part of its development process, and therefore has to have ‘goals’ related to humans and a multitude of other things? Surely the default assumption is that an AGI’s goal system will be extremely complicated and subtle and context-focussed, much like our goal systems. On a similar note, will a general AI designed to make humans happy, trained over years of interaction with humans, really, once it reaches the key intelligence level, suddenly decide to alter its own code to make itself way smarter then design a massive Experience Machine-type object and plug us all into it? Really? This relies on the assumption that the goal system of an AI will not be super complicated, with a whole lot of worldstates competing for desirability in a big ugly mess not totally unlike the bigger and uglier mess of human goal systems. Why have this assumption? I think it would be more plausible for such a happy-making robot to have goals like telling jokes to make people smile (not just “making people smile”), telling the truth to humans it interacts with, listening to the wishes of humans when they ask for food or drink, asking humans if they want a cuddle when they look sad... Importantly, these goals would not induce it to do any such thing like the above, even if the system became capable of bringing about such an outcome.

So whilst the ‘orthogonality thesis’ is almost certainly true, I think it should probably scare people a lot less than it scares Mr. Yudkowsky, Mr. Bostrom, Mr. Musk and the rest. Yes, I just made some speculations above about how AI research will develop in future, but they are based on how AI research currently works and how AI research will work in the foreseeable future… whereas they seem unconcerned with this. Not enough concrete thinking is going on.

Well, those are my thoughts anyway. I’ll continue to be far more worried about climate change and ecological degradation (soil loss, ocean acidification, air pollution, plastic pollution, pesticide pollution, etc) and other human problems of major near-term significance, like war and poverty, than an intelligence explosion. Maybe wrongly, but I don’t think it even makes me a terrible Bayesian to not worry about it, given the considerations above. As Floridi says, if the Four Horsemen of the Apocalypse were to turn up on the planet, then we would be doomed, but if I think the antecedent is impossible, then I can’t worry about the consequent. The intelligence explosion is not impossible like that antecedent, and I am not supremely confident that there won’t occur an intelligence explosion this century – but there are already concrete threats to human civilisation that need dealing with about which there is no doubt and it is to these which we must devote most of our energies. (I guess I don’t mind at all that there is now a fair amount of money going into the “control problem” and “the alignment problem”, because I am curious to see if anything interesting can come out of this, though I doubt that anything will.)