I was recently invited to submit a paper to a forthcoming academic edited volume on the Singularity. As a first step I had to submit an extended abstract, around 1000 words. Here is the abstract I submitted....
Basically, the paper will be a careful examination of the conditions under which a hard takeoff might occur, including an argument (though not formal proof) as to why OpenCog may be hard-takeoff-capable if computer hardware is sufficiently capable at the time when it achieves human-level intelligence.
The Hard Takeoff Hypothesis
Ben Goertzel
Vernor Vinge, Ray Kurzweil and others have hypothesized the future occurrence of a “technological Singularity” -- meaning, roughly speaking, an interval of time during which pragmatically-important, broad-based technological change occurs so fast that the individual human mind can no longer follow what’s happening even generally and qualitatively. Plotting curves of technological progress in various areas suggests that, if current trends continue, we will reach some sort of technological Singularity around 2040-2060.
Of course, this sort of extrapolation is by no means certain. Among many counterarguments, one might argue that the inertia of human systems will cause the rate of technological progress to flatten out at a certain point. No matter how fast new ideas are conceived, human socioeconomic systems may take a certain amount of time to incorporate them, because humans intrinsically operate on a certain time-scale. For this reason Max More has suggested that we might experience something more like a Surge than a Singularity – a more gradual, though still amazing and ultimately humanity-transcending, advent of advanced technologies.
On the other hand, if a point is reached at which most humanly-relevant tasks (practical as well as scientific and technological) are carried out by advanced AI systems, then from that point on the “human inertia factor” would seem not to apply anymore. There are many uncertainties, but at very least, I believe the notion of a technological Singularity driven by Artificial General Intelligences (AGIs) discovering and then deploying new technology and science is a plausible and feasible one.
Within this vision of the Singularity, an important question arises regarding the capability for self-improvement on the part of the AGI systems driving technological development. It’s possible that human beings could architect a specific, stable AGI system with moderately greater-than-human intelligence, which would then develop technologies at an extremely rapid rate, so fast as to appear like “essentially infinitely fast technological progress” to the human mind. However, another alternative is that humans begin by architecting roughly human-level AGI systems that are capable but not astoundingly so – and then these AGI systems improve themselves, or create new and improved AGI systems, and so on and so forth through many iterations. In this case, one has the question of how rapidly this self-improvement proceeds.
In this context, some futurist thinkers have found it useful to introduce the heuristic distinction between a “hard takeoff” and a “soft takeoff.” A hard takeoff scenario is one where an AGI system increases its own intelligence sufficiently that, within a brief period of months or weeks or maybe even hours, an AGI system with roughly human-level intelligence has suddenly become an AGI system with radically superhuman general intelligence. A soft takeoff scenario is one where an AGI system gradually increases its own intelligence step-by-step over years or decades, i.e. slowly enough that humans have the chance to monitor each step of the way and adjust the AGI system as they deem necessary. Either a hard or soft takeoff fits I.J. Good’s notion of an “intelligence explosion” as a path to Singularity.
What I call the “Hard Takeoff Hypothesis” is the hypothesis that a hard takeoff will occur, and will be a major driving force behind a technological Singularity. Thus the Hard Takeoff Hypothesis is a special case of the Singularity Hypothesis.
It’s important to note that the distinction between a hard and soft takeoff is a human distinction rather than a purely technological distinction. The distinction has to do with how the rate of intelligence increase of self-improving AGI systems compares to the rate of processing of human minds and societies. However, this sort of human distinction may be very important where the Singularity is concerned, because after all the Singularity, if it occurs, will be a phenomenon of human society, not one of technology alone.
The main contribution of this paper will be to outline some fairly specific sufficient conditions for an AGI system to undertake a hard takeoff. The first condition explored is that the AGI system must lie in a connected region of “AGI system space” (which we may more informally call “mindspace”) that, roughly speaking,
- includes AGI systems with general intelligence vastly greater than that of humans
- has the “smoothness” property that similarly architected systems tend to have similar general intelligence levels.
If this condition holds, then it follows that one can initiate a takeoff by choosing a single AGI system in the given mindspace region, and letting it spend part of its time figuring out how to vary itself slightly to improve its general intelligence. A series of these incremental improvements will then lead to greater and greater general intelligence.
The hardness versus softness of the takeoff then has to do with the amount of time needed to carry out this process of “exploring slight variations.” This leads to the introduction of a second condition. If one’s region of mindspace obeys the first condition laid out above, and also consists of AGI systems for which adding more hardware tends to accelerate system speed significantly, without impairing intelligence, then it follows that one can make the takeoff hard by simply adding more hardware. In this case, the hard vs. soft nature of a takeoff depends largely on the cost of adding new computer hardware, at the time when an appropriate architected AI system is created.
Roughly speaking, if AGI architecture advances fast enough relative to computer hardware, we are more likely to have a soft takeoff, because the learning involved in progressive self-improvement may take a long while. But if computer hardware advances quickly enough relative to AGI architecture, then we are more likely to have a hard takeoff, via deploying AGI architectures on hardware sufficiently powerful to enable self-improvement that is extremely rapid on the human time-scale.
Of course, we must consider the possibility that the AGI itself develops new varieties of computing hardware. But this possibility doesn’t really alter the discussion so much – even so, we have to ask whether the new hardware it creates in its “youth” will be sufficiently powerful to enable hard takeoff, or whether there will be a slower “virtuous cycle” of feedback between its intelligence improvements and its hardware improvements.
Finally, to make these considerations more concrete, the final section of the paper will give some qualitative arguments that the mindspace consisting of instances of the OpenCog AGI architecture (which my colleagues and I have been developing, aiming toward the ultimate goal of AGI at the human level and beyond), very likely possesses the needed properties to enable hard takeoff. If so this is theoretically important, as an “existence argument” that hard-takeoff-capable AGI architectures do exist – i.e., as an argument that the Hard Takeoff Hypothesis is a plausible one.
Nice!
ReplyDeleteI saw the recent buzz on the China Brain Project, and it looks like good ol' C is part of the mix...
Has anyone considered using Google's 'GO' systems language instead? It compiles fast, and has none of those pesky pointers to worry about.
Sounds like these projects just might make something happen!
1. The threshold for self improving AI is humanity level intelligence, not human level intelligence. That is 7 billion times harder. A human cannot build AI. Without other humans, it is unlikely you would figure out for yourself that you could make a spear out of sticks and rocks. So OpenCog will not launch a singularity. Sorry.
ReplyDelete2. Intelligence depends on knowledge and computing power. An AI can't make itself smarter just by rewriting its own software. It might make itself dumber by accidentally damaging itself by erasing knowledge and then because of that damage be unable or unmotivated to undo it. Improvement requires learning and acquiring resources to build computers.
3. Friendly AI, defined as AI that does what you want, has to be built slowly because doing what you want requires knowing what you know. The AI's learning rate is bounded by how fast you can talk or type. It will take decades. You can't shortcut it by training it once and making billions of copies because the world knows a lot more than you do and most of it is not written down. They all have to be trained.
4. Therefore any fast takeoff will have to acquire knowledge and computing resources from nonhuman sources and be unfriendly, such as self replicating and evolving robots or nanotechnology that replace DNA based life.
5. Avoiding a gray goo accident is insufficient for safe AI. "Doing what we want" means satisfying our goals that were selected for evolutionary fitness in a world without AI. We think we don't want to die because we were programmed to fear the hundreds of things that can kill us. This is not the same thing. Is a robot that imitates your behavior you? If yes, then would you shoot yourself? If no, then would you refuse a procedure that would make you immortal?
6. Assuming you answered "yes", then as a computer you could reprogram your perceptions to live in a fantasy world with magic genies, or reprogram your goals to be satisfied with your current state no matter what it was, including your mortality. You want to be happy. But for all possible goal oriented intelligences that can be described in terms of a utility function, happiness means increasing your utility. All finite state machines, including your brain, have a state of maximum utility in which any thought or perception would be unpleasant because it would result in a different mental state. Well, do you want it or not?
Evolution's solution is to create brains that fear the things that can kill them and put them in bodies programmed to die. You can't fix this. It doesn't matter if evolution is based on DNA or not.
The hardware could allready exist.
ReplyDeleteSpecialised FPGA for visualisation, ( willow garage and other searcher )
Specialised hardware for 3D conceptualisation (gpu )
Specialised hardware for calculus
Specialised hard for neural network (memristor, ...)
Specialised hard for ... etc etc
I think you were right when you said our current computer has enough power, but maybe we need an AGI ( brain emulation or something ) to construct an AGI for it.
We are closed to the "singularity" : the key may be modularity ben : maybe opencog should flexible and should accept more module from other research
Matt, can't some languages (LISP, Scheme, etc.) improve their algorithms on the fly?
ReplyDeleteThese languages have the capability of modifying their code based upon selected conditions.
Isn't this "making itself smarter just be rewriting its own software"?
This comment has been removed by the author.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteNo, I don't think you realize how hard the AI problem is. We pay people $60 trillion per year worldwide to do work that machines could do if they were smart enough. Don't you think that if there was a simple, elegant solution that somebody would have found it by now?
ReplyDeleteLanguages that can modify themselves don't solve the problem of doing so intelligently. It fundamentally violates information theory to be able to do so. Knowledge can't come from nowhere.
Specialized hardware for vision only works for low level perception. You still need to train it on a decade's worth of video to teach it to see. To accomplish that in a decade on a human brain sized neural network you need 1 PB memory and 10 Pflops, or about a million PCs.
Also, your brain uses about 100 calories per hour, more than any other organ except your muscles, but silicon uses 10,000 times more power than neurons for the same computation even if you optimize it for that purpose. Why do you think the Googleplex has cooling towers?
And hardware isn't even the hard part of the problem. Nor is writing the learning algorithms for language and vision and human behavior. The hard part is collecting the knowledge, 10^9 bits from 10^10 brains at a rate of a few bits per second per person to get the few percent that is not written down or known to anyone else. You need that knowledge because a computer can only do what you want if it knows what you know. The reason that it costs a year's salary to replace a skilled employee is that no two jobs or two people are alike. You can't train a robot once and make billions of copies. You have to train each one. At http://mattmahoney.net/agi2.html I estimate it will cost $1 quadrillion, assuming we solve the hardware and learning problems. But automating the economy will be worth it. Just don't expect to be a significant part of it.
@Matt Yes a machine can do every job : OK ? Then you should rethink you economic : because then work does not mean anything : oh yes : open your eyes ! THERE ARE JOBLESS ! The INDUSTRIAL REVOLUTION is 200 years OLD !
ReplyDeleteAnd the tertiary job is disapearing also : Your illusion are dangerous, men !
The technology destroy jobs, the economy just use the possibilities given by technology ... Face you own world !
If you cannot be rationnal don't speak at all !
@Ben Do you know the lyric probabiliy processor ( financed by darpa )
http://www.theregister.co.uk/2010/08/17/lyric_probability_processor/
Well it could help.t
Re: "The threshold for self improving AI is humanity level intelligence, not human level intelligence. That is 7 billion times harder."
ReplyDeleteThat seems rather overstated. Humanity was evidently a self-improving system back in 1800 - when there were only a billion of us - and a billion humans are simply not a billion times harder to construct than one human is.
Re: "Intelligence depends on knowledge and computing power. An AI can't make itself smarter just by rewriting its own software."
ReplyDeleteIt can make its prediction agent better just by thinking about it - which will make it smarter, according to conventional definitions of intelligence.
It needs computing power to do that - but that is usually considered to be pretty cheap, and not a big limiting factor. We are heavily "software limited" today.
So, essentially, a machine could make itself smarter - just by rewriting its own source code.
@anonymous, automation has never led to massive unemployment. It has always led to higher wages and a better standard of living. I know it is easy to see where the old jobs go but hard to see where the new jobs come from. It comes from making stuff cheaper, so people have more money to spend on other stuff. That extra spending creates new jobs.
ReplyDeleteAnd don't worry about all jobs being replaced at once. AI is hard enough that it is happening incrementally. Not that you could stop it if you did worry about it.
@Tim, self improvement through software rewriting violates Wolpert's theorem, which proves that two computers cannot mutually predict each other's actions. Therefore a parent either cannot predict its child, or the child cannot predict the parent. If the parent can predict the child, then the child must be less intelligent. If the child is to be more intelligent, then the parent cannot know which changes will be effective. See http://pre.aps.org/abstract/PRE/v65/i1/e016128
Matt - I'm sorry - but that is ridiculous! We can see systems (like humanity) which have increased in intelligence over time - and we cannot tell whether the world exists under simulation or not. If such systems could not increase in intelligence inside computers (for whatever reason) we would know the world was not being simulated.
ReplyDeleteHumanity improves by adding knowledge (evolution, sensory input, culture) and adding hardware (by reproduction). Wolpert's theorem only prohibits self improvement by modifying one's software. For example, humanity could not increase its average intelligence with eugenics. We need evolution to select for fitness because we aren't smart enough to do it ourselves.
ReplyDeleteEvolution can happen inside computers too - in the form of genetic algorithms. Indeed, inything that can happen in the world can also happen inside a computer - as far as we know - that is the Church–Turing–Deutsch principle. There is no exception for increases in intelligence. Self improvement in software isn't ruled out by any known principles.
ReplyDeleteAn evolutionary simulation is not an example of self improvement. The agents get smarter as measured by a fitness function, but they don't select the fitness function. The programmer did that. It is a search algorithm.
ReplyDeleteIf the agents were initially programmed with a fitness function and selected their own offspring, the algorithm would fail because the function would drift. That's why we don't write them that way.
It doesn't matter what algorithm you use. A computer without input can't increase its knowledge as measured by Kolmogorov complexity, but it can decrease it. You can prove this. http://mattmahoney.net/rsi.pdf
Agents can *contain* evolutionary systems, though. The collective intelligence of a biosphere containing evolving agents can gradually improve (witness the planet earth). That is self-improvement - from the biosphere's perspective. If it happens in the real world, we can do the same thing in software - producing a self-improving software system.
ReplyDeleteYour paper apparently doesn't prove what you claim it does - instead it assumes self-improving software systems are possible, and then tries to put bounds on the *rate* of self-improvement.
In its last sentence, that RSI paper presupposes a link between algorithmic (Kolmogorov) complexity and intelligence. However, no such link is known. Indeed, it is often hypothesised that the Kolmogorov complexity of the entire universe may be only a few hundred bits or so: laws plus initial conditions plus age. If the Kolmogorov complexity of the whole universe could be so low, I see no reason to care if the Kolmogorov complexity of a machine intelligence turns out to be low.
ReplyDeleteResponding to various comments…
ReplyDelete***
We are closed to the "singularity" : the key may be modularity ben : maybe opencog should flexible and should accept more module from other research
***
OpenCog is quite open to integrating modules from wherever .. the limiting factor is volunteers (or $$ to pay employees) to do the integration...
***
No, I don't think you realize how hard the AI problem is. We pay people $60 trillion per year worldwide to do work that machines could do if they were smart enough. Don't you think that if there was a simple, elegant solution that somebody would have found it by now?
***
You could apply that line of reasoning to prove the impossibility of anything exciting being discovered for the first time!!! Nothing valuable can ever be invented, because if it's so valuable, it would have been invented before ;p
***
The hard part is collecting the knowledge, 10^9 bits from 10^10 brains at a rate of a few bits per second per person to get the few percent that is not written down or known to anyone else. You need that knowledge because a computer can only do what you want if it knows what you know.
***
I don't think all that knowledge is needed for an AGI to achieve the same goals that people do. As an analogy, before the advent of factories, craftsmen had a great deal of specialized and diverse knowledge. But it wasn't necessary for factory-builders to absorb all that knowledge to make factories that obsoleted those craftsmen.…
***
A computer without input can't increase its knowledge as measured by Kolmogorov complexity, but it can decrease it. You can prove this. http://mattmahoney.net/rsi.pdf
***
As I've pointed out before, your argument assumes an AI system uncoupled from its environment, and with a fixed base of hardware, and uses a highly questionable measure of knowledge/intelligence/complexity…
-- Ben G
Also I'll paste here a comment from an AGI-list email thread sparked by this blog post:
ReplyDeleteOpenCog is an OSS project that is open to incorporating modules created by various other teams for various purposes... and is being created by a team of people with various backgrounds and ideas and social networks...
And once it's more fully functional, it will be set up to interact with many humans via the Net, thus becoming part of the whole "global brain" we see emerging so powerfully lately...
I'm not agreeing that it would be impossible for an isolated team to build an isolated machine in their basement, that would undergo a hard takeoff and become a super-duper-human AGI. However, I'm agreeing with Joel that this happens not to be what we're doing with OpenCog...
-- Ben
Eliezer Yudkowsky pointed out an error in an earlier version of my RSI draft paper in which I equated knowledge with intelligence. Actually, I believe that intelligence = log(knowledge) + log(computing power). If you have infinite computing power (a Turing machine), then you don't need knowledge.
ReplyDeleteBut of course we don't. I can't prove my assertion. I can only argue that in practice, each doubling of your computer's processing speed or memory or each doubling of available software or data, incrementally increases its value or its usefulness as measured by the number of different problems it can solve by about the same fixed amount.
General, low complexity solutions to AI such as AIXI^tl or Goedel machines have exponential time complexity, consistent with my assertion. Also, you can further simplify the universe from a few hundred bits to a few bits, with a corresponding exponential increase in time, if you enumerate all possible laws of physics. By the anthropic principle, we observe a physics that is just right for the evolution of intelligent life.
That physics, BTW, means we can't build a Turing machine. The largest memory you could build is limited for one by the Bekenstein bound of the Hubble radius, about 2^407 bits.
My comment on the difficulty of AI is the same type of argument that P != NP or that AES encryption is secure. We have no proof. Instead we have lots of smart people who had an incentive to prove otherwise and failed. People tend to underestimate the strength of this argument. For example, there *is* a proof of the impossibility of recursive data compression, but that doesn't stop people from trying. The incentive to solve AI is far greater than any of these other problems. It is as big as our entire economy.
ReplyDeleteI am not arguing that OpenCog won't eventually pass the Turing test or solve other hard AI problems. I am arguing it won't launch a hard takeoff if it does. As a best case, suppose that OpenCog succeeds in uploading Ben and everyone else that worked on the project, so it could pass a Turing test even if it were conducted by their friends and relatives. Now transport it back 100 years in time. Do you think it could self improve when you would have to generate your own electricity and you could not buy new computers or replace broken parts? So why should it self improve given the limitations of today's global infrastructure? You need a system of global scale to self improve, and we already have that.
Re: "My comment on the difficulty of AI is the same type of argument that P != NP or that AES encryption is secure. We have no proof. Instead we have lots of smart people who had an incentive to prove otherwise and failed."
ReplyDeleteI don't think anyone was arguing that it was easy. As you say, the smallest self-improving machine intelligence can't be *too* small - or someone would have found it by now.
Re: "You need a system of global scale to self improve, and we already have that."
ReplyDeleteThe smallest isolated self-improving system at current tech levels looks like an open question to me. One might possibly fit one on Australia, Madagascar - or even New Zealand.
The smallest self improving system is a colony of bacteria.
ReplyDeleteMatt wrote:
ReplyDelete"You need a system of global scale to self improve, and we already have that."
...and...
"The smallest self improving system is a colony of bacteria."
Hmm - it seems as though you are either switching contexts or undermining your own argument.
A colony of bacteria is self improving with respect to intelligence in that it gains both computing power (through reproduction) and knowledge (through evolution). One could argue that this isn't "self" improving since the bacteria themselves don't have an explicit goal of increasing their individual collective intelligence. Rather, it is evolution that acts as if it had that goal.
ReplyDeleteHumans do share the goal of increased intelligence with evolution, but only because an evolved goal of wanting to learn increases evolutionary fitness. We don't share all goals. For example, we don't have the goal of dying, even though for species fitness, replacement makes more sense than repair after some point.
But assuming that the goal is intelligence, then bacteria, humanity, and OpenCog might all be considered self improving systems. OpenCog would be self improving if it could learn and use its skills to earn money that could be used to buy more computers.
My argument is that neither a bacteria colony nor OpenCog is going to launch a hard takeoff because both are far behind the global brain they have to compete with. Their impact on the timing of the Singularity will be small.
The likely signifcance of machine intelligence in that context will be that it allows adding and upgrading of global brain neurons one at a time. Replacing one neuron with a machine may not make much difference - but if you can add or replace a billion of them, then that becomes more significant.
ReplyDeleteIt doesn't seem very likely that machine intelligence will develop separately - so it won't be "behind" the global brain - but rather integrated into it.
@Matt:
ReplyDeleteThe threshold for self improving AI is humanity level intelligence, not human level intelligence. That is 7 billion times harder. A human cannot build AI.
What kind of "argument" is that :-P ?
Of course a human (in theory) can build an AI that is at least as smart as that human. Which means that such an AI could learn like a human can.
And we know that a clever human can learn pretty much everything any other human can learn if he had the time and the computational resources to do so.
So it's just about creating an AI that can learn stuff just like a human can. It's NOT about the knowledge which humanity has acquired throughout the years!
Also there is no such thing as "humanity level intelligence". You confuse intelligence(the ability) with knowledge which has been acquired by people over the years by using their intelligence.
Once we find out how to create this intelligence ability a hard takeoff is of course possible. You'd have an AI then that could learn anything a human can. The difference is that this AI could acquire knowledge much faster than any human could (like by reading the internet). And the more hardware power there is the faster that knowledge can be acquired. And so the hard takeoff begins...
Don't you think that if there was a simple, elegant solution that somebody would have found it by now?
This is because the people working on AI are people which among other things confuse knowledge with intelligence and come to other equally weird conclusions ...
It seems we are not arguing over whether there will be a singularity but rather over how "hard" it will be. The logic of Vernor Vinge is that if humans (plural) can make smarter than human level intelligence, then so could it, only faster. For that logic to follow, then "smarter than human level intelligence" means smarter than all of the humanity that made it, not smarter than a single human.
ReplyDeleteBesides, we have already created smarter than human intelligence. If you don't think so, take this IQ test.
@Matt Mahoney: I have thought about this and looked at your posts very carefully and I'm wondering how much and what kind of dope you are smoking?
ReplyDeleteFor that logic to follow, then "smarter than human level intelligence" means smarter than all of the humanity that made it, not smarter than a single human.
ReplyDeleteYes that would be the case. The AI would be smarter than all of humanity but this is achieved by the AI's ability to acquire knowledge much faster than a single human could.
So in this case you would measure intelligence by how quickly the AI can learn something.
If you leave the factors of computational speed and acquired knowledge away you could say that the intelligence level of a human and an AI would be about the same. So what makes it "smarter" is only the speed and the possibly larger memory so it can acquire more knowledge and that more quickly.
So this is the same as if you as a human scientist had 100.000 years to live. You would have a lot of time to find out new theories, to improve your existing theories and even optimize your workflow and your way of thinking by acquiring new knowledge.
For the AI it is kind of the same I think, although it might achieve what you achieved in 100.000 years much more quickly.
Of course there are limits to how the AI can get smarter. I don't think it will be able to rewrite it's base code completely. I think the basic functions of it's intelligence algorithms will always be the same. Throughout the whole singularity. What might change would be that the AI replaces some of it's general purpose algorithms for searching stuff, sorting stuff and for recognizing patterns but I believe the underlying logic will be the same. Always.
An AI could learn very fast, but not a *friendly* AI. A model of a human mind has a Kolmogorov complexity of 10^9 bits according to Landauer's studies of human long term memory. An AI needs that model to do what you want, rather than literally what you tell it, the way computers work now. A few percent of that knowledge is known only to you and not written down. This specialization among people and the jobs they do is essential to the functioning of our economy. It will take years or decades to collect this knowledge from each person at the rate they can talk or type, regardless of how fast the AI can learn. I analyzed all of this in my AGI design.
ReplyDeleteI don't claim that a hard takeoff is impossible, just not desirable.
All of this is peachy, but then you get down to the details of what does "an improvement of intelligence" constitute exactly?
ReplyDeleteIf you define an improvement of intelligence as being like optimizing a bunch of algorithms, such that you can do more, or the same, with fewer computes (approaching maximum compression) then the chances of a hard takeoff appear grim indeed.
Experience with optimizing systems such as genetics algorithms/programming suggests that the rate of improvement in performance decreases over time and becomes more and more tortuous with small improvements occurring less frequently. There may be occasional discoveries of new vistas on the fitness landscape, but these are not common as we march towards optimum compression.
Thus far nobody seems to have really been able to address this problem of a general optimizer which doesn't run out of steam over time. To show that a hard takeoff is possible, at least in principle, it's going to be necessary to demonstrate that you can devise an optimizer which doesn't run out of steam, and in fact does the opposite.
On the question of adding more hardware, this is only going to get you so far.
ReplyDeleteAs the system becomes more "intelligent", where more intelligent means you get more bang for each computational buck, or spend fewer bucks per bang, it actually requires less hardware over time as its intelligence converges towards a theoretically optimal compression.
The smartest machine possible would only need minimal computation, because its finely tuned heuristics would capture the underlying essence of the universe with exquisite concision (smarter programmers write fewer lines of code). Having more hardware than it really needs would just be energy wasted - similar to the "programmed death" of neurons during development.
Another troubling aspect of discussions on intelligence explosions is the noticeable lack of interest in the environment which the system occupies and questions over what is driving the quest for greater "intelligence". Intelligence isn't a self contained system - it's an active subsystem in the loop responding to some environment, generating variety, taking measurements and utilizing energy.
> The smartest machine possible would only need minimal computation
ReplyDeleteThat's wrong. The smartest machine possible is AIXI, which is not computable. Computable approximations with simple descriptions like AIXI^tl and Godel machines require exponential computation.
I agree with your second comment. An intelligence explosion is a catastrophe (in chaos theory) in evolution.
A machine which requires infinite computational resources is not an optimal calculator - it's the worst possible kind of calculator. If AIXI requires an infinite amount of computation then it is not intelligent (from an observer's point of view it has crashed).
ReplyDeleteFor a machine to be something approaching optimally adaptive within a given environment its rate of information processing and rate of action need to be appropriately coupled with that system. To fast or too slow, and you have maladaptive behaviour (optimisation on the wrong problem, or hunting).
ReplyDeleteI suppose you could have the machine generate far more variety than it really needs to, but in terms of adaptation this is just wasted energy/resources with the wastage becoming ever more cumbersome as time goes on.
To my way of thinking, any consideration of intelligence explosions needs to take into account the variety of the environment in which the system is situated. You would presumably have to show that the variety of the environment increases over time, whilst the system's ability to filter it at least keeps pace. Probably this situation is not tenable for long without introducing other AGIs as a source of variety - analogous to the culturally driven evolution of the human brain.
Hutter proved that the optimal strategy of a reward seeking agent in an unknown but computable environment is to guess that the environment is described by the shortest program consistent with past interaction. It is a formalization and proof of Occam's Razor. Hutter, following Solomonoff, Kolmogorov, and Levin, proved that the solution, which he calls AIXI, is not computable.
ReplyDeleteFurthermore, known approximations to AIXI with simple descriptions have exponential time complexity. This supports my argument that intelligence = log(knowledge) + log(computing power). At the other end of the extreme is a giant lookup table, which is very fast but has exponential size.
Any system which requires infeasible computing resources is not really intelligent - it's dysfunctional - and from an observer's point of view is either in a crashed state or otherwise indefinitely incapacitated. It fails to generate requisite variety due to decision paralysis and so is likely to soon be neutralised by the prevailing environmental perturbations.
ReplyDeleteSo you agree with me :)
ReplyDeleteHaha! But I don't agree!
ReplyDeleteA model of a human mind has a Kolmogorov complexity of 10^9 bits according to Landauer's studies of human
long term memory. An AI needs that model to do what you want, rather than literally what you tell it, the way
computers work now. A few percent of that knowledge is known only to you and not written down.
I'd wager that these 10^9 bits complexity of the brain are somewhat filled up with super non essential data :-P
Consider that evolution just does what "works" and not what would be optimal. Evolution is not intelligent,
because of that I think that there's a lot of rubbish going on in that 10^9 bits of yours :-P
Anyways that's not the point. What's more important is that you said - and I totally do NOT agree on that - is
that "an ai needs THAT model to do what you want, rather than literally what you tell it".
I agree that an AI needs "a model" to make it understand what you want but this model does not need to be
the same model humans use. Neither it must have the same level of complexity. In my opinion complexity is not
the problem here, it's rather how to put all the knowledge together in a way that makes sense and I don't
think that you need a kolmogorov blablabla complexity model for that :-P you just have to grow some new wits...
Syrio, if you think AI is so simple, then why don't you just build one? All these silly companies paying workers $60 trillion/year when they could just put your AI in their robots. Go ahead, I'm waiting...
ReplyDeleteNever said it's simple. It's extremely hard and much work. One gotta be a fool to think it can be done within the next 20 years(frankly I don't expect it within the next 50 years). I am working on some stuff, I admit it, and I think it can be done, but everyone working on some AGI stuff thinks that.
ReplyDeleteBut even if we do not succeed building an AI every little step in the right direction is a good thing.
After reading my previous comment again I got to apologize to you I should have kept a more decent tone.
ReplyDeleteWell, I would be interested in arguments that my numbers are wrong because my cost estimates depend on them. I don't know of anyone who has done a serious cost estimate of AI and I am as shocked as anyone to come up with a result like $1 quadrillion. Most people react with "that can't be right" but don't really offer any alternative numbers that are better than wild guesses. If you say it's not 10^9 bits per person, then what, and how do you know?
ReplyDeleteI haven't looked over your math there but To be honest I generally have my doubts with this kind of calculations and estimates. But where I have problems with is when you say
ReplyDelete"AGI must have knowledge equivalent to the world's population of about 10^10 human brains".
In my opinion this is not true. True you have a lot of expert brains in the world. But all these brains have a lot of basic knowledge in common. I don't expect that knowledge to differ much from one expert to another.
So basically what you do is you take the "lowest common multiple" of all the expert brains you have (in terms of knowledge). Maybe you'd end up with a brain knowledge level equaling some high-school graduate. But the point is that once you have that high-school graduate level of knowledge (+intelligence) that then you could let the AGI learn knowledge by itself as if it would be your son you sent to college to become some computer scientist.
By the way I am not talking about "Friendly AI" here, so don't know how much your arguments rely on that...
But I don't bother with that sort of thing... I really don't think that there will be an AGI threat.
I think it's far more dangerous that the AGI might be abused by vicious criminals...
Another thing I should mention is that optimal machine learning itself may differ much from how we humans learn stuff...
There are much faster ways of learning stuff than the way the human brain does it. I mean how much unnecessary information does the brain store when it learns a new term. It associates all kinds of stuff with a term(like something you smelled when you learned the term; some unrelated thoughts;...)
And who knows what other sort of information the brain stores with every new knowledge piece you learn.
True enough though that artificial neural networks try to emulate this sort of learning but that's not the only possible way of learning. In my AGI approach for example I don't use neural networks at all for learning since I believe that there are more efficient ways to learn stuff.
So just saying that you can't always compare the way the human brain works with the way an artificial intelligence may work and thus it is hard to make estimates based on that.
Most of what you know is shared with others or written down. It is the few percent that only you know that makes AI expensive. About a third of that is work related (assuming you spend a third of your waking hours at work) and therefore important for an AI to know to do your job. The reason that it costs $15K (according to the U.S. Labor Dept) to replace an employee is because every person and every job is unique. That cost doesn't go away when you replace people with robots. It increases because skilled and specialized job turnover cost more as a fraction of salary and those jobs will be the hardest and therefore last to automate. We are chasing a moving target. I realize this cost is still low compared to your total salary, and that is why we will build AI anyway. It just won't be as easy as you think.
ReplyDeleteBTW I am talking about friendly AI. I don't worry that projects like OpenCog will be unfriendly because it is not going to compete with the global brain being built now, which I believe will be friendly. The most dangerous threats come from AI that self improves independent of humans, specifically self replicating and evolving nanotechnology.
There are certainly faster ways to learn some stuff than humans. But the last unsolved problems, like language and vision, seem to require very large neural networks. We tried the likely shortcuts like parsers for language and similar naive algorithms for vision, and they don't work. If you think you can solve these problems without vast computing power and huge training sets, good luck. The AI textbooks will tell you all the ways that don't work.
"It is the few percent that only you know that makes AI expensive. About a third of that is work related and therefore important for an AI to know to do your job."
ReplyDeleteYes and I am saying that the work related stuff can be learned by an AI itself just as a human could. You seem to assume that every human is different and that therefore you'd have to "wire in" all the expert knowledge by hand.
I believe though that this is not the case. I think the expert knowledge does come from working specialized on some areas. By making experiences in an area and analyzing specialized problems and drawing conclusions from that and thus the expert knowledge is built up.
An AI could do that just as any other human could.
"The reason that it costs $15K (according to the U.S. Labor Dept) to replace an employee is because every person and every job is unique."
That is true. It is true because any specialized person has spent years building up its specialized knowledge drawing conclusions from experience and optimizing its work flow and so. It is costly to replace a specialized person because the new person would likely take years too to acquire such specialized knowledge.
Experts are not born with specialized knowledge, instead experts are made by focusing their work and concentration on some distinct expert area.
An AI could just do the same by focusing on some expert area only that it wouldn't take the AI years to acquire the same level of specialization instead it might take only days or even minutes in some cases.
"But the last unsolved problems, like language and vision, seem to require very large neural networks."
This is true, in these cases you have a lot of input data the program has to deal with so it's harder to optimize this. Although I believe that a lot of computation can be ruled out here by looking at the context of a problem.
> it wouldn't take the AI years to acquire the same level of specialization instead it might take only days or even minutes in some cases.
ReplyDeleteNo it wouldn't because a portion of the knowledge that a robot needs to do your job is only in your head. Either you have to train it or it has to relearn it from the people it interacts with like co-workers and customers. Either option is going to be expensive no matter how smart it is and how well it can learn and understand natural language and do research, because the people that interact with it communicate slowly and want to be paid for their time. Do the math, then give me an answer.
Seems we have reached the critical point here...
ReplyDelete"No it wouldn't because a portion of the knowledge that a robot needs to do your job is only in your head."
Matt, I am sorry but this is absurd... An AGI can learn anything itself without teaching. It can figure out the stuff which is needed to do your job on its own. Just like solving a new problem. It will learn to optimize learn from its failures.
So I really don't know how you came to the conclusion that it couldn't and that there would be some special knowledge in your brain that the AI couldn't acquire on its own.
I mean consider how that knowledge came to your brain. You weren't born with it. At some point you figured it out on your own and I am sure an AGI can do that too. Or at least I don't see a valid argument why it couldn't.
Though it goes faster of course when the AGI is taught already existing knowledge. But for that it can read books and so. And the knowledge which isn't inside the books I am sure it can figure it out on its own.
ReplyDeleteAnother way for the AGI to learn to take over your work might be just by watching your every action and run some sort of simulation so it understands why you do something.
ReplyDeleteI never said an AI could not learn your job. I'm assuming it would be smarter than a human, be able to read a book in 1 second with full understanding, copy knowledge from external sources, and so on. This does not make it cheap to relearn what you learned when you took the job from you or the people it will interact with, all the knowledge you never wrote down. It is your time and their time, not the AI's time, that is expensive. Granted, communication is 2 way and half of that time would be the AI's time, which is cheap. But if you think that for a skilled job like management or sales, that 6 months or year's salary for the half you can't automate is wrong, then tell me what numbers and how you got them.
ReplyDeleteAnd yes you could reduce that by public surveillance, recording all of your communication while you work. In fact that's what I propose in my AGI design because it is the cheapest way to build models of human minds without major advances in brain scanning technology.
Ah ok. Now I see some sense in your arguments. You are right some things the AGI will not learn so quickly it might take some time. However it is likely that it still learns faster than any human so for me this would still be fast enough to be a hard takeoff. But you are right that especially the beginning phase will take longer maybe even 50-100 years for the AI to learn everything.
ReplyDeleteBut I'd still regard that amount of time it takes for that as not-worth to mention and I'd say that is still a hard takeoff. So the question is only how hard it will be and I agree with you that it will not come immidiately once an Agi is built.
So thanks for pointing that out.
Still i'd be careful with those numbers of yours. There's a lot of stuff that can't be predicted to precisely. Although of course your calculations go in the right direction showing that especially the beginning/training phase of the earlier singular AGI might take longer than some people think.
One thing I like to add though is that there can still be a singularity without AGI taking over every damn job there is.
ReplyDeleteSo actually I always pictured a different singularity than yours. I always thought that once the AGI is "done" it will do only science on it's own and you send it to another planet in our solar system to replicate and to do research and make it built atom reactors and other energy harvesting so it can replicate and supply all kind of resources to our world.
Friendly AI worshippers will cry out loud now but I am really not convinced by there theories. This might be due to the fact that my own AGI design is made in a way so that it can't and wouldn't ever change its base code and especially the way it sees the world. Still it could do research and replicate. Its knowledge could grow but its motivations would not ever change.
I suppose my view of the Singularity is a bit different. If you plot total knowledge or total computing power against time, you observe a faster than exponential and possibly hyperbolic growth rate that goes to infinity sometime in the 2040's (according to Kurzweil) or 2005-2030 (as predicted by Vinge in 1993).
ReplyDeleteBut think about what infinite knowledge means. It means that what we know about this future is zero. You can't say anything about this future, including whether there is one.
The actual timing of this event, assuming it occurs and assuming that the actual function is hyperbolic and not some other faster than exponential function that grows forever, would violate the laws of physics as we currently understand them, which say that the universe has finite entropy and finite computing power. Of course one of the things we can't predict is whether these laws of physics are correct. We have been wrong before.
I consider the notion that our understanding of time will change in a way that makes predicting the singularity meaningless. We can distinguish between relativistic time, which depends on where you are, how fast you are moving, and the presence of gravitational fields, but does not distinguish between past and future, and information theoretic time, which does have a direction but not a (computable) magnitude. Over (information theoretic) time, computers perform irreversible operations like writing into memory and losing bits of state information, which increases by a corresponding amount its uncertainty about the state of the outside world, what we call thermodynamic entropy. These two notions of time are incompatible because memory elements are made out of particles that bounce off each other in exactly the same way whether going forward or backward. General relativity and quantum mechanics may not be consistent with each other, but they are consistent in that they make no distinction between past and future.
But assuming there is something to information theoretic time, I think this is the kind of time that is interesting to us. We can give it an approximate magnitude by counting bits learned or forgotten. Time seemed to move more slowly as a child because young brains learn faster. Time seems to jump ahead while you are asleep for the same reason. In "perceptual time", the Singularity is infinitely far in the future, no matter when it occurs in physical time because there is an infinite bit difference between these two points.
As far as the near future, which we can say something about without being 100% wrong, I think that AI will not replace all jobs at once because it is so expensive, so it won't have a catastrophic effect on the economy. Also, what is wrong with AI replacing all jobs? Wouldn't you prefer to own a machine that does the work that you used to have to do, and have it turn its paycheck over to you? As long as we have some system that distributes scarce resources fairly, I think we would all prefer to have the choice to not have to work for a living.
I consider the notion that our understanding of time will change in a way that makes predicting the singularity meaningless.We can distinguish between relativistic time, which depends on where you are, how fast you are moving, and the presence of gravitational fields, but does not distinguish between past and future.
ReplyDeleteHmm I think you can just define some "absolute time" by choosing some "moving system" as your "time anchor"(like the earth's timescale). Then you can calculate an absolute time for all other(known) systems relative to that "time-anchor" system.
Also, what is wrong with AI replacing all jobs?
I think the AI should leave some jobs to humans so that they don't feel completely useless. But then you wouldn't be working to survive. There would be enough food and everything and you're survival would be guaranteed without the job.
However I imagine a life without a real job to be pretty dull. There should be some job which offers you the possibility to grow in certain ways(get richer/more powerful/...). So that you have the notion that you can achieve something by working hard and thus giving your life a "sense".
So maybe there wouldn't be even the sort of economic we know today. But instead a controlled system where it is possible for humans to achieve something. Some sort of reward system or so.
I also don't think this exponential growth stated by Kurzweil is a realistic assumption.
ReplyDeleteSure growth and everything will be much faster than before. But there are always limits to what can be done. Limited resources. Limits set by physicis. So it is probably more likely that the singularity will be rather "jump like". There might be a great jump at a point but this might have a short breath so that we return to a very low exponential or even linear growth then.
Prediction is hard, especially about the future. Physics is a model we use to explain our observations. They are not absolute laws. Observations are just bits of input that might have nothing to do with reality. How do you know your brain isn't in a bottle, hooked up to a computer that simulates the world you think you observe? If it is, then how do you know that fundamental concepts like space, time, and matter aren't just abstract concepts invented by this simulation?
ReplyDeleteSo I will defer on long range forecasts, except to say that Occam's Razor (formalized by Solomonoff) says that each bit you add to your story about the future makes it half as likely to be true.
Absolutely agree. Long term prediction is nothing more than some game of luck. Though some predictions are liklier than others. But nothing is certain...
ReplyDeleteWhy do creative/novel science break-throughs come from people who also investigate, what in the future will be looked at as,silly magic, such as Newton's occult studies?
ReplyDeletewhat we model as reality isn't a true replication. it's a model in our head. much more simple or complex,it's a model/representation. It's a static dead model trying to represent an ever complexifying world and it's 2d modeling 3d (metaphor but very true). We are distributed intell. (humans/universe). Subconscious logic, unbreakable from emotion, gives each of us a point of view. This is all relative as well to the actual place in the space of this universe we each take up. To get to some places we must have an arrogance on certain topics that is subconscious. We'd never have the balls to think it otherwise.
also; if we knew everything we wouldn't think(or need to).
Finding out the unknown takes ultra willingness to try on anything including magic since logic can't find the unknown. This is also why faith(which is really unseperable from logic, it's relative) has been the number one software for humans for all of history(literally). Also why a passionate person beats corporate.
ps
people can be computers to and allot are right now.it is just a metaphor you know. we have simple and complex copy and paste intelligence.
love,
distant cuz ,
daniel
So, here we are, less than a year later.
ReplyDeleteI wonder if this will change anyone's mind on the probability of a hard takeoff:
http://www.electronicsweekly.com/blogs/david-manners-semiconductor-blog/2011/10/the-robots-which-teach-themsel.html
Could robots evolve autonomously? The HyperNEAT project at Cornell University has put neural-net-based brains inside robotic bodies and programmed the brains to take sensory inputs from the body and use them to figure out how to control the robotic body. Some brains taught themselves how to do it, others didn't.
The brains were put in different bodies. Some brains learnt to control the body, others failed.
The best-performing brains were replicated and put into the next generation of robotic body.
Eventually the Cornell team produced a robotic body, controlled by a brain which could get it to walk around the lab.
"It looks like a robot that 'wakes up', tries out a new gait, and then 'thinks about it' for a few seconds, before waking up again and trying a new gait," says the project leader Jeffrey Clune, "over time you see that the robot learns how to walk better and better."
A brain transferred to a different type of body - say from two legs to four, can adapt to learn new techniques for control.
Cornell has 3D printed many of the essential components of the robotic bodies like muscles, bones, batteries, wires and computers.
"Eventually, the entire thing will be printed, brains and all," says Clune, "the end game is to evolve robots in simulation, hit print, and watch them walk out of a 3D printer."
http://www.technologyreview.com/blog/arxiv/27291/
Today, Anirban Bandyopadhyay at National Institute for Materials Science in Tsukuba, Japan, unveil a promising new approach. At the heart of their experiment is a ring-like molecule called 2,3-dichloro-5,6-dicyano-p-benzoquinone, or DDQ.
This has an unusual property: it can exist in four different conducting states, depending on the location of trapped electrons around the ring. What's more, it's possible to switch the molecule from one to state to another by zapping it with voltages of various different strengths using the tip of a scanning tunnelling microscope. It's even possible to bias the possible states that can form by placing the molecule in an electric field
Place two DDQ molecules next to each other and it's possible to make them connect. In fact, a single DDQ molecule can connect with between 2 and 6 neighbours, depending on its conducting state and theirs. When one molecule changes its state, the change in configuration ripples from one molecule to the next, forming and reforming circuits as it travels.
Given all this, it's not hard to imagine how a layer of DDQ molecules can act like a cellular automaton, with each molecule as a cell in the automaton. Roughly speaking, the rules for flipping cells from one state to another are set by the bias on the molecules and the starting state is programmed by the scanning tunnelling microscope.
And that's exactly what these guys have done. They've laid down 300 DDQ molecules on a gold substrate, setting them up as a cellular automaton. More impressive still, they've then initialised the system so that it "calculates" the way heat diffuses in a conducting medium and the way cancer spreads through tissue.
And since the entire layer is involved in the calculation, this a massively parallel computation using a single layer of organic molecules.
Bandyopadhyay and co say the key feature of this type of calculation is the fact that one DDQ molecule can link to many others, rather like neurons in the brain. "Generalization of this principle would...open up a new vista of emergent computing using an assembly of molecules," they say.
Clearly an intriguing prospect.
obat alami ambeien untuk ibu hamil - Obat wasir ambeien parah
ReplyDeleteobat alami ambeien luar - Obat herbal wasir ambeien parah
obat alami ambeien berdarah - Obat untuk wasir ambeien parah
obat ampuh ambeien wasir terdaftar BPOM - obat wasir ambeien yg ampuh
obat bermutu untuk ambeien wasir parah - obat kutil kemaluan terbaik
obat alami untuk ambeien wasir parah - cara menghilangkan kutil kemaluan
obat tradisional ambeien wasir - cara menyembuhkan kutil kemaluan
obat wasir ambeien tanpa operasi - obat herbal ampuh ambeyen wasir yg manjur
obat wasir ambeien bpom - obat kutil kemaluan yg manjur
obat herbal alami untuk kutil pada kemaluan pria wanita - obat kutil kemaluan mujarab