To follow this blog by email, give your address here...

Sunday, September 28, 2014

Speculations Toward a Precise Model of Morphic Fields

Gentle Reader Beware: This post presents some fairly out-there ideas about the nature of memory and the relationship between the mind and the universe!  If you're a hard-core psi skeptic or a die-hard materialist you may as well move on and save yourself some annoyance ;-) …

On the other hand, if you're intrigued by new potential ways of connecting known science with the "paranormal", and open to wacky new ways of conceptualizing the universe, please read on !! …

Rupert Sheldrake, Morphic Fields and Psi

In summer 2012, when Ruiting and I were in the UK for the AGI-12 conference at Oxford, we had the pleasure of stopping by the London home of maverick scientist Rupert Sheldrake for a meal and a chat.  (It was a beautiful British-style home with the look of having been host to a lifetime of deep thinking.  The walls were covered floor-to-ceiling with bookshelves containing all manner of interesting books.   We also met Rupert's very personable wife, who is not a scientist but shares her husband's interest in trans-materialist models of the universe.)

I have been fascinated by Sheldrake's idea since reading his book "A New Science of Life" in the 1980s.  His idea of a "morphic field" -- a pattern-field, coupled with yet in some ways distinct from the material world we see around us, shaping and shaped by the patterns observable in matter -- struck me at first sight as intriguing and plausible.  The mathematician in me found Sheldrake's descriptions of the morphic field idea a bit fuzzy, but then I feel that way about an awful lot of biology.  At very least it has always seemed to me an intriguing direction for research.

It also occurred to me, when I first encountered his ideas, that morphic fields could provide some sort of foundation for explaining telepathy. The basic idea of the morphic field is simply that there is a "pattern  memory field" in the universe, which records any pattern that occurs anywhere, and then reinforces the occurrence of that pattern elsewhere.   I reflected on the phenomenon of twin telepathy and it seemed very "morphic field" ish in nature.

More recently, Damien Broderick and I have co-edited a book called "The Evidence for Psi", to appear early next year, published by McFarland Press.   In the book we have gathered together various chapters summarizing  empirical data regarding psi phenomena, attempting to broadly summarize the empirical case that psi is a real phenomenon.  Sheldrake contributed a chapter to our book, summarizing experiments he did on email and telephone telepathy.  I had previously read  Sheldrake's description of his experimental work on dogs anticipating when their owners will get home, and been impressed by his careful and practical methodology.

While "Evidence for Psi" is still awaiting release, I'll point readers interested in the existing corpus of evidence regarding the existence of psi phenomena to my Psi Page, which contains links to a couple prior books I recommend on the topic.   "Evidence for Psi" contains a more up-to-date and systematic overview of the evidence, but it's not out quite yet.

Damien and I are also planning to edit a sequel book on "The Physics of Psi", covering various theories of how psi works.   I've proposed my own sketchy theory in a 2010 essay, which proposed a certain extension to quantum physics that seems to have potential to explain psi phenomena.  I actually have more recent and detailed thoughts these lines, which I'll hint at toward the end of this monster blog post ... but will not enlarge on completely here as it's a long story -- of course I'll lay these ideas out in a chapter of "The Physics of Psi" when the time comes!

While researching possible extensions to quantum theory that might help explain psi, I noticed a paper by famous physicist Lee Smolin presenting an idea called the "Precedence Principle", which struck me as remarkably similar to Sheldrake's morphic field theory.   I discussed this similarity in a previous blog post.

During our visit to Rupert's house, he gave us a gift to take with us -- a copy of his book "Science Set Free".   Being a really nice guy as well as a brilliantly creative thinker, I'm sure Rupert will not be too annoyed at me for repaying his kind gift by writing this blog post, which criticizes some of his ideas while building on others! 

I skimmed the book shortly after receiving it, but only recently started reading through it more carefully.   The overall theme is a call for scientists to look beyond a traditional materialistic approach, and open their minds to the possibility that the universe is richer, more complex, and more holistic than materialist thinking suggests.   Morphic fields are mentioned here and there, as one kind of scientific hypothesis going beyond traditional materialism and potentially explaining certain data.

All this was also the topic of Sheldrake's controversial TEDx talk a couple years back, which was removed from the TEDx archive, apparently due to the controversial nature of Sheldrake's work in general.  For a lengthy online discussion of this incident, see this page….  As I said in my contribution to that discussion, I don't think they were right to remove his video from their archive.  I've heard far more out-there TEDx talks than Rupert's, so it's obviously not the contents of his talk that caused its removal -- it's his general reputation, which someone apparently decided would sully TED's reputation in some circles they values.   Urrrgh.   I generally think TED is great, but I don't like this decision at all.

In general I'm supportive of Rupert's call for science to be more open-minded, and to look beyond traditional materialist approaches.   To me, science is centrally about a process of arriving at agreement among a community of minds regarding which observations should be accepted as collectively valid, and which explanations should be accepted as simpler.  Nothing in this scientific process requires the assumption that matter is more primary than consciousness (for example).  Nor are the notions of a "morphic field", or of precognition or ESP etc., "unscientific" in nature.

The main problem with the morphic field theory as Sheldrake lays it out is, in my view, its imprecision.   From the view of science as a community process of agreeing which observations are collectively valid and which explanations are simple, an issue with Sheldrake's "morphic field" view is that it's not simple at all to figure out how to apply it to a given context.   Different scientists might well come up with very different, and perhaps mutually incompatible, ways of using it to explain a given set of observations, or predict future observations in a given context.  This fuzziness is a kind of complexity, which makes my personal scientific Occam's Razor heuristic uneasy.

For now, what I want to talk about are some of Rupert Sheldrake's comments on memory, in "Science Set Free."   This will segue into some wild-ass quasi-mathematical speculations on how one might go about formalizing the morphic field idea in a more rigorous way than has been done so far.

Memory Traces versus Morphic Fields

In Chapter 7 of "Science Set Free", Sheldrake contrasts two different theories of human memory -- the "trace theory", which holds that memories are embodies as traces in organisms' brains; and the morphic resonance theory, which holds that memories are contained in a morphic field.  Of course the trace theory is the standard understanding in modern neuroscience.   On the other hand, he quotes the neuroscientist Karl Pribram as supporting an alternative understanding,
"Pribram … thought of the brain as a 'waveform analyzer' rather than a storage system, comparing it to a radio receiver that picked up waveforms from the 'implicate order', rendering them explicate.   This aspect of his thinking was influenced by the quantum physicist David Bohm, who suggested that the entire universe is holographic, in the sense that wholeness is enfolded into every part.

According to Bohm, the observable or manifest world is the explicate or unfolded order, which emerges from the implicate or enfolded order.  Bohm thought that the implicate order contains a kind of memory.  What happens in one place is 'introjected' or 'injected' into the implicate order, which is potentially present elsewhere; thereafter when the implicate order unfolds into the explicate order, this memory affects what happens, giving the process very similar properties to morphic resonance.   In Bohm's words, each moment will 'contain a projection of the re-injection of the previous moments, which is a kind of memory; so that would result in a general replication of past forms' "

When I briefly spoke with Karl Pribram on these matters in 2006 (when at my invitation he spoke at the AGI-06 workshop in Bethesda, the initial iteration of the AGI conference series), he seemed a lot less definitive than Sheldrake on the "brain as antenna" versus "brain as storehouse of memories" issue, but on the whole the story he told me was similar to Sheldrake's summary.   Pribram was trying to view the brain as a quantum-mechanical system in a state of macroscopic quantum coherence (perhaps related to coherent states in water megamolecules in the brain, as conjectured by his Japanese collaborators Jibu and Yasue), and then to look at perception as involving some sort of quantum coupling between the brain and environment. 

I actually like the "implicate order" idea; and Bohm's late-life book "Thought as a System" had a huge impact on me.   The first version of my attempt to formalize a theory of psi phenomena --  Morphic Pilot Theory -- was inspired by both morphic fields and Bohm's pilot wave theory of quantum mechanics (though the end part of this blog post presents some ways in which I'm recently trying to go beyond the particulars of that formulation).

However, I really can't buy into Sheldrake's rejection of the massive corpus of neurobiological evidence in favor of what he calls the "trace theory."   There is just a massive amount of evidence that, in a fairly strong sense, an awful lot of memories ARE actually stored "in the brain."

As just one among many examples, I recently looked through the literature on "concept neurons" -- neurons that fire when a person sees a certain face (say, Jennifer Aniston, in the common example).   But there are hundreds of other examples where neuroscientists have figured out which neurons or neuronal subnetworks become active when a given memory is recalled….   The idea that the brain is more like a radio receiver (receiving signals from the morphic field) than a storehouse of information, seems to me deeply flawed.

Sheldrake says
"The brain may be more like a television set than a hard drive recorder.   What you see on TV depends on the resonant tuning of the set to invisible fields.  No one can find out today what programs you watched yesterday by analyzing the wires and transistors in your TV set for traces of yesterday's programs."

While I salute the innovative, maverick thinking underlying this hypothesis, I definitely can't agree.  I very strongly suspect that you COULD tell what TV program a person watched yesterday, by analyzing their brain's connectome.  We can't carry out this exact feat yet,but I bet it will be possible before too long.  We can already tell what a person is looking at via reading out information from their visual cortex, for example.

The main point I want to make here, though, is that one doesn't have to view the trace theory of memory and (some form of) the morphic field theory of memory as contradictory.

The brain, IMO, is plainly not much like a radio receiver or antenna -- it does contain specific neurons, specific subnetworks and specific dynamical patterns that correlate closely with specific memories of various sorts.   Neuroscience data says this and we have to listen.

However, this doesn't rule out the possibility that some sort of "morphic field" could also exist, and could also play a role in memory.

Pattern Completion and Morphic Fields

It seems to me that a better analogy than a radio receiver, would be pattern completion in attractor neural networks.

In a Hopfield neural net, one "trains" the network by exposing it to a bunch of memories (each one of which is a pattern of activity across the network, in which some neurons are active and others are not).   Then, once the network is trained, if one exposes the network to PART of some memory, the nonlinear dynamics of activation flowing through the neural net will cause the whole memory to emerge in the network.   The following figure illustrates this in some simple examples.

Figure illustrating neural net based pattern completion, borrowed from [Ritter, H., Martinetz, Th., Schulten, K. (1992): Neural Computation and Self-organizing Maps. Addison Wesley,].  (a) The Hopfield net consists of 20 x 20 neuroids, which can show two states, illustrated by a dot or a black square, respectively. The weights are chosen to store 20 different patterns; one is represented by the face, the other 19 by different random dot patterns. (b) After providing only a part of the face pattern as input (left), in the next iteration cycle the essential elements of the final pattern can already be recognized (center), and the pattern is completed two cycles later (right). (c) In this example, the complete pattern was used as input, but was disturbed by noise beforehand (left). Again, after one iteration cycle the errors are nearly corrected (center), and the pattern is complete after the second iteration (right)

What does this have to do with morphic fields?  

My suggestion is that, potentially, the trace of a memory in an organism's brain, could be considered as a PART of the totality of that memory in the universe.  The nonlinear dynamics of the universe could be such that: When the PART of a memory existing in an organism's brain is activated, then via a pattern-completion type dynamic, the rest of the memory is activated. 

Furthermore, if some memory is activated in the broader universe, then the nonlinear dynamics coupling the rest of the universe with the organism's brain, could cause a portion of that memory to form within the organism's brain.

In the analogy I'm suggesting here, the analogue of the whole Hopfield neural network in which the overall memory would be activated, would be some form of "morphic field." 

In this hypothetical model, the portion of the "universal nonlinear dynamical system" that resides in an organism's brain is not behaving much like an antenna.  It's not just tuning into channels and receiving what is broadcast on them.  Rather, in this model, the brain stores its own memory-fragments and has its own complex dynamics for generating them, modifying them, revising them, and so forth.  But these memory-fragments are nonlinearly coupled with broader memory patterns that exist in a nonlinear-dynamical field that goes beyond the individual organism's rain and body.

In sum, the idea I'm proposing is that
  • a morphic field may be modeled as a nonlinear self-organizing network, including material entities like brains and bodies as a portion
  • memories may be viewed as patterns spread across large portions of a morphic field
  • the portion of a memory that is resident in an organism's brain as a "memory trace" may be viewed as a "memory fragment" from a morphic field perspective; and may   trigger a broader memory to emerge across the morphic field via "pattern completion" type dynamics
  • the emergence of a broader memory across the morphic field, may cause certain memory-fragments to emerge in an organism's brain

This seems a consistent, coherent way to have both morphic fields AND standard neurobiological memory traces.

I'm not claiming to have empirical evidence for this (admittedly out-there and eccentric) perspective on memory.  Nor am I claiming that this constitutes a precise, rigorous, testable hypothesis.   It doesn't.  All I'm trying to do in this post is articulate a conceptual approach that makes the morphic field hypothesis consistent with the almost inarguably strong observation that neural memory traces are real and are powerfully explanatory regarding many aspects of human memory. 

Morphic Fields and Psi, Once Again

Ah -- OK but, what aspects of memory would one need to invoke these broader-memory morphic fields to explain?

It's possible that morphic fields play a small but nontrivial role in a wide variety of memory phenomena, across the board.  This would fit in with Jim Carpenter's theories in his book First Sight, which argues that weak psi phenomena underlie our intuitive understandings of everyday situations.

And it's also possible that one thing distinguishing psi phenomena from ordinary cognition, is a heavy reliance on the morphic-field components of memories.

To turn these vague conceptual notions into really useful scientific theories, would require a more rigorous theory of how morphic fields work.  I have some thoughts along those lines but will save a full, detailed exposition of these for another time.  For now I'll just give a little hint...

How Might One Model Morphic Fields?

OK, now I'm going to go even further "out there", alongside with getting a bit more technical...

A model of morphic fields has to exist within some model of the universe overall.

Existing standard physics models don't seem to leave any natural place for morphic fields.  However, existing standard physics models are also known to be inadequate to explain known physical data in a coherent, self-consistent way (as e.g. general relativity and quantum field theory haven't yet been unified into a single theory).   This certainly gives some justification for looking beyond the standard physics approaches, in searching for a world-model that is conceptually compatible with morphic fields.

The basic ideas I'll outline here could actually be elaborated within many different approaches to theoretical physics.  However, they are easiest and most natural to elaborate in the context of discrete models of the universe -- so that's the route I'll take here.   Discrete models of the universe have been around a while, e.g. the Feynman Checkerboard and its descendants.

One of the more interesting discrete approaches to foundational physics is Causal Sets.  Basically, in causal set theory, "spacetime" is replaced by a network of nodes interconnected by directed edges.   A directed edge indicates an atomic flow of causality.

I suspect it may be interesting to extend the causal set approach into what I call a "causal web" -- in which directed hyperlinks span triples of nodes.  A hyperlink pointing from (A,B) to C indicates a flow of causality from the pair (A,B) to C.   Local field values at A and local field values at B then combine to yield local field values at C.   This combination may be represented as multiplication in some algebra, so one can write F_C(t+1) = F_A(t) * F_B(t), where t refers to a sort of "meta-time" or "implicate time", distinct from the time axis that forms part of the spacetime continuum we see. 

Figuring out the right way to represent electromagnetic and quark fields this way is an interesting line of research, which I've been playing with occasionally in recent weeks.   Gravitation, on the other hand, I would suggest to represent more statistically, as an "entropic force" of a sort arising emergently from dynamics on the causal web.   I'll write another post about that later.

(More broadly, I think one could show that continuous field theories, within fairly broad conditions, can be emulated by causal webs within arbitrarily small errors.    Conceptually, causal webs are a bit like discrete reaction-diffusion equations; and it's known that discrete reaction-diffusion equations can be mapped into discrete quantum field theories.)

The main point I want to explore here is how one might get some sort of morphic field to emerge from this sort of framework.  Namely: One could potentially do so by positing a field, living at the nodes in the causal web, which is acausal in nature, and propagates symmetrically, flowing both directions along directed links.  This would be a "pattern field."   

Imagine running hypergraph pattern mining software - like, say, OpenCog's Pattern Miner -- on a causal web.  This would result in a large index, indicating which patterns occur how often in the web.  Atoms and molecules would emerge as pretty frequent patterns, for example; as would radioactive decay events.  Spatial, temporal and spatiotemporal patterns would be definable in this way.

Each node in the causal web can then be associated with a "pattern set" indicating the frequent patterns that it belongs to, indexed by their frequency (and perhaps by other quantities, such as their surprisingness), and retaining information regarding what slot in the pattern the current node fits into.

One can then view these pattern sets as comprising additional nodes and links, to be added to the web.  Two nodes that are part of the same pattern, even if distant spatiotemporally, would then be linked together by the nodes and links comprising the pattern.  These are non-causal links, representing commonality of pattern, independent of spatiotemporal causality.

Given this framework, we can introduce an additional dynamic: a variant of what philosopher Charles Peirce called "the tendency to take habits."   Namely, we can posit that: Patterns that have a high surprisingness value are more likely to persist in the causal web.  

By "surprisingness value" I mean here that the pattern is more probable than one would infer from looking at its component parts.  As a first hypothesis one can use the I-surprisingness as defined in OpenCog's pattern mining framework.

Among other things, this implies that: When one instance of pattern P is linked with an instance of pattern Q, this increases the odds that another instance of pattern P is linked with some instance of pattern Q. 

Or, a little differently, this "Surprising Multiverse" theory could be viewed as a variation of the Jungian notion of "synchronicity" -- which basically posits that meaningful combinations of events may occur surprisingly often, due to some sort of acausal connecting principle.  (As an aside, I actually first learned about Synchronicity from the Police album way back when -- thanks, Sting!)

Viewed in quantum-theoretic terms, this is a statement about the amplitude (complex probability) distribution over possible causal webs (or more properly, actually: over possible histories of causal webs, where a causal web history is defined as a series of causal-web states so that each one is consistent with the previous and subsequent according to causal web dynamics....  If a causal web is deterministic then each causal web corresponds with just one causal web history, but we don't need to assume this.)   It is a statement that causal web histories with more surprising patterns, should be weighted higher when doing Feynman sums used to determine what happens in the world.

How does a pattern completion type dynamic happen, then, in this perspective?  Suppose that, in a particular part of the causal web, a certain pattern emerges.  The existence of this pattern influences the surprisingness values of other pattern-instances, situated other places in the web.  It thus influences the weightings of Feynman sums occurring all around the web, thus influencing the probabilities of various events.

We thus have a non-local, acausal connecting principle: the surprising-pattern-based weighting of possible causal web histories in Feynman sums.  The "morphic field" is then modeled, not exactly as a "field", but as a multiverse-wide , ongoing dynamic re-weighting of possible universes according to the surprisingness of the patterns they contain (noting that the surprisingness of a universe changes over time as it evolves).   (And note also that nothing is literally represented as a "field" in the causal web approach; fields are replaced in this model by discrete dynamics on hypergraphs representing pre-geometric structures below the level of spacetime.)
For example, suppose one identical twin falls in love with a brown-haired dentist.  There are possible universes (causal web histories) in which both twins fall in love with brown-haired dentists, and others in which only one does, and the other falls in love with a green-haired chiropodist or whatever.  The universes in which both twins fall in love with brown-haired dentists will have an additional surprising pattern as compared to the other universes, and hence will be weighted slightly higher.

Or, suppose a woman's brain remembers what she watched on TV last night.  Again, it will be more surprising, probabilistically, if others know this as well -- so the universes in which others do, will be weighted slightly higher.

Now, there are many different ways to measure surprisingness, so that this approach to more formally specifying the morphic field hypothesis must be considered a research direction rather than a definite theory.  All I'm suggesting here is that it's an interesting direction.

When digging into the details of these ideas, an important thing to think about is: Surprising to whom?  Based on whose expectations?  Surprising to the universe?  Or surprising to some particular observer?  In the relational interpretation of quantum theory, all observations occur relative to some observer -- so this is probably the best way to think about it.

The decline effect -- in which psi experiments start to decay in effectiveness after some time has passed -- begins to seem perhaps conceptually explicable in this framework.   Once a psi phenomenon has been demonstrated enough times, to a given class of observers, it fails to be surprising to them, so it fails to be weighted higher in the relevant Feynman sums and doesn't happen anymore.   (Indeed this is extremely hand-wavy, but as I already emphasized, I'm just trying to point in an interesting direction!)

It's also worth noting that one could also extend the sum over causal webs that are inconsistent in terms of temporal direction.  That is, causal webs containing circular arrow structures.  What would likely happen in this case is that, as you add up the amplitudes of all the different causal webs, the causally inconsistent ones tend to cancel each other out, and the overall sum is dominated by the causally consistent ones.  However, this wouldn't be guaranteed to happen, and the surprise bias could in some cases intersect interestingly with this phenomenon, enabling circularly-causal webs to "occasionally" dominate the amplitude sum.

Anyway, I've certainly raised more questions than I've answered here.   But perhaps I've convinced some tiny fraction of readers that there is some hope, by modifying existing (admittedly somewhat radical) physics models, to come up with a coherent formal model of morphic fields.  Getting back to issues of memory, my feeling is that such a formal model is likely to yield a "pattern completion" type theory of morphic memory, rather than a "television receiver" type theory.

In Praise of Wild Wacky Weirdness ... and Data

I've spun out some wacky ideas here, and probably weirded out a lot of readers, but so it goes!   One of the messages of Sheldrake's book "Science Set Free" that I really like is (paraphrasing): Open your mind and rethink every issue from first principles.  Just try to understand, and don't worry so much about agreeing with prevailing points of view; after all, prevailing points of view have been proved wrong many times throughout history.  The ideas given here are presented very much in that spirit.

Another key message of Sheldrake's book, however, is (paraphrasing again): Do pay attention to data.  Look at data very carefully.  Design your own experiments to explore your hypotheses, gather your own data, and study it.   This is one of the next important steps in exploring the ideas presented here.  How could this sort of formalized morphic field explain the various data collected in "The Evidence for Psi", for example?

The journey continues...

Wednesday, September 24, 2014

Semihard Takeoff

Whenever I talk about the future of AGI, someone starts talking about the possibility that AGI will "take over the world."

One question is whether this would be a good or bad thing -- and the answer to that is, of course, "it depends" ... I'll come back to that at the end of this post.

Another relevant question is: If this were going to happen, how would it most likely come about.  How would an "AGI takeover" be likely to unfold, in practice?

One option is what Eliezer Yudkowsky has called AI "FOOM" ...  i.e. a "Hard Takeoff"  (a possibility which I analyzed a bit , some time ago...)

The basic idea of AI Foom or Hard Takeoff is that, sometime in the future, an advanced AGI may go from relatively innocuous subhuman-level intelligence all the way up to superhuman intelligence, in 5 minutes or some other remarkably short period of time.....  By rewriting its code over and over (each time learning better how to rewrite its code), or assimilating additional hardware into its infrastructure, or whatever....

A Hard Takeoff is a special case of the general notion of an Intelligence Explosion -- a process via which AGI gets smarter and smarter via improving itself, and thus getting better and better and faster and faster at making itself smarter and smarter.   A Hard Takeoff is, basically,  a really really fast Intelligence Explosion!

Richard Loosemore and I have argued that an Intelligence Explosion is probable.   But this doesn't mean a Hard Takeoff is probable.

Nick Bostrom's nice illustration of the Hard Takeoff idea

What often seems to happen in discussions of the future of AI (among hardcore futurist geeks, anyway) is something like:

  • Someone presents the Foom / Hard Takeoff idea as a scary, and reasonably likely, option
  • Someone else points out that this is pretty unlikely, since someone watching the subhuman-level AGI system in question would probably notice if the AGI system were ordering a lot of new hardware for itself, or undertaking unusual network activity, or displaying highly novel RAM usage patterns, or whatever...

In spite of being a huge optimist about the power and future of AGI, I actually tend to agree with the anti-Foom arguments.   A hard AGI takeoff in 5 minutes seems pretty unlikely to me.  

What I think is far more likely is an Intelligence Explosion manifested as a "semi-hard takeoff" -- where an AGI takes a few years to get from slightly subhuman level general intelligence to massively superhuman intelligence, and involved various human beings, systems and institutions in the process.

A tasty semihard cheese -- appropriate snack food 
for those living through the semihard takeoff to come
Semihard cheeses are generally good for melting;
and are sometimes said to have the greatest complexity and balance. 

After all, a cunning and power-hungry human-level AGI wouldn't need to suddenly take over the world on its own, all at once, in order to gain power.  Unless it was massively superhuman, it would probably consider this too risky a course of action.   Rather, to take power, a human-level AGI would would simply need to accumulate a lot of money (e.g. on the financial markets, using the superior pattern recognition capability it could achieve via tightly integrating its mind with statistical and machine learning software and financial, economic and news databases) and then deploy this wealth to set up a stronghold in some easily-bought nation, where it could then pay and educate a host of humans to do its bidding, while doing research to improve its intelligence further...

Human society is complex and disorganized enough, and human motivations are complex and confused enough, and human judgment is erratic enough, that there would be plenty of opportunities for an early-stage AGI agent to embed itself in human society in such a way as to foster the simultaneous growth of its power and intelligence over a period of a few years.   In fact an early-stage AGI probably won't even need to TRY for this to happen -- once early-stage AGI systems can do really useful stuff, various governments, companies and other organizations will push pretty hard to use these systems as thoroughly as they can, because of the economic efficiency and scientific and media status this will bring.

Once an AGI is at human level and embedded in human society in judicious ways, it's going to be infeasible for anyone to get rid of it -- and it's going to keep on growing in intelligence and power, aided by the human institutions it's linked with.   Consider, e.g., a future in which

  •  Azerbaijan's leaders get bought off by a wildly successful AGI futures trader, and the nation becomes an AGI stronghold, complete with a nuclear arsenal and what-not (maybe the AGI has helped the country design and build nukes, or maybe it didn't need the AGI for that...).   
  • The nation the AGI has bought is not aggressive, not attacking anyone -- it's just sitting there using tech to raise itself out of poverty ... doing profitable deals on the financial markets, making and selling software products/services, patenting inventions, ... and creating a military apparatus for self-defense, like basically every other country.   

What happens then?  The AGI keeps profiting and self-improving at its own pace, is what happens?  Is the US really gonna nuke a peaceful country just for being smart and getting rich, and risk massive retaliation and World War III?  I doubt it....  In its comfy Azerbaijani stronghold, the AGI can then develop from human-level to massively transhuman intelligence -- and then a lot of things become possible...

I have spun out one scenario here but of course there are lots of others.  Let's not allow the unrealism of the "hard takeoff in 5 minutes and the AGI takes over the world" aka "foom" scenario to blind our minds to the great variety of other possibilities....  Bear in mind that an AGI going from toddler-level to human-level in 5 years, and human-level to superhuman level in 5 more years, is a FOOM on the time-scale of human history, even if not as sudden as a 5 minute hard takeoff on the time-scale of an individual human life...

So how could we stop a semihard takeoff from happening?   We can't really -- not without some sort of 1984++ style fascist anti-AI world dictatorship, or a war destroying modern society projecting us back before the information age.   And anyway, I am not in favor of throttling AGI development personally; I doubt the hypothetical Azerbaijanian AGI would particularly want to annihilate humanity and I suspect transhuman AGIs will do more good than harm, on average over all possible worlds....  I'm not at all sure that "an AGI taking over the world" -- with the fully or partly witting support of some group(s) of humans -- would be a bad thing, compared to other viable alternatives for humanity's future....

In terms of risks to humanity, this more realistic "semihard takeoff" development scenario highlights where the really onerous risks probably are.   SIAI/MIRI and the Future of Humanity Institute seem to spend a lot of energy thinking about the risk of a superhuman AGI annihilating humanity for its own reasons; but it seems to me a much more palpable and probable risk will occur at the stage where an AGI is around human-level but not yet dramatically more powerful and intelligent than humans, so that it still needs cooperation from human beings to get things done.  This stage of development will create a situation in which AGI systems will want to strike bargains with humans, wherein they do some things that certain humans want, in order to get some things that they want...

But obviously, some of the things that some humans want, are highly destructive to OTHER humans...

The point is, there is a clear and known risk of early-stage AGIs being manipulated by humans with nasty or selfish motives, because many humans are known to have nasty or selfish motives.   Whereas the propensity of advanced AGIs to annihilate lesser sentiences, remains a wild speculation (and one that I don't really find all that credible).....

I would personally trust a well-designed, self-improving AGI more than a national government that's in possession of the world's smartest near-human-level AGI; AGIs are somewhat of a wild card but can at least be designed with initially beneficent motivational systems, whereas national governments are known to generally be self-serving and prone to various sorts of faulty judgments....  This leads on to the notion of the AI Nanny, which I've written about before.   But my point here isn't to argue the desirability or otherwise of the AI  Nanny -- just to point out the kind of "semihard takeoff" that I think is actually plausible.

IMO what we're likely to see is not a FOOM exactly, but still, a lot faster than AI skeptics would want to accept....   A Semihard Takeoff.  Which is still risky in various ways, but in many ways more exciting than a true Hard Takeoff -- because it will happen slowly enough for us to watch and feel it happen....