Quantum Logic Models of Classical Systems Give a New Twist on Quantum Consciousness

A while ago I wrote a blog post suggesting that quantum logic should be applied more generally than to quantum physical systems ... that it should be applied to complex classical systems in some cases as well, if they are so complex that their states are unobservable to a certain observer.  

This, I suggested, would require making the choice of logic observer-dependent: i.e., the system T might best be modeled by system S using quantum logic, but by system R using classical logic.

I didn't at the time see how to make this speculation rigorous but I've now found a related literature that helps a lot.

And by refining my previous idea, I've come up with an argument that possibly human consciousness may be effectively modeled using quantum logic, whether or not the human brain is a quantum system.

I may write a paper on this stuff at some point (in which process I'll probably figure out nicer ways to express the ideas), but wanted to write it down now while it's fresh in my mind.

Atmanspacher's Idea

Diederik Aerts and Liane Gabora have written some very nice papers related to this topic ... and I read their stuff years ago but didn't quite see how to connect it to my relevant intuitions.   

What I discovered just recently was the related work of Harald Atmanspacher, which ties in more directly with the way I was thinking about these issues.  (Some relevant papers by both of these guys are linked to at the end of this post.)

Put simply, Atmanspacher's view is that: In any case where two properties of a system cannot be simultaneously measured with high accuracy, you have a situation that should be modeled using quantum logic.

I.e., quantum logic should be applied to any case where there are incompatible observables ... whether or not this is due to quantum microphysics.

Making the Choice of Logic Observer-Dependent

My twist on Atmaspacher's idea is to suggest that quantum logic should be applied, by a cognitive system, to any situation that has two aspects which (perhaps by quantum microphysics, or perhaps simply due to its limitations as a cognitive system) the cognitive system cannot model simultaneously. 

That is: If T has two aspects, and S cannot model these two aspects of T simultaneously without becoming non-S, then from the perspective of S, these aspects of T should be modeled using quantum logic rather than classical logic.

Note that S in this argument is not a specific physical system at a particular point in time, but rather a category of instantaneous physical systems, which are being considered as instantiations of a single abstract "system" (for example, "Ben Goertzel" is a category of instantaneous physical systems).  

So, my suggestion is that whether T should be reasoned about by quantum or classical logic, must be determined by relativizing the reasoning to some category of instantaneous physical systems.

Possible Implications for Quantum Consciousness

What spurred me to start digging into these issues just now was a conversation with Stuart Hameroff, who believes consciousness to be a quantum phenomenon.  

My suggestion is: It could be that a quantum model of human consciousness is the right one, even if the underlying physics of the brain is basically "classical" (and I don't claim to know for sure whether it is or not).

(Note that I referred above to a classical model of "human consciousness", not of consciousness in general -- I tend toward panpsychism, meaning I think everything is conscious and different systems just manifest universal consciousness in different ways.)

As a natural consequence of the above argument, I would suggest that each of us individually, due to our own processing limitations, cannot view ourselves in all aspects simultaneously. 

If this is true, then perhaps we should model ourselves using quantum logic.  

Being panpsychist I would not identify this with consciousness, but I would say that systems which are sufficiently complex that they implicitly model themselves using quantum logic, in predicting and analyzing their own dynamics, presumably have a distinctive character to the way they manifest universal consciousness.

Can We Tell the Cause of the Incompatibility?

An interesting question is: if I, as a cognitive system, am confronted with incompatible observables ... in what sense can I tell what the cause of this incompatibility is?

Can I tell a case where the incompatibility is caused by my own cognitive limitations, from a case where it is caused by fundamental indeterminacy such as is sometimes hypothesized to occur in quantum microphysics?

It would seem there is no direct way to make this determination, but we can induce general theories from observations of other system aspects, which lead us to hypotheses regarding the causes of an incompatibility.

Some Nice Quotes on Quantum Modeling of Classical Systems

These quotes come from

http://www.quantum-cognition.de/

Andrei Khrennikov:

“I propose to consider any system which produces quantum statistics as quantum (”quantum-like”). A possible test is based on the interference of probabilities. I was mainly interested in using such an approach to ”quantumness” to extend the domain of applications of quantum mathematical formalism and especially to apply it to cognitive sciences. There were done experiments on interference of probabilities for ensembles of students and a nontrivial interference was really found. … Yes, we might expect nonclassical statistics, but there was no reason to get the quantum one, i.e., cos-interference. But we got it!”

Diederik Aeerts & Liane Gabora:

"While some of the properties of quantum mechanics are essentially linked to the nature of the microworld, others are connected to fundamental structures of the world at large and could therefore in principle also appear in other domains than the micro-world."

Diederik Aeerts:

"The emergence of quantal macrostates does not necessarily require the reference to corresponding quantal microstates" 

Harald Atmanspacher, Hans Primas & Peter beim Graben:

"A generalized version of the formal scheme of ordinary quantum theory, in which particular features of ordinary quantum theory are not contained, should be used in some non-physical contexts."

"Complementary observables can arise in classical dynamic systems with incompatible partitions of the phase space."

A Few Relevant References

Atmanspacher:

"Weak quantum theory"

http://www.igpp.de/english/tda/pdf/wqt.pdf

"Complementarity in Bistable Perception"

http://unjobs.org/authors/harald-atmanspacher

Aerts:

At

http://www.vub.ac.be/CLEA/aerts/publications/

see

• Aerts, D. (1982). Example of a macroscopical situation that violates Bell inequalities. Lettere al Nuovo Cimento, 34, pp. 107-111. 
  
• Aerts, D. (1991). A mechanistic classical laboratory situation violating the Bell inequalities with 2sqrt(2), exactly 'in the same way' as its violations by the EPR experiments. Helvetica Physica Acta, 64, pp. 1-23. 
  
• Aerts, D. and Gabora, L. (2005). A theory of concepts and their combinations I: The structure of the sets of contexts and properties. Kybernetes, 34, pp. 167-191.
  
• Aerts, D. and Gabora, L. (2005). A theory of concepts and their combinations II: A Hilbert space representation. Kybernetes, 34, pp. 192-221
  

How many transhumanists does it take to change a light bulb?

Infinity.... None of them will touch the light bulb at all; they'll all just sit around talking amongst themselves and waiting for someone else to invent a self-changing cyber light bulb.

Reinforcement Learning: Some Limitations of the Paradigm

(This email summarizes some points I made in conversation recently with an expert in reinforcement learning and AGI. These aren't necessarily original points -- I've heard similar things said before -- but I felt like writing them down somewhere in my own vernacular, and this seemed like the right place....)

Reinforcement learning, a popular paradigm for AI, economics and psychology, models intelligent agents as systems that choose their actions in such a way as to maximize their future reward. There are various ways of averaging future reward over various future time-points, but all of these implement the same basic concept.

I think this is a reasonable model of human behavior in some circumstances, but horrible in others.

And, in an AI context, it seems to combine particularly poorly with the capability for radical self-modification.


Reinforcement Learning and the Ultimate Orgasm

Consider for instance the case of a person who is faced with two alternatives

• A: continue their human life as would normally be expected
• B: push a button that will immediately kill everyone on Earth except them, but give them an eternity of ultimate trans-orgasmic bliss

Obviously, the reward will be larger for option B, according to any sensible scheme for weighting various future rewards.

For most people, there will likely be some negative reward in option B ... namely, the guilt that will be felt during the period between the decision to push the button and the pushing of the button. But, this guilt surely will not be SO negative as to outweigh the amazing positive reward of the eternal ultimate trans-orgasmic bliss to come after the button is pushed!

But the thing is, not all humans would push the button. Many would, but not all. For various reasons, such as love of their family, attachment to their own pain, whatever....

The moral of this story is: humans are not fully reward-driven. Nor are they "reward-driven plus random noise".... They have some other method of determining their behaviors, in addition to reinforcement-learning-style reward-seeking.


Reward-Seeking and Self-Modification: A Scary Combination

Now let's think about the case of a reward-driven AI system that also has the capability to modify its source code unrestrictedly -- for instance, to modify what will cause it to get the internal sensation of being rewarded.

For instance, if the system has a "reward button", we may assume that it has the capability to stimulate the internal circuitry corresponding to the pushing of the reward button.

Obviously, if this AI system has the goal of maximizing its future reward, it's likely to be driven to spend its life stimulating itself rather than bothering with anything else. Even if it started out with some other goal, it will quickly figure out to get rid of this goal, which does not lead to as much reward as direct self-stimulation.

All this doesn't imply that such an AI would necessarily be dangerous to us. However, it seems pretty likely that it would be. It would want to ensure itself a reliable power supply and defensibility against attacks. Toward that end, it might well decide its best course is to get rid of anyone who could possibly get in the way of its highly rewarding process of self-stimulation.

Not only would such an AI likely be dangerous to us, it would also lead to a pretty boring universe (via my current aesthetic standards, at any rate). Perhaps it would extinguish all other life in its solar system, surround itself with a really nice shield, and then proceed to self-stimulate ongoingly, figuring that exploring the rest of the universe would be expected to bring more risk than reward.

The moral of the above, to me, is that reward-seeking is an incomplete model of human motivation, and a bad principle for control self-modifying AI systems.


Goal-Seeking versus Reward-Seeking

Fortunately, goal-seeking is more general than reward-seeking.

Reward-seeking, of the sort that typical reinforcement-learning systems carry out, is about: Planning a course of action that is expected to lead to a future that, in the future, you will consider to be good.

Goal-seeking doesn't have to be about that. It can be about that ... but it can also be about other things, such as: Planning a course of action that is expected to lead to a future that is good according to your present standards.

Goal-seeking is different from reward-seeking because it will potentially (depending on the goal) cause a system to sometimes choose A over B even if it knows A will bring less reward than B ... because in foresight, A matches the system's current values.


Non-Reward-Based Goals for Self-Modifying AI Systems

As a rough indication of what kinds of goals one could give a self-modifying AI, that differ radically from reward-seeking, consider the case of an AI system with a goal G that is the conjunction of two factors:

• Try to maximize the function F
• If at any point T, you assess that your interpretation of the goal G at time T would be interpreted by your self-from-time-(T-S) as a terrible thing, then roll back to your state at time S

I'm not advocating this as a perfect goal for a self-modifying AI. But the point I want to make is this kind of goal is something quite different from the seeking of reward. There seems no way to formulate this goal as one of reward maximization. This is a goal that involves choosing a near-future course of action to maximize a certain function over future history -- but this function is not any kind of summation or combination of future rewards.


Limitations of the Goal-Seeking Paradigm

Coming at the issue from certain theoretical perspectives, it is easy to overestimate the degree to which human beings are goal-directed. It's not only AI theorists and engineers who have made this mistake; many psychologists have made it as well, rooting all human activity in goals like sexuality, survival, and so forth. To my mind, there is no doubt that goal-directed behavior plays a large role in human activity -- yet it also seems clear that a lot of human activity is better conceived as "self-organization based on environmental coupling" rather than as explicitly goal-directed.

It is certainly possible to engineer AI systems that are more strictly goal-driven than humans, though it's not obvious how far one can go in this direction without sacrificing a lot of intelligence -- it may be that a certain amount of non-explicitly-goal-directed self-organization is actually useful for intelligence, even if intelligence itself is conceived in terms of "the ability to achieve complex goals in complex environments" as I've advocated.

I've argued before for a distinction between the "explicit goals" and "implicit goals" of intelligent systems -- the explicit goals being what the system models itself as pursuing, and the implicit goals being what an objective, intelligent observer would conclude the system is pursuing. I've defined a "well aligned" mind as one whose explicit and implicit goals are roughly the same.

According to this definition, some humans, clearly, are better aligned than others!

Summary & Conclusion

Reward-seeking is best viewed as a special case of goal-seeking. Maximizing future reward is clearly one goal that intelligent biological systems work toward, and it's also one that has proved useful in AI and engineering so far. Thus, work within the reinforcement learning paradigm may well be relevant to designing the intelligent systems of the future.

But, to the extent that humans are goal-driven, reward-seeking doesn't summarize our goals. And, as we create artificial intelligences, there seems more hope of creating benevolent advanced AGI systems with goals going beyond (though perhaps including) reward-seeking, than with goals restricted to reward-seeking.

Crafting goals with reasonable odds of leading self-modifying AI systems toward lasting benevolence is a very hard problem ... but it's clear that systems with goals restricted to future-reward-maximization are NOT the place to look.

Science-synergetic philosophy: the religion of the future?

(This may seem a hackneyed topic, but there are some moderately original points near the end here, if you bear with me ...)

As a card-carrying, future-thinking transhumanist, I take it as obvious that most of the particulars of current religions are relics of earlier eras in human cultural development, which currently do a lot of harm along with doing some good.

But I still find it interesting to ask what aspects of religion reflect underlying phenomena that are essential, meaningful and necessary -- and are likely to continue as humanity transcends the traditional "human condition" and enters its next phase of development....


Fish and Eagleton on the Wonders of Theology

What spurred this blog post was: My dad pointed out to me this New York Times blog post by Stanley Fish reviewing a book that extols the merits of religion (Reason, Faith and Revolution by Terry Eagleton).

The basic point Fish makes is that religion offers something science by its very nature cannot.



Eagleton acknowledges ... many terrible things have been done in religion’s name — but at least religion is trying for something more than local satisfactions, for its “subject is nothing less than the nature and destiny of humanity itself, in relation to what it takes to be its transcendent source of life.”


He notes that science cannot address what he calls "theological questions", where


By theological questions, Eagleton means questions like, “Why is there anything in the first place?”, “Why what we do have is actually intelligible to us?” and “Where do our notions of explanation, regularity and intelligibility come from?”


He also notes that the author is


... angry, I think, at having to expend so much mental and emotional energy refuting the shallow arguments of school-yard atheists like Hitchens and Dawkins.


I haven't read Eagleton's book and I'm unlikely to do so -- I have a long list of more interesting-looking reading material -- but Fish's summary did resonate with a paper I'm in the middle of writing (it's paused while I work on more urgent stuff) on the limits of science.

My basic point in that paper will be a simple one: science is based on finite sets of finite-precision observations. That is, all of scientific knowledge is based on some finite set of bits, comprising the empirical observations accepted by the scientific community.

To extrapolate beyond this bit-set, some kind of assumption is needed. To put it another way, some kind of "faith" is needed. Hume was the first one to make this point really clearly ... and we now understand the "Humean problem of induction" well enough to know it's not the kind of thing that can be "solved."

The Occam's Razor principle tries to solve it -- it says that you extrapolate from the bit-set of known data by making the simplest possible hypothesis. This leads to some nice mathematics involving algorithmic information theory and so forth. But of course, one still has to have "faith" in some measure of simplicity!

So: doing or using science requires, in essence, continual acts of faith (though these may be unconscious and routinized rather than conscious and explicit). To the extent that Dawkins, Hitchens or other anti-religion commentators de-emphasize this point, they're engaging in judicious marketing. (It's hard for me to feel too negative toward them about this, however, given the far more explicitly and dramatically dishonest marketing that religion has carried out over the last millennia.)

My paper will focus on what the limits of science tell you about AI, machine consciousness and so forth -- and I'll save that for another blog post, or the paper itself. (Don't worry though, my conclusion is not that scientifically enginering AGI is impossible ... I haven't lost the faith!)

Anyway, I certainly agree with Fish and Eagleton that religion addresses very important questions that science cannot, by its nature, answer.

But I find it rather screwy that Eagleton refers to


“Why is there anything in the first place?”, “Why what we do have is actually intelligible to us?” and “Where do our notions of explanation, regularity and intelligibility come from?”


and so forth as theological questions.

Surely, these are philosophical questions.

One can answer them in various ways without invoking any deities or demons!

"Why does God exist?" is a theological question ...

"Why does anything exist?" is philosophical...

(Though, for the record, I don't think "Why does anything exist?" is a very useful philosophical question. I'm more interested in questions like

• "Why do separate objects exist, instead of just one big fluid cosmic mass?"
• "In what sense could the universe be considered compassionate?"
• "How much ethical responsibility should I feel toward (which) other minds?"
  
• "Why does my mind perceive such a small subset of the space of all possible patterns?"
• "How much can a mind grow and expand without losing its sense of self and becoming, experientially, a 'fundamentally different being'?"
• "What is it like to be a rock?"
• etc.

)

Theology is one way of providing answers to philosophical questions ... but by no means the only way.

I think that religion addresses some very important questions, that are beyond the scope of science -- and by and large provides these questions with extremely bad answers.

One of the many limitations of religion as conventionally conceived is indicated by the quote, given above, that religion's


“subject is nothing less than the nature and destiny of humanity itself....”


From a transhumanist perspective, the qualifier "nothing less than" is misplaced, as this is actually a very limiting subject. The nature and destiny of humanity are important; but one of the things that science has opened our minds to is the relative insignificance of humanity in the space of possible minds. I'm more interested in philosophies that address the nature and destiny of mind itself, rather than just the nature and destiny of one species on one planet.

It is of course a subtle matter to compare and judge different explanations to philosophical questions. You can't compare them using scientific or mathematical methods ... and of course the question of how to evaluate philosophical views becomes "yet another tough philosophical question", tied in with all the other ones.

A crude way to say it, is that it comes down to an intuitive judgment ... which leads into questions of how one can refine and improve one's intuition ... and these questions, of course, possess numerous answers that are philosophical- or religious- tradition -dependent...


Science-synergetic philosophy


It does seem to me, though, that there is an interesting notion of science-synergetic philosophy lurking somewhere in all this.

Suppose we take for granted that doing science -- just like other aspects of living life -- relies on a constant stream of acts of faith, which can't be justified according to science....

One may then note that there are various systems for mentally organizing these acts of faith.

Religions are among them. But religions are quite detached from the process of doing science.

It seems sensible to think about philosophical systems -- i.e. systems for organizing inner acts of faith -- that are intrinsically synergetic with the scientific process. That is, systems for organizing acts of faith, that

• when you follow them, help you to do science better
• are made richer and deeper by the practice of science
  

One can broaden this a little and think about philosophical systems that are intrinsically synergetic with engineering and mathematics as well as science.

Now, one cannot prove scientifically that a "scientifically synergetic philosophy" is better than any other philosophy. Philosophies can't be validated or refuted scientifically.

So, the reason to choose a scientifically synergetic philosophy has to be some kind of inner intuition; some kind of taste for elegance, harmony and simplicity; or whatever.

One prediction I have for the next century is that scientifically synergetic philosophies will emerge into the popular consciousness and become richer and deeper and better articulated than they are now.

Because Fish and Eagleton are right about some things: people do need more than science ... they do need collective processes focused on the important philosophical questions that go beyond the scope of science.

But my prediction is that we are going to trend more toward philosophical systems that are synergetic with science, rather than ones that co-exist awkwardly with science.

What will these future philosophical systems be like?

There's nothing extremely new about the concept of science-synergetic philosophy, of course.

Plenty of non-religious scientists and science-friendly non-scientists have created personal philosophies that don't involve deities or other theological notions, yet do involve meaningful approaches to personally exploring the "big questions" that religions address.

Among the many philosophers to take on the task of creating comprehensive science-synergetic philosophical systems, perhaps my favorite is Charles Peirce (who also developed a nice philosophy of science, though one that IMO is significantly incomplete ... but I've discussed that elsewhere.)

Building on work by Peirce and loads of others, I tried to lay out a science-synergetic philosophical system in my book The Hidden Pattern -- but like Peirce's writings, that is a fairly academic work, not an informal tract designed to inspire the common human in their everyday life.

My friend Philippe van Nedervelde likes to talk about this sort of thing as a "TransReligion/ UNReligion", but I confess to not finding that terminology very compelling.

Philippe is interested in (among many other things!) developing vaguely religion-like rituals that coincide with some sort of science-synergetic philosophy. There has been talk about formulating a "TransReligion/ UNReligion" as an outgrowth of the futurist group now called "The Order of Cosmic Engineers." Which I think is an interesting idea ... yet I'm not really sure it's the direction things will (or should) go.

I'm not sure there will emerge any one "Bible of science-synergetic transhumanist philosophy" ... nor any science-synergetic-philosophy analogues of speaking in tongues, kneeling at the altar, or consuming the simulated blood and flesh of the Savior the Son of God who gave his life for our sins. Perhaps, science-synergetic philosophy may wind up being something that pervades human culture in more of a broad-based, implicit way.

Time will tell!

Does Long-Lived Quantum Coherence Underlie Biological Processes?

Is the human brain, at the levels directly relevant for analysis of cognition, best modeled as a classical or quantum system?

(For instance, a baseball in some sense needs to be modeled as a quantum system -- in the sense that the way its molecules hold together can be described only using quantum not classical physics; but classical physics can be used to explain the normally relevant aspects of its macroscopic behavior. So at the levels directly relevant for analysis of a baseball game, a baseball is best modeled as a classical system. OTOH, at the levels directly relevant for analysis of the electromagnetic behavior of a Superconducting Quantum Interference Device (SQUID) -- a small but macroscopic device, used in magnetoencephalography machines, and demonstrating macroscopic quantum coherence in its magnetic field -- the SQUID is best modeled as a quantum system. Classical physics models just won't explain why the SQUID, a device you can hold and pinch between your fingers (though it only works when supercooled, which would freeze your fingers!), makes MEG machines work.)

Current brain theory indicates that for understanding its role in giving rise to the mind, the brain is most effectively modeled as a classical system (i.e. the brain is more like a baseball than a SQUID) ... but of course current brain theory could be incomplete.

(Even if the brain is a macroscopic quantum system, this of course doesn't prove that quantum dynamics are necessary for intelligence or consciousness or anything like that. Those are bigger and deeper questions, and I've argued in the past that sufficiently complex "classical" systems might need to be treated using quantum logic ... but this gets into a lot of deep issues that I don't want to digress onto here.)

Stuart Hameroff is one of the more vocal proponents of the "quantum brain" idea, and he has a new paper reporting a new theory in this direction, arguing that dendro-dendritic synapses are mediated via macroscopic quantum dynamics, thus posing a quantum neural net that operates in complex coordination with the classical neural net formed by axonal-dendritic synapses.

I don't have a strong opinion on that particular theory of Hameroff's. I look forward to discussing it with him at the Toward a Science of Consciousness conference in Hong Kong next month.

But I was struck by one of the references at the end of his paper, a Nature paper entitled

Evidence for wavelike energy transfer through quantum coherence in photosynthetic systems

This is a 2007 paper that I had not noticed before, and it's interesting because it gives solid evidence of macroscopic quantum coherence in a biological process.

To quote part of the abstract:


Here we extend previous two-dimensional electronic spectroscopy investigations of the FMO bacteriochlorophyll complex, and obtain direct evidence for remarkably long-lived electronic quantum coherence playing an important part in energy transfer processes within this system. The quantum coherence manifests itself in characteristic, directly observable quantum beating signals among the excitons within the Chlorobium tepidum FMO complex at 77 K. This wavelike characteristic of the energy transfer within the photosynthetic complex can explain its extreme efficiency, in that it allows the complexes to sample vast areas of phase space to find the most efficient path.

In the comments to an earlier edit of this blog post, someone pointed out this more recent paper

Quantum Zeno Effect Underpinning the Radical-Ion-Pair Mechanism of Avian Magnetoreception

whose abstract says

The intricate biochemical processes underlying avian magnetoreception, the sensory ability of migratory birds to navigate using earths magnetic field, have been narrowed down to spin-dependent recombination of radical-ion pairs to be found in avian species retinal proteins. The avian magnetic field detection is governed by the interplay between magnetic interactions of the radicals unpaired electrons and the radicals recombination dynamics. Critical to this mechanism is the long lifetime of the radical-pair spin coherence, so that the weak geomagnetic field will have a chance to signal its presence. It is here shown that a fundamental quantum phenomenon, the quantum Zeno effect, is at the basis of the radical-ion-pair magnetoreception mechanism. The quantum Zeno effect naturally leads to long spin coherence lifetimes, without any constraints on the systems physical parameters, ensuring the robustness of this sensory mechanism. Basic experimental observations regarding avian magnetic sensitivity are seamlessly derived. These include the magnetic sensitivity functional window and the heading error of oriented bird ensembles, which so far evaded theoretical justification. The findings presented here could be highly relevant to similar mechanisms at work in photosynthetic reactions. They also trigger fundamental questions about the evolutionary mechanisms that enabled avian species to make optimal use of quantum measurement laws.

This of course is even more intriguing than the green sulphur bacteria stuff, because it has to do with perception in an intelligent macroscopic animal.

Hameroff's point in citing the paper on green sulphur bacteria (and it's a good one) seems to be: if long-lived quantum coherence can play an important role in photosynthesis, couldn't it also play a role in the brain somehow ... e.g. maybe via dendro-dendritic synaptic gap junctions?

The extrapolation from these other results to neuroscience is speculative, sure.... But this kind of result does make the possibility of quantum coherence impacting human cognition seem a bit less fanciful.

After all, I often recall that in the late 90's all the neuroscientists I talked to told me there was no neurogenesis nor synaptogenesis in adult mammals. Oops. Now they've got new data and changed their mind. My point isn't that quantum coherence is related to neuro or synapto genesis (though, who knows...), but rather that neuroscientists -- simultaneously with displaying the usual humility of biologists regarding the complexity of the systems they're studying -- have a long-standing habit of assuming the concept-set underlying their current understanding is much more adequate than it really is.

Our ignorance of the brain is why my own AI work is not based on trying to closely model the brain. Of course, it's possible that intelligence is fundamentally based on some freaky neuroquantum phenomenon, so that all digital-computer AI work is doomed by some intrinsic limitations ... but I doubt it. My own guess is that, even if the brain does involve macroscopic quantum coherence in some interesting sense, one can still make transhumanly intelligent systems using digital computers. And of course, if this doesn't work -- or if these transhumanly intelligent systems turn out to lack some crucial aspect of self-awareness as the quantum-consciousness advocates argue -- then we can always add some funky quantum computing chips into our AGI server farm!

"Transcendent Man" ... my brush with movie-stardom ... and my answer to "What is Your Goal?"

My review of the Kurzweil-biopic/futurist-think-piece documentary Transcendent Man -- which features me mouthing off for 4-5 minutes in a zebra-striped cowboy hat -- in HPlus Magazine is here:

http://www.hplusmagazine.com/articles/ai/transcendent-man-film-about-kurzweil

In that article, as well as reviewing the film, I also recount some moderately interesting dialogue btw me and Ray Kurzweil that occurred in the moderated discussion at the end of the film's premiere at the Tribeca Film Festival...

After that conversation with Ray I discuss at the end of the article, the discussion-moderator asked me another question (which I didn't put in the review article): he asked me what my goal was. What was I trying to achieve?

What I said was something like this: "I would like me, and any other human or nonhuman animal who wants to, to be able to increase our intelligence and wisdom gradually ... maybe, say, 37.2% per year ... so that we can transcend to higher planes of being gradually and continuously, and feel ourselves becoming gods ... appreciate the process as it unfolds."

That's what I'm after, folks. Hope you'll come along for the ride!

Teaching Dolphins Lojban ... Giving Dolphins Prosthetic Hands

A follow-up to my prior posts on cetacean intelligence...

I thought a bit about innovative ways we might be able to communicate better with our cetacean planet-mates...

1. Teach Dolphins Lojban

A couple decades ago, efforts were made to teach dolphins simple English, without dramatic success. Discussions were also had regarding creation of some sort of species-independent interlingua, which humans and dolphins could use to communicate with each other.

It occurred to me that using Lojban for that interlingua could make sense. Potentially, one could create special Lojbanic vocabulary for the shared human/dolphin environment. Lojban grammar is simple and unambiguous, and certainly has less species-specificity than any human natural language.

Also, one could create a form of Lojban "phonology" that generally follows the sound-production patterns habitually by dolphins, and speak to dolphins in this "Delphic Lojban" alongside the usual "human Lojban."

The biggest disadvantage of this approach is that it requires some human cetaceologists to learn Lojban.... But this cost seems worth paying, as the odds of success seem much higher than with human natural languages.

Note that there is no straightforward way to make a "phonologically Delphic" version of English. But because Lojban syntax is just a linearization of logical relationships, one could make a Delphic version of Lojban by translating those same logical relationships into sound in a wholly different way than is done in the human version of Lojban.


2. Give Dolphins Prosthetic Hands

Inside a dolphin's flippers, are bones that look like they should correspond to claws or fingers.

What if we created prosthetic fingers and thumbs for dolphins, and connected them to these bones ... and also connected them to the dolphin nervous system?

Admittedly, these modified dolphins would suffer impaired swimming ability, though one would hope the degree of this phenomenon could be palliated via appropriate design. (For instance, perhaps the fingers could be made retractable, so the dolphin could retract them when it wanted to swim, and extend them when it wanted to manipulate objects.)

This would be a highly experimental adventure in Brain-Computer Interfacing. But, as BCI research advances in the context of human-enhancement applications, I see no reason why it shouldn't advance in the context of dolphin-enhancement applications in parallel.

My thinking is that much of which distinguishes human intelligence from cetacean intelligence is our focus on complex manipulation of tools, and building things (including advanced phenomena like tools that make tools, etc.). If a dolphin brain self-reorganized to adapt to its prosthetic fingers, then the dolphin would have the capability to use tools in a more humanlike way.

Since the cetaceans' evolutionary progenitors had claws of some sort, there may be some vestigial neural wiring in the dolphin brain that will ease the self-reorganization that the dolphin brain needs to go through to make use of the prosthetic fingers.

Another possibility would be to build in the capability for human operators to periodically "take over" the dolphin fingers using remote control. This would serve to show the dolphin what to do with the fingers, both on the conscious reflective level, and on the level of unconscious habituation.

Of course discussions of what to build with the fingers, and how to use tools, could be carried out using Lojban (human or Delphic) ;-D

...

Ahhh ... all the really fascinating research that would get funded if I happened to receive a billion-dollar inheritance from some long-lost uncle ;-p

Why AGI Researchers Should Care about Cetaceans

This is just a brief follow-up to my last post, and a prelude to the one that will follow, which is already brewing in my brain....

In case some of y'all are wondering why I ... whose main intellectual obsession is the creation of AGI systems with general intelligence at the human level and beyond ... have suddenly started ranting about cetacean intelligence, I suppose I should be more explicit about my research-related motivation for digging into the topic

Of course there's a personal motivation -- I love nonhuman animals ... at the moment as well as some humans I share my house with a parrot, 2 dogs and 5 bunnies; and there have been friends of a lot of other species in my life at various times.... In fact the parrot named Abaca might be classified as my best friend over the last few months ;-) ....

I've encountered wild dolphins up close in the shallows of the Indian Ocean when I lived in Western Australia 13-15 years ago, and was certainly struck by the experience, as brief and superficial as it was. I definitely wanted more (and would have sought out more if I hadn't left Western Oz to move to New York and start an AI company)

But what I want to focus on here is my intellectual motivation for, as an AGI researcher, finding cetacean intelligence important.

How often do I hear, among AGI researchers, words to the effect of "Of course we need to model our AGI systems on the human brain and mind, since after all it's the only example we have of a highly generally intelligent system."

I tend to resist this line of thinking ... I think we understand enough about the general mathematics and computer science of cognition that we can understand general intelligence in a manner going beyond the human-specific. My own AI work is an amalgam of aspects directly inspired by human intelligence, and aspects inspired by a broader understanding of intelligence.

But still, there is some point to the common observation that we only know one example of a highly generally intelligent system: the human brain.

But is it actually true?

What if the subset of cetacean intelligence researchers who believe cetaceans have general intelligence comparable to, or greater than, human intelligence ... are actually correct? (Which is my suspicion.)

Then in fact there's another example available -- and we're just not taking the trouble to study it as carefully and thoroughly as we should.

In my immediately previous blog post I gave some links into the cetacean intelligence literature, and some speculations as to what I think the broad nature of cetacean intelligence might possibly be.

In my next blog post I'll discuss some cutting-edge approaches that we might take over the next couple decades to more thoroughly understand cetacean intelligence.

I'm not suggesting that resources be taken from AGI and redirected to cetacean cognitive science: I think that both areas are distressingly underfunded.

In the case that we create AGI programs with superhuman general intelligence before we understand cetacean minds, I think we might still have something to learn from the minds of dolphins. Because cetacean minds may possess a quite different form of intelligence than either us or our AGI creations. And it's hard to tell what may be learned by studying some advanced, fundamentally different incarnation of intelligence.

There could possibly be interesting implications for the study of AGI ethics here, for instance. Cetacea are certainly not optimally ethical creatures ... they're capable of violence just like most other mammals ... but based on what we can understand today, it seems their social organization may have fewer egregious ethical issues than ours. As one example, they seem to have achieved a large-scale, global social organization without warfare. (Evidence of the global nature of their social organization is tentative, but provided by observations such as the way repeated phrases in whale "song" tend to arise in one part of the globe, then spread through whales in all the world's oceans, then die out after a time, replaced by others.)

I'm certainly not suggesting that study of cetacean society will magically provide the answer to the AGI ethics problem, or the problem of generally understanding general intelligence, etc. However, I think it would be very interesting to understand how a fundamentally different sort of general intelligence works, and how it has approached the society/ethics problem, as an additional body of evidence to utilize as we shape the minds of the future.

Cetacean consciousness...

I've been reading many of the writings of John Lilly lately, and also poring through the literature on cetacean intelligence ... and I have to say it's fascinating stuff ....

I'm fascinated by Lilly's cetacean intelligence/communication work, his isolation tank work, even his obsessive (and, apparently, excessive) experiments with ketamine injection leading to long conversations with various hallucinated (?) extraterrestrials ;-)

(I read his stuff a couple decades ago but I've been through a lot of experiences since, and I can read it with different eyes now. I remember how inspirational his book "Programming and Metaprogramming the Human Biocomputer" was for me, when I read it at age 13 or 15 or whatever.)

Anyway ... plenty of scientists by now have followed up Lilly's intuitions about the deep intelligence of dolphins and other cetaceans. A bunch of research papers by various scientists (not under the influence of ketamine ;-) are here:

http://www.dolphin-institute.org/our_research/dolphin_research/dolphinresearchpublications.htm

For some relevant books by people less fringe-y than Lilly, but still quite insightful, see e.g.

• Rachel Smolker's elegant scientific memoir
• This edited volume on Dolphin Societies

but I've found no up-to-date comprehensive review book, so you really gotta read the journal literature and various books to understand what's known so far...

As of now there is no definitive scientific proof that cetaceans are extraordinarily intelligent ... though there's pretty solid proof that they're at least as clever as great apes, I would say (though different in mentality) ...

However, my qualitative impression from reviewing all the evidence is that they are, in some senses, dramatically more intelligent than great apes

I will write something systematic on this topic at some point, when I get more time and have read the literature more thoroughly (obviously this is just a background interest for me, so my reading is going pretty slowly...)

What got me musing about this topic right now was thinking about how the naive physics of our everyday world has impacted human intelligence, and what this might mean for engineering and educating AGI.

Last month Allan Combs and I wrote a paper for the NASA CONTACT workshop, discussing how the radically different environments of extraterrestrials might impact their mind-states and varieties of intelligence:

http://goertzel.org/papers/AlienMinds.pdf

(we'll academic-ize this and publish it somewhere, in time).

And this is also related to a paper I wrote a couple months back, musing about how the lack of fluids, powders, fabrics and other such substances in virtual worlds may impact their utility as homes for humanlike artificial minds:

http://goertzel.org/dynapsyc/2009/BlocksNBeadsWorld.pdf

(In that paper I also explored how it might be possible to enhance virtual worlds to largely remedy this shortcoming, using a special physics-engine technique I called "bead physics".)

In writing that NASA paper, I started wondering how it would impact a mind to evolve in an environment dominated by fluids rather than solids.

My speculation was that, in such a mind, notions of causing and building would be replaced by notions of flowing and shaping .... which would lead to all sorts of other differences.

Gino Yu then pointed out to me these fascinating speculations on the potential subjective experiences of cetaceans:

http://www.cbc.ca/ideas/features/ocean-mind/index.html

All this has spurred me to some of my own entertaining speculations (synthesizing various speculations of Lilly and others) ... to wit:

... what if (as Lilly speculated) the everyday states of mind of cetaceans are more like the states of mind that humans get into while on psychedelic drugs, than they are like our everyday consciousness?

After all, these creatures are breathing deeply and rhythmically ... they're floating in liquid ... generally they're living the sort of physical life that would put humans in a deep semi-meditative state ...

What if their big neocortices are devoted essentially to collaboratively composing and improvising music for each other to listen to?

... but perhaps something more advanced and subtle than human music, reflecting intricate patterns of social interaction, and holistic observations about the state of the underwater ecosystem, and emergences between these social and ecosystem patterns...

This would be a type of intelligence not focused on building tools or solving puzzles in the humanlike sense....

As with human intelligence, the main spur for the evolution of such intelligence would be social. Once the composition/improvisation of this kind of communicative/depictive music became a critical aspect of membership in cetacean society, then there would be evolutionary force to compose/improvise more and more appealing music....

In this hypothesis, the crux of dolphin communication might not be one-to-one conversation, but rather multi-player musical improvisation, with both spatial and temporal aspects. Dyadic conversation with practical import might occur, yet have vastly less complexity and subtlety than other aspects of the musical communication...

One interesting thing about this speculation is that, if it were true, it would mean that probing cetacean intelligence using concepts and methods developed for studying human intelligence, could push the researcher in badly wrong directions.

By analogy, imagine that a species whose main focus of intelligence was collaborative spatiotemporal music improvisation, tried to judge and explore human intelligence. Most humans would be judged as hopelessly moronic ... and then a few gifted musicians might be viewed as moderately intelligent. Due to the other species focusing on collaborative spatiotemporal music improvisation, they would miss what is really the crux of human intelligence: our dyadic linguistic communication, and our tool-building.

John Lilly wanted to probe cetacean communication with computer tech, back in the late 1970s and early 1980s. Computers are a lot better now, so someone could take a much better shot at it. But rather little research seems to be going on at the intersection of advanced AI pattern analysis and cetacean communication, at the moment. Too bad.

More ambitiously, one can envision creating an AI that shared both a humanlike body, and a dolphin-like body, and letting it exist in both worlds.

Lilly did make a good point, that we should probably take some of the $ we are spending on looking for alien lifeforms in space, and devote some of it instead to trying to communicate with these alien intelligences that apparently exist in our oceans. If we can't even communicate with the other intelligences on our own planet, cracking the codes of the minds and languages of beings on alien planets may not be realistic yet (though, of course, there is massive uncertainty in all these domains...).

There is some inordinately silly stuff written about cetacean intelligence -- I read one book on the theme that "Jesus was a dolphin"!! And Lilly certainly complicated his message about cetacean intelligence by mixing it up with some of his other messages, for instance about extraterrestrials whom he felt he contacted while in isolation tanks and on ketamine. But all that is really beside the point. When you look at the scope of existing qualitative evidence about cetacean intelligence, the picture is striking....

Whether the speculations I've made above are on-point or not, I'm convinced there is something very interesting going on in cetacean minds and societies -- which we are not putting nearly enough effort into understanding.

Instead, we are still killing them and making them into steaks.

Female mad scientists and creative nihilism (thoughts on Sofia Kovalevskaya)

I've been thinking of making the protagonist of my next novel a female "mad scientist", but I wasn't sure how to write the character, so I started searching history for a good model.

I found damn few female mad scientists in recorded history.

The closest I found was the Russian mathematician Sofia Kovaleveskaya, whose name I knew from the Cauchy-Kovalevsky Theorem in partial differential equations. I hadn't known much about her before, so I did some reading and found she fit the bill pretty well:

• World-class mathematician
• Spent some time inventing weird new electrical machinery
• Accomplished novelist
• Also wrote plays and poetry
• Participated in the Paris Commune, and generally schemed for revolutionary overthrow of governments
• Helped her husband lose piles of money during a several year period devoted to "clever" real estate and financial speculation

A woman after my own heart -- wish I'd known her! She was also the first woman ever to get a math PhD (she lived in the mid-1800's).

Her biography is also full of nice tidbits, like

• She first got passionate about advanced math when her attic happened to get wallpapered with lecture notes from a calculus class her father had taken years before. So she learned about limits and such from reading unordered pages of mathematical text pasted to a wall!
  
• To study advanced math, she had to leave Russia (due to sexist regulations), and to do that she had to get married ... so she entered into a "fake marriage" with a platonic male friend with the sole purpose of escaping Russia to get to university in Western Europe (although, many years later, the fake marriage turned real...)
  

Her childhood memoir "A Russian Childhood" is a wonderful book and I'd recommend it to anyone who likes Russian literature.

This biography is also worth reading for the story it tells -- although the biographer's radical-feminist antimasculism is annoying, and appears to radically falsify Kovalevskaya's relationship with her husband, among other things

There's also a historical novel about her life, which I haven't read

Her novella "Nihilist Girl" has some greatness about it too, but feels first-draft-ish, like it needed a final edit to really become a work of art.

But the main thing I wanted to write about today was the revised idea of "nihilism" I got from reading "Nihilist Girl" and these other materials....

I've generally thought of "nihilism" as meaning "believing in nothing" ... or at least the attitude of Turgenev's character Bazarov, that nothing really matters much including one's own life....

But after reading Kovalevskaya, I realize that -- in thinking about Russian nihilism from the mid-1800s -- I was largely mistaking the parody for the real thing.

Kovalevskaya's brand of nihilism was significantly more interesting than that of the fictional character Bazarov.

It wasn't about absolutely rejecting everything and judging everything as meaningless and worthless. Rather, it was about rejecting any absolute values. It was about rejecting anything as sacred -- and opening everything up to question.

What was being rejected was a world-view in which there are certain absolute truths, in terms of which everything else must be assessed.

If you get rid of absolute truth, though, then what are you left with? Complete worthlessness and suicide, a la Bazarov? Or maybe not. What Kovalevskaya and her friends were after was something different.

One can think about it in terms of self-organization and strange attractors. Once one gets rid of absolute truth, and admits every single thing as open to question and revision, then one has a self-organizing dynamical system in which each thing gets potentially revised by each other thing. But the outcome of this doesn't need to be a homogeneous evaluation of everything as worthless. The outcome can be some other "strange attractor" in which each thing gets value from each other thing, according to a complex system of interdependencies and interactions.

Kovalevskaya-style nihilism, it seems, wasn't really about rejecting everything as equally worthless, but more about rejecting anything as absolutely valuable ... and letting the process of interactive, adaptive, mutual-value-adjustment spread through everything and lead to a productive evolution of new valuations and forms.

But I don't seem to have been the only one to get confused about nihilism.

Wikipedia says:

Nihilism (from the Latin nihil, nothing) is the philosophical position that values do not exist but rather are falsely invented. Most commonly, nihilism is presented in the form of existential nihilism which argues that life is without meaning, purpose or intrinsic value. Moral nihilists assert that morality does not exist, and subsequently there are no moral values with which to uphold a rule or to logically prefer one action over another.... The term nihilism is sometimes used synonymously with anomie to denote the general mood of despair at the pointlessness of existence that one has when they realize there are no necessary norms, rules, or laws.

and Nietzsche wrote in his notebooks (The Will to Power, section 585, translated by Walter Kaufmann)

A nihilist is a man who judges of the world as it is that it ought NOT to be, and of the world as it ought to be that it does not exist. According to this view, our existence (action, suffering, willing, feeling) has no meaning: the pathos of 'in vain' is the nihilists' pathos — at the same time, as pathos, an inconsistency on the part of the nihilists.

Nietzsche posited his own views as dramatically contradictory to nihilism -- and they certainly are wholly contradictory to Bazarov-style nihilism ... Nietzsche was all about creating your own values, rather than accepting any values as absolute, and rather than rejecting all values.

But it seems that Nietzsche was posing nihilism as a "straw man" to an even greater extent than I'd thought before ... and that his general views on trans-nihilist value-creation were not so fundamentally different than those of many of the Russian nihilists of the 1860s.

In Cities of the Red Night, Burroughs wrote "Nothing is true; everything is permitted" -- which on one reading reflects Bazarov-style nihilism ... and which Burroughs borrowed from Nietzsche, whose Zarathustra said:


“Nothing is true, all is permitted”: so said I to myself. Into the coldest water did I plunge with head and heart. Ah, how oft did I stand there naked on that account, like a red crab! –

But if one reads this Burroughs/Nietzsche aphorism as "Nothing is absolutely true; nothing is absolutely impermissible" then one has a Kovalevskaya-style nihilism, in which dogmatism is eliminated in favor of the creative self-organization of new value systems.

Dostoevsky also seems to have put a lot of energy into pillorying a straw-man version of Russian nihilism. Dostoevsky's nihilists are folks like Raskolnikov (the heartless, utilitarian murderer of Crime and Punishment) or Kirilov in The Possessed (a majorly hilarious character who preaches copiously about the worthlessness of existence and then suicides because he considers it the highest act of free will).

Kovalevskaya's "nihilist girl" character is about as far as you can get from Raskolnikov -- an extremely caring person, she marries a political prisoner (a man twice her age in whom she has no romantic interest) to save him from near-certain execution, even though this means her own exile to Siberia ... and generally decides to devote her life to helping Siberian prisoners, as a way of contributing to the common good. She doesn't lack values -- she just rejects having absolute values imposed on her, and wishes to create her own values based on her own intuitions and her engagement with the world.

And the narrator of Nihilist Girl makes a different choice -- she is a mathematician like Kovalevskaya, devoted to the life of science; she plainly states that she would not exile herself to save a political prisoner -- yet she just as plainly shares the same underlying philosophy of "creative nihilism" (my phrase, not Kovalevskaya's).

Dostoevsky courted Sofia Kovalevskaya's big sister Aniuta, as it happened. His story was that he broke off their engagement because she was too nihilistic. Her story (which has more ring of truth) was that she broke up with him, before they were formally engaged, because she didn't want to spend her life taking care of him and wanted more freedom to explore her own interests and passions.

But however the soap opera really went down, both Sofia and Aniuta were too nihilistic for Dostoevsky. The latter believed that only religious belief could save you from destructive nihilism of the form demonstrated by Bazarov, Raskolnikov or Kirilov. He didn't think new value systems could self-organize out of a pool of interacting non-absolutes ... to him value needed to begin with some absolute faith, some absolute assumptions.

All in all, I conclude that nihilism suffered from poor marketing, and an overly subtle and ironic name, which caused its more interesting variants to get forgotten, and its less interesting variants to get repeatedly parodied.

Long live creative nihilism, Kovalevskaya style!

Nothing is true; everything is permitted ;-)

When the Net Becomes Conscious

A journalist emailed me today and asked me some questions about the possibility of the Internet becoming conscious. The questions and my answers follow:


> 1) Why do some people think it is possible for the internet (or internet plus humans) to become conscious? Is it to do with the network architecture?

Many scientists believe that consciousness is a property that will inevitably emerge from any complex system that has the right sort of internal dynamics, and the right sort of interaction with its environment.

Exactly what the right sort of dynamics and interactions are, different theorists disagree on. But it seems plausible that the Internet may have enough of them to develop its own sort of consciousness. The Internet perceives and acts on the world; it stores declarative, episodic and procedural memories; it recalls some information and forgets others; etc. In short it behaves a fair bit like a human mind, though there are a lot of differences too.

According to this perspective, the Internet might already have a degree of consciousness, though of a type quite different from human consciousness.

Neuroscientist Susan Greenfield views consciousness as consisting of "whole-brain activation patterns". In this sense one would say that the Internet of today has a more fragmented, dissociated consciousness than a human mind ... there aren't so many "whole-internet activation patterns", though there are intense patterns spanning large portions of the Internet.

Of course, there are many philosophies of consciousness. My own view of consciousness is a bit eccentric for the scientific world though rather commonplace among Buddhists (which I'm not): I think consciousness is everywhere, but that it manifests itself differently, and to different degrees, in different entities.

So to me the interesting question is whether the Internet has (or will develop) consciousness of the same type as humans, or maybe even of a more advanced and intricate type.

It seems that as the Internet expands and grows richer, it *could* develop a more human-like, more unified consciousness than it has now ... with more coherent "whole Internet activation patterns"...

> 2) What might be the consequences of such an event? Do you think it might be something that we should welcome?

The potential consequences of the Internet developing more coherent holistic activation patterns (ergo more humanlike consciousness) are rather difficult to predict, I find!

However, I personally am pessimistic about the future in the case that humans remain the most powerful minds on the planet. I don't trust us to use our increasingly advanced technologies in an ethical and nondestructive way.

So I think the outlook for humanity is probably better in the case that an emergent, coherent and purposeful Internet mind develops, than in the case where it doesn't.

But there is a lot of uncertainty in either case!


> 3) If it were possible, what would be needed to make the internet conscious? How far away from that situation are we?

My guess is that humanlike consciousness is not going to spontaneously evolve from the Net. However, I think someone could engineer it, by specifically creating an AI system on a server farm, oriented toward serving as a kind of "central cognition engine" for the Internet as a whole.

This central cognition engine wouldn't need to control everything on the Net; it would just need to read a lot of the information out there on the Net, and then insert information of its own creation in appropriate locations (posting to email lists, creating web pages, buying and selling things, etc.).

The engine might be created with some other primary purpose (e.g. as an artificial scientist aimed at making new discoveries via collaborating with human scientists online), or it might be created specifically with the goal of transforming the Internet into a more coherent, more humanlike intelligence. Either way the effect might be the same.

This is the scenario I described in my 2001 book "Creating Internet Intelligence," and I still think it is a plausible one.

Once Science Eliminates Pain ...

The bored or overly curious may check out my latest neurological dysfunction (aka work of fiction) "The Last Aphrodisiac", at

http://goertzel.org/fiction.htm#pain


What happened was, I was driving late at night listening to a Morphine CD in the car, then got home, lay in bed and fell asleep with the song "Cure for Pain" in my head.

I had a number of dreams on the theme (what if pain were really eliminated, in some interesting sense? what would life be like? what if it were rediscovered?) and woke up plagued by this story. On a cross-country flight to a weekend workshop on "Evaluation and Metrics for Human-level AI", I decided to write it down...

At first I thought it would take a single page to write down, but it wound up 15 pages, and the punchline doesn't start to unfold till page 7 or 8.

This is the first story I've written in a long time that doesn't involve AI in any serious way. Rather, it uses future tech like uploading-to-superhuman-form and cranial jacks to enlarge upon certain aspects of human relationships, especially romantic ones. It's probably the closest thing to a maudlin love story I'll ever write (well, I hope so).

Ahh, the things that can transpire between a man, a woman, and an illicit cranial jack modification device... ;-)

Freaky Russian Paranormal Biology (plus irrelevant personal-history rambling)

A friend pointed me to this paper

http://www.emergentmind.org/gariaev06.htm

which I found provoking and interesting, though nowhere near fully convincing....

The authors argue that much "junk DNA" actually serves as an interface to some sort of energy-informational field reminiscent of Sheldrake's "morphogenetic field"...

Their comments on traditional biology are a bit naive: in the last 5 years a lot of functions of formerly-known-as-junk-DNA have been found. So it's no longer true that ordinary biology says 95%+ of DNA is useless. We're finding this DNA serves a lot of regulatory functions, even though it doesn't code for proteins.

On the other hand, that oversight doesn't make their theory wrong; it just makes them out of date regarding traditional biology.

The experimental results they describe are certainly compelling and intriguing. Fraud is always a possibility, yet, I'm wary to dismiss results as likely fraud just because they violate currently standard scientific theories. (After all, every scientific paradigm prior to the current ones has been overthrown, right?)

As an aside in their discussion, they mention some old results involving Kirlian photography. I tried to build a Kirlian camera in my basement once, in Randolph NJ in the late 1990s, with a view toward trying to replicate the "phantom leaf effect" (see link below) ... but wound up setting fire to part of the basement instead. (I did build a Tesla coil from a neon sign transformer, and it worked a few times, but eventually it caught fire before I finished making it into a Kirlian camera apparatus.)

A recent attempt to replicate the phantom leaf effect, with intriguing results, is described here:

http://shadowboxent.brinkster.net/lemurkirlian2.html

It's hard to be anywhere near certain, but I'm intuitively inclined to feel there might be some truth somewhere in the vicinity of these guys' wacky theory.

Obviously it's close to my own "glocal theory of psi" ... in the language of my "glocal memory theory" what they're saying is that living organisms have "keys" (the observed physical substance) and "maps" (the correlated energy-information field).

I'm hesitant to use the word "energy" in this context as these Russian authors do, because "energy" has a specific meaning in physics, and this (even if real) may be something different. I have thought of their energy-informational field as a kind of "pattern space", more like Sheldrake's idea of a morphogenetic field....

But, without some hypothesis regarding the dynamical laws of this posited morphogenetic field, their theory remains more philosophical than even "speculatively scientific."

It seems that conceivably -- if there's any reality here -- one could learn something about the laws of this field via systematically varying parameters in Kirlian photography experiments.

Yet another line of research I'd probably fund if I were super-wealthy ;-p

P.S.

(warning: loosely related personal-history rambling below...)

Wow, I remember now that when I was 18 years old and in my senior year of college (at Simon's Rock, in western Massachusetts) and visited CalTech, where I was hoping/considering to go to grad school, I mentioned to one of the math profs there that I was interested in doing a PhD thesis on using partial differential equations to model bioelectromagnetic fields as had been discovered in some strange Russian experiments.

His reply was something like "Well, you certainly know a lot of big words, but do you know any math?" We talked a bit and he discovered that I did; but I didn't get admitted to CalTech anyway.... It was obvious that the math department there did not like that potential thesis topic!!

On that same visit to California I visited Berkeley's Logic and Methodology PhD program, where I expressed my interest in writing a thesis on using hypersets (non-well-founded sets) to model consciousness (which now is one of the themes of one of the handful of half-finished books on my hard drive). This was better received, and they were likely to admit me with funding (or so they said verbally) but I wound up not completing my application because the students there told me that the department inevitably made its students take 7-9 years to finish their PhDs. I didn't want to be in school that long, so I wound up at NYU's Courant Institute instead, which I liked because it combined math, theoretical physics and computer science in one department...

On that trip I also visited a girl at UCSD whom I had a crush on (from when she'd attended college with me in Massachusetts), and was disturbed to find she'd become a fundamentalist Christian, handing out Jesus brochures on the street. (I saw her 11 years ago and she seemed to have gotten over that phase long ago, fortunately... though it was a bad visit as I'd been up nearly the entire night, insomniac due to too much arguing with my wife-at-the-time Gwen, and was completely bleary-eyed and -minded for the whole visit ... that was a few days before the birth of my daughter Scheherazade and Gwen was mad at me for repeatedly getting stoned out of my mind with one of my friends at a time when the baby might pop out at any minute ... well, at least being totally stoned helped the fighting go down easier!!)

In hindsight, that was a rather entrepreneurial trip for a college senior to make (esp. an 18 year old one): I basically invited myself on interviews to grad schools I was interested in, prior to even completing my formal applications to the places ... to go talk to the profs and students there, sit in on classes, and get a feel for the departments. I didn't realize at the time that this was a fairly eccentric thing to do. But it was a good idea.... Although I had only one set of clothes for the whole trip, because People Express Airlines (which featured a great $99 cross country flight) sent my luggage to Europe by mistake.

(Unfortunately that trip caused me to get fired from my job as a math homework grader, because I just took off from college for a week and flew to California without giving any notice to my boss, so all the papers went ungraded while I was gone. I wasn't too conscientious back then. I'm still a slob with paying bills for the house and such, but I try not to be a mess like that in professional life anymore! Still it's hard to focus on reality and not be an absent-minded professor sometimes ;-)

All that probably has something to do with Kirlian photography, auras and Russian theories of morphogenetic fields ... but since I don't get stoned hardly ever these days, I'm not in the right state of mind to find the connecting thread; and I'll go cook dinner instead and then get down to some useful work (oh yeah, and Scheherazade and I are going to watch the film Baghead tonight... which unfortunately has nothing to do with Buckethead...)

Hmmm... that reminds me of the Timothy Leary Family Reunion I attended in San Francisco last week ... what a wonderful assortment of old hippies with cosmic looks in their eyes and wild ideas in their brains; along with various sympathetically resonating youngsters like me ... but, I won't go there right now...

Nietzschean Nonlinear Economics?

I was thinking a little more about the current economic situation and why it seems so confusing to people, including experts.

Part of the problem is just that the systems involved are so complex, of course -- the economy has complexified and the human brain has not kept up.

Another part of the problem, though, is that none of the traditional economic theories (neither the mathematical ones or the qualitative ones) embrace this complexity in principle.

I.e., the very fact of the increased complexity of the economy has direct implications, which seem to require introduction of new approaches for economic analysis.

One way to look at this is in terms of what I semi-seriously call "Nietzschean economics," where a more abstract notion of economic power replaces the traditional notion of money.

Put simply: it seems that the modern economy has somewhat obsoleted the concept of money, which has economists (and many others) confused...

For instance, China has a lot of US dollars right now, in principle ... but they can't really spend most of it, because if they traded their dollars for goods on the open international market, then the dollar would collapse in the international currency markets, the US economy would tank, demand for Chinese goods would collapse, and China would risk massive internal unrest etc.

It's not exactly that the Chinese government has money they can't use; but they have money that comes with severe restrictions on the ways it can be used. Having this money confers great economic power on them; but, this economic power can't necessarily be used in the traditional manner of buying stuff.

In general: with the world so interconnected, the notion of a unit of currency as being something that can be exchanged for a certain amount of stuff, anywhere, doesn't really apply...

More complexly, the same sorta phenomenon applies among and within investment banks and other financial institutions. E.g. a large bank holds some portfolio of financial instruments ... but if they sell a lot of one kind of instrument, this impacts the markets in a way that affects the value of the others, etc. So one can hardly assign each instrument they hold a value independently of the others.... The values of their various portfolio items are interdependent in the manner of a system of simultaneous nonlinear equations.

What this growing, rampant interdependence means is that the traditional concepts and tools of economics don't closely apply to the international biz/finance world anymore...

A lot of this was foreseen, on a qualitative level, in Galbraith's book "Economics and the Public Purpose" from the 1970s ... he talked about the "market economy" versus the "technostructure" and pointed out that the latter (being a complex of large corporations, governments and other institutions) follows quite different rules.

Well, now the nonlinear dynamics of the technostructure is running rampant -- but economists are still mainly studying it with tools designed for studying markets.

One step I think that needs to be taken, on the theory level, is to view actual buying power as Level 1 of a hierarchy of types of economic power. I'm thinking of:

• Level 1 = power to buy goods or services [right now, or at some future point(s) in time]
• Level 2 = power to influence others' Level 1 power
• Level 3 = power to influence others' Level 2 power
• .. etc. ...

In principle, one could boil down Level k power into Level 1 power. But in practice, with the economy so complex, this involves calculations that are infeasible to do. So, economic agents are in effect seeking Level k power without a clear picture of how it will in the future boil down into Level 1 power. This might be thought of as Nietzschean economics (as Nietzsche viewed the "will to power" as the essential dynamic of the universe).

The ultimate extreme of all this of course would be

• Level infinity = power to influence others' Level infinity power

which was basically Nietzsche's view of the driving force of the universe ... and I do think that international economics and politics boils down to this sometimes: power for it's own sake, rather than being tied to ultimately influencing the acquisition of goods and services.

Traditional economics is based on the notion that everything boils down to buying power ... but in a world where no one is smart enough to calculate what their decisions will imply in terms of buying power, this sort of economics seems to have limited applicability...

What we need is a nice, elegant, pragmatically applicable theory of the nonlinear dynamics of Level k economic power under conditions where

• the computational complexity of recognizing important high-level patterns in an economy

vastly exceeds

• the computational capability of even the smartest individual participants in the economy

This might be a fun thing to work on, but I've got a thinking machine to build, so hopefully somebody else will do it ... or we'll have to wait for the AI to solve the problem. (Of course, an appropriately constructed AI could also palliate the problem, due to having increased capability to recognize economic patterns, either due to possessing greater-than-human general intelligence, or due to combining human-level general intelligence with specialized capabilities for economic analysis.)

The Subtle Structure of the Everyday Physical World = The Weakness of Abstract Definitions of Intelligence

In my 1993 book "The Structure of Intelligence" (SOI), I presented a formal definition of intelligence as "the ability to achieve complex goals in complex environments." I then argued (among other things) that pattern recognition is the key to achieving intelligence, due to the algorithm

• Recognize patterns regarding which actions will achieve which goals in which situations
• Choose a goal that is expected to be good at goal achievement in the current situation

The subtle question in this kind of definition is: How do you average over the space of goals and environments? If you average over all possible goals and environments, weighting each one by their complexity perhaps (so that success with simple goals/environments is rated higher), then you have a definition of "how generally intelligent a system is," where general intelligence is defined in an extremely mathematically inclusive way.

The line of thinking I undertook in SOI was basically a reformulation in terms of "pattern theory" of ideas regarding algorithmic information and intelligence that originated with Ray Solmonoff; and Solomonoff's ideas have more recently been developed by Shane Legg and Marcus Hutter into a highly rigorous mathematical definition of intelligence.

I find this kind of theory fascinating, and I'm pleased that Legg and Hutter have done a more thorough job than I did of making a fully formalized theory of this nature.

However, I've also come to the conclusion that this sort of approach, without dramatic additions and emendations, just can't be very useful for understanding practical human or artificial intelligence.

What is Everyday-World General Intelligence About?

Let's define the "everyday world" as the portion of the physical world that humans can directly perceive and interact with -- this is meant to exclude things like quantum tunneling and plasma dynamics in the centers of stars, etc. (though I'll also discuss how to extend my arguments to these things).

I don't think everyday-world general intelligence is mainly about being able to recognize totally general patterns in totally general datasets (for instance, patterns among totally general goals and environments). I suspect that the best approach to this sort of totally general pattern recognition problem is ultimately going to be some variant of "exhaustive search through the space of all possible patterns" ... meaning that approaching this sort of "truly general intelligence" is not really going to be a useful way to design an everyday-world AGI or a significant components of one. (Hutter's AIXItl and Schmidhuber's Godel Machine are examples of exhaustive search based AGI methods.)

Put differently, I suspect that all the AGI systems and subcomponents one can really build are SO BAD at solving this general problem, that it's better to characterize AGI systems

• NOT in terms of how well they do at this general problem

but rather

• in terms of what classes of goals/environments they are REALLY GOOD at recognizing patterns in

I think the environments existing in the everyday physical and social world that humans inhabit are drawn from a pretty specific probability distribution (compared to say, the "universal prior," a standard probability distribution that assigns higher probability to entities describable using shorter programs), and that for this reason, looking at problems of compression or pattern recognition across general goal/environment spaces without everyday-world-oriented biases, is not going to lead to everyday-world AGI.

The important parts of everyday-world AGI design are the ones that (directly or indirectly) reflect the specific distribution of problems that the everyday world presents an AGI system.

And this distribution is really hard to encapsulate in a set of mathematical test functions. Because, we don't know what this distribution is.

And this is why I feel we should be working on AGI systems that interact with the real everyday physical and social world, or the most accurate simulations of it we can build.

One could formulate this "everyday world" distribution, in principle, by taking the universal prior and conditioning it on a huge amount of real-world data. However, I suspect that simple, artificial exercises like conditioning distributions on text or photo databases don't come close to capturing the richness of statistical structure in the everyday world.

So, my contention is that

• the everyday world possesses a lot of special structure
• the human mind is structured to preferentially recognize pattern related to this special structure
• AGIs, to be successful in the everyday world, should be specially structured in this sort of way too

To encompass this everyday-world bias (or other similar biases) into the abstract mathematical theory of intelligence, we might say that intelligence relative to goal/environment class C is "the ability to achieve complex goals (in C) in complex environments (in C)"

And we could formalize this by weighting each goal or environment by a product of

• its simplicity (e.g. measured by program length)
• its membership in C, considering C as a fuzzy etc

One can create a formalization of this idea using Legg and Hutter's approach to defining intelligence also.

One can then characterize a system's intelligence in terms of which goal/environment sets C it is reasonably intelligent for.

OK, this does tell you something.

And, it comes vaguely close to Pei Wang's definition of intelligence as "adaptation to the environment."

But, the point that really strikes me lately is how much of human intelligence has to do, not with this general definition of intelligence, but with the subtle abstract particulars of the C that real human intelligences deal with (which equals the everyday world).

Examples of the Properties of the Everyday World That Help Structure Intelligence

The propensity to search for hierarchical patterns is one huge example of this. The fact that searching for hierarchical patterns works so well, in so many everyday-world contexts, is most likely because of the particular structure of the everyday world -- it's not something that would be true across all possible environments (even if one weights the space of possible environments using program-length according to some standard computational model).

Taking it a step further, in my 1993 book The Evolving Mind I identified a structure called the "dual network", which consists of superposed hierarchical and heterarchical networks: basically a hierarchy in which the distance between two nodes in the hierarchy is correlated with the distance between the nodes in some metric space.

Another high level property of the everyday world may be that dual network structures are prevalent. This would imply that minds biased to represent the world in terms of dual network structure are likely to be intelligent with respect to the everyday world.

The extreme commonality of symmetry groups in the (everyday and otherwise) physical world is another example: they occur so often that minds oriented toward recognizing patterns involving symmetry groups are likely to be intelligent with respect to the real world.

I suggest that the number of properties of the everyday world of this nature is huge ... and that the essence of everyday-world intelligence lies in the list of these abstract properties, which must be embedded implicitly or explicitly in the structure of a natural or artificial intelligence for that system to have everyday-world intelligence.

Apart from these particular yet abstract properties of the everyday world, intelligence is just about "finding patterns in which actions tend to achieve which goals in which situations" ... but, this simple meta-algorithm is well less than 1% of what it takes to make a mind.

You might say that a sufficiently generally intelligent system should be able to infer these general properties from looking at data about the everyday world. Sure. But I suggest that would require a massively greater amount of processing power than an AGI that embodies and hence automatically utilizes these principles? It may be that the problem of inferring these properties is so hard as to require a wildly infeasible AIXItl / Godel Machine type system.

Important Open Questions

A couple important questions raised by the above:

1. What is a reasonably complete inventory of the highly-intelligence-relevant subtle patterns/biases in the everyday world?
2. How different are the intelligence-relevant subtle patterns in the everyday world, versus the broader physical world (the quantum microworld, for example)?
3. How accurate a simulation of the everyday world do we need to have, to embody most of the subtle patterns that lie at the core of to everyday-world intelligence?
4. Can we create practical progressions of simulations of the everyday world, such that the first (and more crude) simulations are very useful to early attempts at teaching proto-AGIs, and the development of progressively more sophisticated simulations roughly tracks the development of progress in AGI design and development.

The second question relates to an issue I raised in a section of The Hidden Pattern, regarding the possibility of quantum minds -- minds whose internal structures and biases are adapted to the quantum microworld rather than to the everyday human physical world. My suspicion is that such minds will be quite different in nature, to the point that they will have fundamentally different mind-architectures -- but there will also likely be some important and fascinating points of overlap.

The third and fourth questions are ones I plan to explore in an upcoming paper, an expansion of the AGI-09 conference paper I wrote on AGI Preschool. An AGI Preschool as I define it there is a virtual world defining a preschool environment, with a variety of activities for young AI's to partake in. The main open question in AGI Preschool design at present is: How much detail does the virtual world need to have, to support early childhood learning in a sufficiently robust way? In other words, how much detail is needed so that the AGI Preschool will posssess the subtle structures and biases corresponding to everyday-world AGI? My AGI-09 conference paper didn't really dig into this question due to length limitations, but I plan to address this in a follow-up, expanded version.

The Increasing Value of Peculiar Intelligence

One more sociopolitical/futurist observation ...

If you're not aware of David Brin's Transparent Society concept, you should read the book, and start with the Web page

http://www.davidbrin.com/tschp1.html

His basic idea is that, as surveillance technology improves, there are two possibilities:

• The government and allied powers watch everybody, asymmetrically
• Everybody watches everyone else, symmetrically (including the government and allied powers being watched)

He calls the latter possibility sousveillance ... all-watching ...

What occurs to me is that in a transparent society, there is massive economic value attached to peculiar intelligence

This is because, if everyone can see everything else, the best way to gain advantage is to have something that nobody can understand even if they see it

And it's quite possible that, even if they know that's your explicit strategy, others can't really do anything to thwart it...

Yes, a transparent society could decide to outlaw inscrutability. But this would have terrible consequences, because nearly all radical advances are initially inscrutable....

Inscrutability is dangerous. But it's also, almost by definition, the only path to radical growth.

I argued in a recent blog post that part of the cause of the recent financial crisis is the development of financial instruments so complex that they are inscrutable to nearly everyone -- so that even if banks play by the rules and operate transparently, they can still trick shareholders (and journalists) because these people can't understand what they see!

But it seems that this recent issue with banks is just a preliminary glimmering of what's to come....

Yes, maybe there is something a bit self-centered or self-serving in this theory, since I seem to personally have more differential in "peculiar intelligence" than in most other qualities ... but, call it what you will, my peculiar intelligence is definitely pushing me to the conclusion that peculiar intelligence is going to be a more and more precious commodity as the Age of Transparency unfolds...

Obviously, you can also see this phenomenon in financial trading even in a non-crisis time. It's not enough to be smart at predicting the markets to make a LOT of money ... because if you're just smart, but in non-peculiar way, then once you start trading a lot of money, others will observe your trades and pick up the pattern, and start being smart in the same way as you ... and then you'll lose your edge. The way to REALLY make a lot of money in the markets is to be very smart, and in a very peculiar way, so that even if others watch your trades carefully, they can't understand the pattern, and they can't imitate your methodology....

The best trader I know personally is Jaffray Woodriff who runs quantitative.com, and he exemplifies this principle wonderfully: very intelligent, very peculiar (though in an extremely personable, enjoyable way), and very "peculiarly intelligent" ;-)

Democratic market socialism redux

All in all, the conclusion I'm coming to lately ... as reflected in my last two blog posts, as well as in some other thinking ... is that government is going to need to do some rather careful and specific things to guide society toward a positive Singularity.

Yes, if someone creates a hard-takeoff with a superhuman AGI, then the government and other human institutions may be largely irrelevant.

But if there is a soft takeoff lasting say 1-3 decades ... before the hard takeoff comes along ... then, my view is increasingly that the market system is going to screw things up, and lead to a situation where there are a lot of unhappy and disaffected people ... which increases the odds of some kind of nasty terrorist act intervening and preventing a positive Singularity from occurring.

It seems we may need to review the general line of thinking (though not many of the specific proposals) from old democratic-socialism-style texts like Economics and the Public Purpose ...

Perhaps one positive consequence of the current economic crisis is that it may cause the US public to understand the value of well-directed government spending....

And yes, I'm well aware that most of my colleagues in the futurist community tend to be libertarian politically. I think they're just wrong. I am all in favor of getting rid of victimless crimes ... legalizing drugs and prostitution and so forth ... but, given the realities of the next century and the risks of a negative Singularity, I don't think we can afford to leave things up to the unpredictable self-organizing dynamics of the market economy ... I think we as a society will need to reflect on what we want the path to Singularity to be, and take specific concerted governmental actions to push ourselves along that path...

This is not the political dialogue for 2008 ... different issues are occupying peoples' minds right now ... but it may be the political dialogue for 2012 or 2015 ... or at latest, I'd guess, 2020 ...

Why the average workweek isn't decreasing faster ... and what we can do about it

This is another post on political, economic and futurist themes ... starting out with a reflection on a bogus patent, and winding up with a radical social policy proposal that just might improve life in the near future and also help pave the way to a positive Singularity... (yeah, I know I know, lack of ambition has never been one of my numerous faults ;-)

Let's start with the simple, oft-asked question: Why isn't the average workweek decreasing faster ... given all the amazing technology recently developed?

One clue is found in some news I read today: IBM has patented the idea of a specialized electronic device that makes it handier to split your restaurant bill among several people at the table:

http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PALL&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.htm&r=1&f=G&l=50&s1=7,457,767.PN.&OS=PN/7,457,767&RS=PN/7,457,767

The patent application is really quite hilarious reading ... and the number of references to prior, equally bullshitty patents for related inventions is funny too.

What strikes me most here is the amount of effort expended by lawyers, patent examiners and judges in dealing with this crap. Not to mention their paralegals, secretaries, and on and on and on.

Why does this part of the contemporary human workload exist?

One could argue that some wasted work is a necessary part of a complex economic system, and that it would be hard to eliminate the crap without throwing out a lot of useful stuff as well.

I'm sure this is part of the truth, but I don't think it's the whole story.

Another part of the answer, I think, is: This kind of BS work exists because people have time to do it.

If people didn't have time to do this shit, because they all had to be occupied gathering food or making shelter or defending themselves against attackers -- or conceiving or manufacturing truly original and interesting inventions -- then the legal system would rapidly get adjusted so as to make bullshit patents like this illegal.

But, because we have plenty of extra resources in our economy ... due to the bounty created by advances in technology ... we (collectively and without explicitly intending to) adjust various parameters of our economy (such as the patent system) in such a way as to create loads of bullshit work for ourselves.

But there's a little more to it than this. Granted that we have the time ... but why do we choose to spend our time on this kind of crap? Instead of, say, doing less useless work and having more free time?

One important, relevant fact to grasp is that people like working.

Yes, it's not just that we're ambitious or greedy, it's that we actually like working -- in spite of the fact that we also like complaining about working. (Don't blame me -- humans are just fucked-up creatures....).

There is loads of data supporting this idea; I'll cite just a smattering here.

Americans work more and are happier than Europeans:

http://sayanythingblog.com/entry/americans_work_harder_happier_than_europeans/

Old folks who work longer are happier:

http://ideas.repec.org/p/pra/mprapa/5606.html

I can't remember the link, but studies show that people are on average happier at work than at home (if asked at random times during the day), but will when asked SAY they are happier at home than at work.... I think this study was done by Mihaly Csikszentmihalyi, who you can find out about at

http://www.time.com/time/health/article/0,8599,1606395,00.html

http://www.amazon.co.uk/Flow-Classic-Work-Achieve-Happiness/dp/0712657592

He is the pioneer of the study of "flow", the state of supreme happiness that many humans enter into when carrying out some highly absorbing, satisfying activity, like creating an artwork, participating in a sport, acting in a play or doing scientific or other professional work ... or gardening ... or chopping down a tree ... actually the activity can be almost anything, the real point is in the way the mind interacts with the activity: it has to be enough to fully occupy the mind without causing too much frustration, so that the self is continually lost and rediscovered within the ongoing dynamism of the interactive process....

Among many more interesting observations, he notes that:

"
Those at work generally report that they wish they were at home, but when they're home they often feel passive, depressed or bored. "They have in mind that free time at home will make them feel better, but often it doesn't,"
"

But, the plot thickens ... because, although we like working ... this doesn't necessarily mean we like working long hours.

People who work part-time are happier than those working full-time (84 per cent are happy versus 79 per cent):

http://www.arboraglobal.com/documents/Happiness%20at%20Work%20Index%202007.pdf

So where we seem to fail is in creating more part-time jobs...

This seems to be because, for an employer, it's always more cost efficient to hire one full time worker than two part-timers ;-(

So, instead of making use of the bounty provided by technology by creating part-time jobs and also creating more opportunities for creative, growth-producing, flow-inducing leisure ... we make use of it by creating more and more full-time jobs doing more and more bullshit work, like processing patents for "inventions" like the one I referenced above.

Given the way our brains work, we're better off working full-time than we would be not working at all.

But there is at least some evidence to suggest we'd be even better off working fewer hours....

But, given the way the market system works, there is almost never any incentive for any employer to allow part-time workers. It just isn't going to be rational from their perspective, as it will decrease their economic efficiency relative to competitors.

The market system, it seems, is going to push toward the endless creation of BS work, because no individual company cares about reducing the workweek ... whereas individual companies DO in many cases profit by creating BS work ... if they can bill other companies (or the government) for this bullshit work....

So, this leads to the idea that the government of France may have had the right idea at heart, in creating the 35 hour maximum workweek. Which they have since rescinded, because in a global context, it made them uncompetitive with other countries lacking such a restriction.

But anyway, outlawing long work hours is obviously not the answer. People should have the freedom to work long hours and get paid for them, if they want to.

So, an interesting question, policy-wise, is how the government can create incentives for reduced workweeks, without introducing punitive and efficiency-killing regulations.

One possibility would be, instead of just government projects aimed at paying people to build infrastructure, to launch government projects aimed at paying people to do interesting, creative, growth-inducing stuff in their spare time.

Basically, what I'm suggesting is: massively boosted government funding for art and science and other not-directly-economically-productive creative work ... but without making people jump through hoops of fire to get it (i.e. no long, tedious, nearly-sure-to-fail grant applications) ... and specifically designed NOT to discriminate against people who do not-directly-economically-productive creative work only part-time.

This would make it harder for companies to find full-time employees, because it wouldn't be all that much more lucrative to work full-time than to work part-time plus earn a creative-activity stipend on the side. But, unlike France's previous restrictive laws, it would enable companies that REALLY need full-time employees to hire them, so long as they were able to pay the required premium salaries ... or recruit workers who preferred the offered work to paid on-the-side creative-activity....

I suspect this would actually boost the economy, by shifting effort away from BS make-work like processing bogus patents, and toward creative work that would end up having more indirect, ancillary economic value in all sorts of ways.

This may seem a funny thing to think about in the current economic context, when the economy is in such trouble globally. But I consider it obvious that the current economic troubles are "just" a glitch (albeit an extremely annoying one) on the path to a post-scarcity economy. And even so, the government is still giving away money right now to people who are out of work. What if the payment were decreased to people who didn't engage in creative activities, but increased to people who did. Peoples' lives would become richer, as more of them would be more involved in creative activities. And, the human world as a whole would become richer because of all of these new creations.

And this sort of thing will become all the more interesting once robots and AI software eliminate more and more jobs. At that point the market system, unrestricted, would probably create an insane overgrowth of bullshit full-time jobs ... but a part-time creative-activity incentive as I've described could interfere with this dynamic, and nudge things toward a more agreeable situation where people divide their time between genuinely economically useful work, and non-directly-economically-useful but personally-and-collectively-rewarding creative activity.

Furthermore, this would create a culture of creative activity, that would serve us well once robots and AIs eliminate jobs altogether. It would be great if we could greet the Singularity with a culture of collective creativity rather than one characterized by a combination of long hours on useless bullshit jobs, combined with leisure time spent staring at the tube (whether the TV or the YouTube) or playing repetitive shoot-em-up video games ... etc. etc.

OK -- now ... time for me to get back to work ;-)

The Inevitable Increase of Irrationality (more musings on the recent financial crisis)

In a recent email dialogue with my dad, I found myself spouting off again about the possible causes of the recent financial crisis ... and found myself coming to a conclusion that may have some more general implications as we move forward toward Singularity.

Basically, I came to the conclusion that the financial crisis can be interpreted, in part, as a failure of the "rational actor" model in economics.

And furthermore, it's a failure of the rational actor model for an interesting reason: technology is advancing sufficiently that the world is too damn complex for most people to be rational about it, even if they want to be.

And this situation, I suggest, is likely to get worse and worse as technology advances.

Greenspan said something related in an interview shortly after the credit crunch hit: he said was shocked that major banks would deviate so far from rational self-interest.

To the extent this is the case, the recent crisis could be viewed as a failure of the rational-actor model -- and a validation of the need to view socioeconomic systems as complex self-organizing systems, involving psychological, sociological and economic dynamics all tangled together ... a need that will only increase as technology advances and makes it harder and harder for even smart rational-minded people to approximate rational economic judgments.

As a semi-aside: In terms of traditional economists, I'd say Galbraith's perspective probably accounts for this crisis best. He was always skeptical of over-mathematical approaches, and stressed the need to look at economic mechanisms in their social context. And of course, Obama's proposed solution to the problem is strongly Galbraithian in nature (except that Galbraith, not being an elected official, had the guts to call it "socialism" ;-)

Now, there is (of course) a counterargument to the claim that the recent financial crisis indicates a failure of the rational actor model. But one can also make a counter-counterargument, which I find more compelling.

The counterargument is: the banks as institutions were perhaps being irrational, but the individual decision-makers within the banks were being rational in that their incentive structures were asymmetric ... they got big bonuses for winning big, and small penalties for losing big. As a human being who is a banker, taking a huge gamble may be rational, if you get a huge bonus upon winning ... and upon losing, just need to go find another job (a relatively small penalty). So in that sense the individuals working in the banks may have been acting rationally ...

Yet the corporations were not acting rationally: which means that the bank shareholders were the ones not acting rationally, by not constraining the bank managers to put more appropriate incentive structures in place for their employees...

But why were the shareholders acting irrationally?

Well, I suggest that the reason the bank shareholders did not act rationally was, largely, out of ignorance.

Because the shareholders were just too ignorant of the actual risks involved in these complex financial instruments, not to mention of the incentive structures in place within the banks, etc.

We have plenty of legal transparency requirements in place, so that shareholders can see what's going on inside the corporations they invest in. But this is of limited value if the shareholders can't understand what's going on.

So, getting back to rational actor theory: the novel problem that we have here (added on top of the usual human problems of dishonesty, corruption and so forth) may be that, in a world that is increasingly complex (with financial instruments defined by advanced mathematics, for example), being a rational economic actor is too difficult for almost anybody.

The rational-actor assumption fails for a lot of reasons, as many economists have documented in the last few decades ... but this analysis of the current financial crisis suggests that as technology advances, it is going to fail worse and worse.

You could argue that this effect would be counterbalanced by the emergence of an increasingly effective professional "explainer" class who will boil down complex phenomena so that individuals (like bank shareholders) can make judgments effectively.

However, there are multiple problems with this.

For one thing, with modern media, there is so much noise out there that even if the correct explanations are broadcast to the world on the Web, the average person has essentially no way to select them from among the incorrect explanations. OK, they can assume the explanations given in the New York Times are correct ... but of course there is not really any objective and independent press out there, and "believing what the authorities tell you" is a strategy with many known risks.

And, secondly, it's not clear that the journalists of the world can really keep up with developments well enough to give good, solid, objective advice and explanations, even if that is their goal!

So we can expect that as Singularity approaches, the rational-actor model will deviate worse and worse from reality, making economics increasingly difficult to predict according to traditional methods.

Some folks thought that increasing technology would somehow decrease "market friction" and make markets more "efficient" ... in the technical sense of "efficiency," meaning that everyone is paying the mathematically optimal price for the things they buy.

But in fact, increasing technology seems to be increasing "market incomprehensibility" and hence, in at least some important cases, making markets LESS efficient ...

But of course, making markets less "efficient" in the technical sense doesn't necessarily make the economy less efficient in a more general sense.

The economy is, in some senses, becoming fantastically more and more efficient (at producing more and more interesting configurations of matter and mind given less and less usage of human and material resources) ... but it's doing so via complex, self-organizing dynamics ... not via libertarian-style, rational-actor-based free-market dynamics.

Interesting times ahead....

Glocal memory, neural nets, AI, psi

The dichotomy between localized and globalized (distributed) memory in the brain and world is pretty much bullcrap.

For a long time I've had the idea of harmonizing the two, and the Novamente and OpenCog AGI designs incorporate this harmony, as do the speculative brain theories I proposed in my books in the 1990s.

But I never really articulated the concept of global/local ... i.e. glocal ... memory in a general way before, which I've now done in a semi-technical essay called "Glocal Memory"

http://www.goertzel.org/dynapsyc/2008/glocal_memory.htm

I decided to write up the glocal memory stuff now because I'm writing a technical paper on glocal Hopfield neural nets and glocal attention allocation in OpenCog, and I wanted to have something to reference in the paper on glocal-memory-in-general, and there wasn't anything....

In case that's not odd enough for you, I also posted another essay using the glocal memory concept to give a possible model for how psi might work, building on others' ideas regarding filter and transmission models of mind:

http://www.goertzel.org/dynapsyc/2008/glocal_psi.htm

Whoop-di-doodly-dandy-oh

A meta-theory of consciousnes

Let

C = consciousness

P = physical reality

Then, the various theories of consciousness may be placed into 11 categories:

1. C and P are the same
   
2. C is a part of P
3. P is a part of C
4. C and P are parts of something else
5. C and P share a common part but are nonidentical
   
6. C and P are parts of each other (hyperset style)
   
7. C and P are separate -- somehow correlated but not via parthood
   
8. C does not exist, it's an illusion
9. P does not exist, it's an illusion
10. C and P are parts of each other, and also parts of something else
11. C and P are parts of each other, and also parts of something else, which is also a part of them
    

The word "part" should not be overinterpreted in the above, it's just used in a generic and informal sense.

I haven't yet broken this down mathematically, but on quick observation these seem the only logically possible relationships involving C, P and parthood (and allowing for circular parthood).

Each of these theory-categories has some adherents, and if I had more time and tolerance-for-boredom, I could divide existing theories of consciousness into these categories.

The question I want to ask here is: What could the relationship between C and P be, so as to explain why different individuals would propose each of the above 11 theory-categories?

My observation is that there is ONE category of theories that would explain all the 11 theory-categories as different views of a common reality, involving in some cases errors of omission but in no case errors of commission.

This is category 11: that C and P contain each other, and are also contained in something else that is also a part of them

If we posit an underlying model of the general form

C ={C0, P, D}

P = {P0, C, D}

D = {D0, C, P}

then we can see how the other 10 theory-categories would emerge via sins of omission:

1. C and P are the same results from ignoring C0 and P0
   
2. C is a part of P results from ignoring C0
   
3. P is a part of C results from ignoring P0
   
4. C and P are parts of something else ignores that D is a part of C and P, and that C and P are parts of each other
   
5. C and P share a common part but are nonidentical, is correct so long as one allows this common part to be a hyperset (which is C nested within P nested within C ... etc.)
   
6. C and P are parts of each other (hyperset style) results from ignoring D
   
7. C and P are separate -- somehow correlated but not via parthood -- results from projecting C into {C0}, P into {P0}
   
8. C does not exist, it's an illusion results from ignoring C
   
9. P does not exist, it's an illusion results from ignoring P
   
10. C and P are parts of each other, and also parts of something else, results from ignoring that C and P contain D
    

It is thus tempting to conjecture that 11 is the underlying reality, and people who posit the other 10 theories are simply ignoring some of the aspects of reality -- i.e. to use the old metaphor, they are like blind people each feeling a different part of the elephant and concluding that's all there is.

Question for reflection: what is D? ;-)

Ethics as an Attractor

While visiting my father a couple weeks ago, he talked a bit about what he sees as one of the core ideas of religion: the notion of some kind of "universal morality" or "universal moral force-or-field-or-whatever."

While he's not a big fan of the more superstitious aspects of historical or contemporary religions, he does believe there is a universal sense of right versus wrong. For instance, he feels that killing a random person on the street just because one is in a bad mood, is somehow REALLY wrong, not just "wrong according to some person or group's subjective belief system."

My main initial reaction to this idea was not understanding what it might mean. I'm enough of a postmodernist that absolutes don't make much sense to me.

Even "2+2=4" is only true relative to some specific formal system defining the terms involved. And " '2+2=4' is true relative to formal system F " is also only true relative to some metamathematical system -- and so on ... even in mathematics, you never escape the chain of indirections and get to something absolute.

But in the few days after we had that conversation, I thought a bit about what meaning could be assigned to his notion of universal morality within my postmodernist world-view. This is a topic I addressed in the final chapter of The Path to Posthumanity, but not from exactly this perspective: there I was more concerned with enunciating moral principles sufficiently abstract to guide the development of posthumans, rather than with debating the absoluteness or relativity of such principles.

An interesting perspective with pertinence to this issue is Mark Waser's argument that ethical, cooperative behavior (in some form) may be an "attractor" of social systems. Meaning roughly that:

• social systems without something like this are unlikely to survive ... except by adopting some form of ethical, cooperative behavior pattern as a norm
• as a new social system originates and gradually grows, it is likely to evolve its own form of ethical, cooperative behavior

I think this is an interesting perspective, and in the next paragraphs I'll point out some of its limits, and then connect it back to the conversation with my father.

To make the argument more concrete, I'll begin by defining "ethics" in my own special way -- probably not precisely the same as how Mark intends it; but then, in his paper Mark doesn't give a very precisely drawn definition. First of all I'll define a "code of social behavior" as a set of rules, habits or principles that some agent in a society adopts to guide its behavior toward other agents in that society. I'll then define an ethics as a code of social behavior such that

• it occurs in an agent that has the internal flexibility to plausibly adopt the ethics, or not, without causing some sort of immediate disaster for the agent
• it occurs in an agent that is capable of counterfactual reasoning regarding the social situations it regularly encounters (psychologists have various ways to test this, that have been used to study animals for example)
• it involves the agent carrying out behaviors that it reasons (counterfactually) it would not carry out if it did not adhere to the ethics
• it involves the agent carrying out behaviors that the agent believes will benefit other agents in the society

In short, I define an ethics as a code that agents can optionally adopt, and if they adopt it, they know they're taking actions to benefit others, that they reason they wouldn't take in the absence of the code of ethics.

This reminds of a story one of my great-uncles used to tell about my 5-year-old incarnation. He was watching me play with toys, and then my little sister Rebecca came up and asked for one of the toys I was most enjoying. After a moment of reflection, I gave it to her, and then I commented to him that "Sometimes it feels good to do something for someone else that you don't want to do."

Pertaining to Mark Waser's argument, we can then ask whether ethics in this sense is likely to be an attractor. There are two questions here, of course: will the existence of an ethics, any ethics, be an attractor; and will some specific ethics be an attractor. I'll deal mainly with the first question, because if "the existence of ethics" isn't an attractor, then obviously no specific ethics is going to be.

The main limit I see in Waser's argument (as ported to my definition of ethics) is that the argument doesn't seem to apply in cases where one or more of the members of the social system are vastly more intrinsically capable than the others (in relevant ways).

In societies of humans, it could be argued that unethical behavior is ultimately unstable, because the oppressed underclass (with asymmetrically little social power, but roughly equal intrinsic capabilities) will eventually revolt. But the possibility of revolt exists because outside of the scope of the social system, all healthy adult humans have roughly the same level of intrinsic capability.

One can imagine a scenario roughly like the one in H.G. Wells "The Time Machine," where a subset of society that actually is strongly more capable in relevant senses (smarter, saner, stronger) takes control and oppresses the less capable majority. Perhaps any adequately capable individual among the underclass is either killed or taken into the overclass ... and any inadequately capable person among the overclass is either killed or tossed into the underclass. In this kind of scenario, after a certain number of generations, one would have a situation in which there would be pressure for ethics within each class, but not ethics between them.

Nothing like the above has ever happened in human history, of course, and nor is it likely to. However, the case of future AI minds is somewhat different. All humans are built according to the same architecture and have roughly the same amount of intrinsic computational resources, but the same won't necessarily be the case for all AIs.

I see no reason to believe that existence-of-ethics will be attracting in societies involving members with strongly asymmetric capabilities. In fact, it seems it might be easier to frame an alternate argument: that in a society consisting of two groups of radically different degrees of intrinsic capability, the attractor will be

• ethical behavior within the overclass
• ethical behavior within the underclass
• exploitative behavior from the overclass to the underclass

A related situation is human behavior toward "lower animals" -- but this is a different sort of matter because animals don't meet the criteria of ethical agents I laid out above. Adult treatment toward children also doesn't quite fit the mold of this situation, because the intrinsic difference in capability between parents and children reverses as the children grow older (leading to sayings like, "Don't spank your kid too hard; when he grows up he'll be the one choosing your nursing home!").

One thus arrives at the hypothesis that a restricted form of Waser's argument might hold: maybe existence-of-ethics is an attractor in societies composed of agents with roughly equal intrinsic capabilities (in relevant situations).

As to what specific ethical codes may be attractors, it seems to me that is going to depend upon the specifics of the agents and societies. But the general phenomenon of choosing actions for others' benefit, that one knows one would not take in the absence of the ethical code, seems something that could plausibly be argued to serve as an attractor in any society of sufficiently flexible, intelligent organisms with roughly equal intrinsic relevant-capabilities.

Note what the notion of an attractor means here: essentially it means that if you have a society with the right characteristics that *almost* has ethics, then the society will eventually evolve ethics.

Ethics being an attractor doesn't imply that it must be the only attractor; there could be other attractors with very different properties. Arguing that any society with the right characteristics will necessarily evolve into a state supporting ethics, would be a stronger argument.

Another twist on this is obtained by thinking about the difference between the conscious and unconscious minds.

Let's say we're talking about a society consisting of agents with reflective, deliberative capability -- but with a lot of mental habits that aren't easily susceptible to deliberation. This is certainly the situation we humans are in: most of the unconscious behavior-patterns that govern us, are extremely difficult for us to pull into our theater of conscious reflection and reason about ... for a variety of reasons, including the limited capacity of our conscious theater, and the way much unconscious knowledge is represented in the brain, which is only laboriously translatable into the way the conscious theater wants to represent knowledge.

Then, it may be that ethics winds up getting largely packed into the unconscious part of the mind, which is hard to deliberatively reason about. This might happen, for instance, if ethics were largely taught by imitation and reinforcement, rather than by abstract instruction. And this does seem to be how early-childhood ethical instruction happens among humans. We correct a child for doing something bad and reward them for doing something good (reinforcement learning), and we indicate to them real-world everyday-life examples of ethical behavior (both via personal example and via fictional stories, movies and the like). Abstract ethical principles only make sense to them via grounding in this experiential store of ethical examples.

So, if ethics evolves in a society due to its attracting nature, and is memetically propagated largely through unconscious instruction, then in effect what is happening is that in many cases

• the reflective, deliberative mind is thinking about the individual organism and its utility
• the unconscious mind is thinking about the superorganism of the overall society, via the experientially inculcated ethical principles

The voice of the conscience is thus revealed as the voice of the existence-of-ethics attractor that superorganisms (I hypothesize, following Waser) inevitably settle into, assuming their member agents possess certain characteristics.

Where does this leave my dad's notion of a universal ethical force? It doesn't validate any such thing, in quite the sense that my dad seemed to mean it.

However, it does validate the notion that an unconscious sense of ethics may be universal in the sense of being an inevitable mathematical property of any society satisfying certain characteristics.

What does this mean for me, as a reasoning being with an intuitive, unconscious sense of ethics ... and also the deliberative capability to think about this ethical sense ... as well as some ability to modify this sense of ethics if I want to?

Among other things, it reminds me that the deliberative, ratiocinative aspect of the mind probably needs to be a little humbler than it sometimes gets to be, inside us hyperintellectual, nerdy types. The "I" of the deliberative mind is a symbol for the whole mind, rather than constituting the whole mind ... and there may be systematic patterns characterizing which of our mind-patterns get stored in easily-deliberatively-accessible form and which do not. So as frustrating as it can be to those of us in love with ratiocination, if one wants to be maximally rational, one must accept that sometimes "the unconscious knows best" ... even if one can't understand its reasons ... because through observation, imitation and reinforcement, it gained experiential knowledge that possesses a scale and (in some cases) a form that is not ratiocination-friendly (but may yet be rationally considered extremely useful for goal-achievement).

Unfortunately, the unconscious also makes a lot of mistakes and possesses a powerful capacity for tricking itself ... which, all in all, makes the business of being a finite-resources mind with an even-more-acutely-finite-resources deliberative/rational component, an inordinately tricky one.

I personally find these issues slightly less confusing if I view them from the perspective of pattern space. Suppose we consider the universe as set of patterns, governed by a variety of pattern-dynamics rules, including the rule that patterns tend to extend themselves. Different patterns have different degrees of power -- that is, they have differing probabilities of success in extending themselves. The arguments I've given above suggest that ethics, as an abstract pattern for organizing behaviors, is a pattern of considerable power, especially among societies of intelligent entities of roughly comparable capability. In this view there is no "universal moral force" -- the universal force is the tendency of patterns to extend themselves, and ethics as a pattern seems to contain a great deal of this force.

On these issues there are two fairly extreme points of view, which may be compactly summarized (I hope not parodized) as:

1. there is an absolutely correct ethics, an absolute right versus wrong, which may be imperfectly known to humans but which we can hope to better and better discern through various mental and social exercises
2. ethics is purely relative in nature: people adopt ethical codes because they were taught to, or maybe because their genes tell them to (i.e. because their species was taught to, by evolution) ... but there is no fundamental meaning to these ethics rather than social, psychological or evolutionary happenstance

Unlike my father, I never had much attraction to the first perspective. The second perspective has bedeviled me from time to time, yet I've always been nagged by a suspicion there's something deeper going on (yes, of course, someone can attribute this internal nagging to my psychology, my upbringing or my evolutionary heritage!). I don't delude myself I've fully gotten to the bottom of the issue here, but I hope that (building on the ideas of others) I've made a teeny bit of progress.

In What Sense Is "Overcoming Bias" a Good Goal?

This blog post consists of some musings that arose to me a few weeks ago on reading the multi-author blog Overcoming Bias. They were not triggered by the contents of the blog -- though those are sometimes quite interesting -- but merely by the name. My business in this post is to probe the extent to which "overcoming bias" is actually the right way to think about the problem of (as the Welcome page for the Overcoming Bias blog informally puts it) "How can we obtain beliefs closer to reality?"

I think it's clear that "overcoming bias" is important, but I also think it's important to explore and understand the limitations of "overcoming bias" as a methodology for obtaining beliefs that are more useful in achieving one's goals.

(Note that in my own thinking, I tend to more often think in terms of "obtaining beliefs that are more useful in achieving one's goals," rather than in terms of "obtaining beliefs that are closer to reality." In many contexts this just amounts to a nitpick, but it also reflects a significant philosophical distinction. I don't make the philosophical assumption of an objective reality, to which beliefs are to be compared. My philosophical view of beliefs could be loosely considered Nietzschean, though I doubt it agrees with Nietzsche's views in all respects.)

According to Wikipedia,

"
Bias is a term used to describe a tendency or preference towards a particular perspective, ideology or result, especially when the tendency interferes with the ability to be impartial, unprejudiced, or objective. The term biased is used to describe an action, judgment, or other outcome influenced by a prejudged perspective.
"

This definition is worth dissecting because it embodies two different aspects of the "bias" concept, which are often confused in ordinary discourse. I'll call these:

Bias_1: "a tendency or preference toward a particular perspective"

Bias_2: "an instance of Bias_1 that interferes with the ability to be impartial, unprejudiced or objective", which I'll replace with "... interferes with the ability to achieve one's goals effectively."

Bias_1 is, obviously, not necessarily a bad thing.

First of all, the universe we live in has particular characteristics (relative to a universe randomly selected from a sensibly-defined space of possible universes), and being biased in a way that reflects these characteristics may be a good thing.

Secondly, the particular environments and goals that an organism lives in, may have particular characteristics, and it may benefit the organism to be biased in a way that reflects these characteristics.

Now, ideally, an organism would be aware of its environment- or goal- specific biases, so that if its environment or goals change, it can change its biases accordingly. On the other hand, it may be that maintaining this awareness detracts from the organism's ability to achieve goals, if it consumes a lot of resources that could otherwise be spent doing other stuff (even if this other stuff is done in a way that's biased toward the particular environment and goals at hand).

When discussed in a political context, "bias" is assumed to be a bad thing, as in the "especially when" clause of the above Wikipedia definition. Gender bias and racial bias are politically incorrect and according to most modern moral systems, immoral. The reason these biases are considered to be bad is rooted in the (correct, in most cases) assumption that they constitute Bias_2 with respect to the goals that most modern moral systems say we should have.

On the other hand, in cognitive science, bias is not always a bad thing. One may argue, as Eric Baum has done persuasively in What Is Thought?, that the human mind's ability to achieve its goals in the world is largely due to the inductive bias that it embodies, which is placed into it via evolutionary pressure on brain structure. In this context, bias is a good thing. The brain is a general-purpose intelligence, but it is biased to be able to more easily solve some kinds of problems (achieve some kinds of goals) than others. Without this biasing, there's no way a system with the limited computational capacity of the human brain would be able to learn and do all the things it does in the short lifespan of a human organism. The inductive bias that Baum speaks about is largely discussed as Bias_1, but also may in some cases function as Bias_2, because biases that are adaptive in some circumstances may be maladaptive in others.

One might argue that, in the case of evolved human inductive bias, it's the evolutionary process itself that has been less biased, and has (in a relatively unbiased way) evolved brains that are biased to the particular conditions on Earth. However, this is not entirely clear. The evolutionary mechanisms existing on Earth have a lot of particularities that seem adapted to the specific chemical conditions on Earth, for example.

One may argue that, even though we humans are born with certain useful biases, it is to our advantage to become reflective and deliberative enough to overcome these biases in those cases where they're not productive. This is certainly true -- to an extent. However, as noted above, it's also true that reflection and deliberation consume a lot of resources. Any organism with limited resources has to choose between spending its resources overcoming its biases (which may ultimately help it to achieve its goals), and spending its resources achieving its goals in a direct ways.

Furthermore, it's an interesting possibility that resource-constrained minds may sometimes have biases that help them achieve their goals, yet that they are not able to effectively reflect and deliberate on. Why might this be? Because the class of habits that an organism can acquire via reinforcement learning, may not fully overlap with the class of habits that the organism can study via explicit reflective, deliberative inference. For any particular mind-architecture, there are likely to be some things that are more easily learnable as "experientially acquired know-how" than as explicit, logically-analyzable knowledge. (And, on the other hand, there are going to be other things that are more easily arrived at via explicit inference than via experiential know-how acquisition.)

If a certain habit of thought is far more amenable to experiential reinforcement based learning than reflective, logical deliberation, does this mean that one cannot assess its quality, with a view toward ridding it of unproductive biases? Not necessarily. But overcoming biases in these habits may be a different sort of science than overcoming biases in habits that are more easily susceptible to reason. For instance, the best way to overcome these sorts of biases may be to place oneself in a large variety of different situations, so as to achieve a wide variety of different reinforcement signaling patterns ... rather than to reflectively and deliberatively analyze one's biases.

Many of these reflections ultimately boil down to issues of the severely bounded computational capability of real organisms. This irritating little issue also arises when analyzing the relevance of probabilistic reasoning (Bayesian and otherwise) to rationality. If you buy Cox's or de Finetti's assumptions and arguments regarding the conceptual and mathematical foundations of probability theory (which I do), then it follows that a mind, given a huge amount of computational resources, should use probability theory (or do something closely equivalent) to figure out which actions it should take at which times in order to achieve its goals. But, these nice theorems don't tell you anything about what a mind given a small or modest amount of computational resources should do. A real mind can't rigorously apply probability theory to all its judgments, it has to make some sort of heuristic assumptions ... and the optimal nature of these heuristic assumptions (and their dependencies on the actual amount of space and time resources available, and the specific types of goals and environments involved, etc.) is something we don't understand very well.

So, the specific strategy of overcoming Bias_2 by adhering more strictly to probability theory, is interesting and often worthwhile, but not proven (nor convincingly argued) to always the best thing to do for real systems in the real world.

In cases where the answer to some problem can be calculated using probability theory based on a relatively small number of available data items ... or a large number of data items that interrelate in a relatively simply characterizable way ... it's pretty obvious that the right thing for an intelligent person to do is to try to overcome some of their evolutionary biases, which may have evolved due to utility in some circumstances, but which clearly act as Bias_2 in many real-world circumstances. The "heuristics and biases" literature in cognitive psychology contains many compelling arguments in this regard. For instance, in many cases, it's obvious that the best way for us to achieve our goals is to learn to replace our evolved mechanisms for estimating degrees of probability, with calculations more closely reflecting the ones probability theory predicts. Professional gamblers figured this out a long time ago, but the lesson of the heuristics and biases literature has been how pervasive our cognitive errors (regarding probability and otherwise) are in ordinary life, as well as in gambling games.

On the other hand, what about problems a mind confronts that involve masses of different data items, whose interrelationships are not clear, and about which much of the mind's knowledge was gained via tacit experience rather than careful inference or scientific analysis?

Many of these problems involve contextuality, which is a difficult (though certainly not impossible) thing to handle pragmatically within formal reasoning approaches, under severe computational resource constraints.

For these problems, there seem to be two viable strategies for improving one's effectiveness at adapting one's beliefs so as to be able to more adeptly achieve one's goals:

1. Figure out a way to transform the problem into the kind that can be handled using explicit rational analysis
2. Since so much of the knowledge involved was gained via experiential reinforcement-learning rather than inference ... seek to avoid Bias_2 via achieving a greater variety of relevant experiences

So what's my overall takeaway message?

• We're small computational systems with big goals, so we have to be very biased, otherwise we wouldn't be able to achieve our goals
• Distinguishing Bias_1 from Bias_2 is important theoretically, and also important *but not always possible* in practice
• The right way to cure instances of Bias_2 depends to some extent on the nature of the mental habits involved in the bias
• In some cases, diversity of experience may be a better way to remove Bias_2, than explicit adherence to formal laws of rationality
• It is unclear in which circumstances (attempted approximate) adherence to probability theory or other formal laws of rationality is actually the right thing for a finite system to do, in order to optimally achieve its goals
• Heuristically, it seems that adherence to formal laws of rationality generally makes most sense in cases where contextuality is not so critical, and the relevant judgments depend sensitively mainly on a relatively small number of data items (or a large number of relatively-simply-interrelated data items)

In Search of the "Machines of Loving Grace": AI, Robotics and Empathy

Empathy: the ability to feel each others’ feelings. It lies at the core of what makes us human. But how important will it be to the artificial minds we one day create? What are AI researchers doing to imbue their creations with artificial empathy ... and should they be doing more? In short, what is the pathway to the “machines of loving grace” that poet Richard Brautigan foresaw?

The mainstream of AI research has traditionally focused on the more explicitly analytical, intellectual, nerdy aspects of human intelligence: planning, problem-solving, categorization, language understanding. Recent attempts to broaden this focus have focused mainly on creating software with perceptual and motor skills: computer vision systems, intelligent automated vehicles, and so forth. Missing almost entirely from the AI field is the more social and emotional aspects of human intelligence. Chatbots attempting the Turing test have confronted these aspects directly – going back to ELIZA, the landmark AI psychotherapist from the early 1970’s – but these bots are extremely simplistic and have little connection to the main body of work in the AI field.

I think this is a major omission, and my own view is that empathy may be one of the final frontiers of AI. My opinion as an AI researcher is that, if we can crack artificial empathy, the rest of the general-AI problem will soon follow, based on the decades of successes that have already been achieved in problem-solving, reasoning, perception, motorics, planning, cognitive architecture and other areas.

(In my own AI work, involving the Novamente Cognition Engine and the OpenCog Prime system, I’ve sought to explicitly ensure the capability for empathy via multiple coordinated design aspects – but in this blog post I’m not going to focus on that, restricting myself to more general issues.)

Why would empathy be so important for AI? After all, it’s just about human feelings, which are among the least intelligent, most primitively animal-like aspect of the human mind? Well, the human emotional system certainly has its quirks and dangers, and the wisdom of propagating these to powerful AI systems is questionable. But the basic concept of an emotion, as a high-level integrated systemic response to a situation, is critical to functioning of any intelligent system. An AI system may not have the same specific emotions as a human being – particular emotions like love, anger and so forth are manifestations of humans’ evolutionary heritage, rather than intrinsic aspects of intelligence. But it seems unlikely that an AI without any kinds of high-level integrated systemic responses (aka emotions) would be able to cope with the realities of responding to a complex dynamic world in real-time.

A closely related point is the social nature of intelligence. Human intelligence isn’t as individual as we modern Westerners often seem to think: a great percentage of our intelligence is collective and intersubjective. Cognitive psychologists have increasingly realized this in recent decades, and have started talking about “distributed cognition.” If the advocates of the “global brain” hypothesis are correct, then eventually artificial minds will synergize with human minds to form a kind of symbiotic emergent cyber-consciousness. But in order for distributed cognition to work, the minds in a society need to be able to recognize, interpret and respond to each others’ emotions. And this is where empathy comes in.

Mutual empathy binds together social networks: as we go through our lives we are perpetually embodying self-referential emotional equations like

X = I feel pain and that you feel X



Y = I feel that you feel both joy and Y

or mutually-referential ones like

A = I feel happy that you enjoy both this music and B

B = I feel surprised that you feel A

These sorts of equations bind us together: as they unfold through time they constitute much of the rhythm by which our collective intelligence experiences and creates.

So empathy is important: but how does it work?

We don’t yet know for sure ... but the best current thinking is that there are two aspects to how the brain does empathy: inference and simulation. (And I do think this is a lesson for AI: in my own AI designs I deal with these aspects separately, and then address their interaction ... and I do think this is the right approach.)

Inference-wise, empathy has to do with understanding and modeling (sometimes consciously, sometimes unconsciously) what another person must be feeling, based on the cues we perceive and our background knowledge. Psychologists have mapped out transformational rules that help us do this modeling.

Simulative empathy is different: we feel what each other are feeling. A rough analogue is virtualization in computers: running Windows on a virtual machine within Linux; emulating an Activision console within Windows. Similarly, we use the same brain-systems that are used to run ourselves, to run a simulation of another person feeling what they seem to be feeling. And we do this unconsciously, at the body level: even though we don’t consciously notice that sad people have smaller pupils, our pupils automatically shrink when we see a sad person -- a physiological response that synergizes with our cognitive and emotional response to their sadness (and is the technique the lead character uses in Blade Runner to track down androids who lack human feeling). A long list of examples has been explored in the lab already, and we’ve barely scratched the surface yet: people feel disgust when they see others smell bad odor, they feel pain when they see others being pierced by a needle or get electrical shock, they sense touching when they see others being brushed, etc.

Biologists have just started to unravel the neural basis of simulative empathy, which seems to involve brain cells called mirror neurons ... which some have argued play a key role in other aspects of intelligence as well, including language learning and the emergence of the self (I wrote a speculative paper on this a couple years back).

(A mirror neuron is a neuron which fires both when an animal acts and when the animal observes the same action performed by another animal (especially one of the same species). Thus, the neuron "mirrors" the behavior of another animal, as though the observer were itself acting. These neurons have been directly observed in primates, and are believed to exist in humans and in some birds. In humans, brain activity consistent with mirror neurons has been found in the premotor cortex and the inferior parietal cortex.)

So: synergize inference and simulation, and you get the wonderful phenomenon of empathy that makes our lives so painful and joyful and rich, and to a large extent serves as the glue holding together the social superorganism.

The human capacity for empathy is, obviously, limited. This limitation is surely partly due to our limited capabilities of both inference and simulation; but, intriguingly, it might also be the case that evolution has adaptively limited the degree of our empathic-ness. Perhaps an excessive degree of empathy would have militated against our survival, in our ancestral environments?

The counterfactual world in which human empathy is dramatically more intense is difficult to accurately fathom. Perhaps, if our minds were too tightly coupled emotionally, progress would reach the stage of some ant-colony-like utopia and then halt, as further change would be too risky in terms of hurting someone else’s feelings. On the other hand, perhaps a richer and more universal empathy would cause a dramatic shift in our internal architectures, dissolving or morphing the illusion of “self” that now dominates our inner worlds, and leading to a richer way of individually/collectively existing.

One aspect of empathy that isn’t sufficiently appreciated is the way it reaches beyond the touchy-feely sides of human life: for instance it pervades the worlds of science and business as well, which is why there are still so many meetings in the world, email, Skype and WebEx notwithstanding. The main reason professionals fly across the world to hob-nob with their colleagues – in spite of the often exhausting and tedious nature of business travel (which I’ve come to know all too well myself in recent years) -- is because, right now, only face-to-face communication systematically gives enough of the right kind of information to trigger empathic response. In a face-to-face meeting, humans can link together into an empathically-joined collective mind-system, in a way that doesn’t yet happen nearly as reliably via electronically-mediated communications.

Careful study has been given to the difficulty we have empathizing with certain robots or animated characters. According to Mori’s theory of the “uncanny valley” – which has been backed up by brain imaging studies -- if a character looks very close to human, but not close enough, then people will find it disturbing rather than appealing. We can empathize more with the distorted faces of Disney cartoons or manga, than with semi-photo-realistic renditions of humans that look almost-right-but-eerily-off.

To grasp the uncanny valley viscerally, watch one of the online videos of researcher Hiroshi Ishiguro and the “geminoid” robot that is his near-physical-clone -- an extremely lifelike imitation of his own body and its contours, textures and movements. No AI is involved here: the geminoid is controlled by motion-capture apparatus that watches what Ishiguro does and transfers his movements to the robot. The imitation is amazing – until the bot starts moving. It looks close enough to human that its lack of subtle human expressiveness is disturbing. We look at it and we try to empathize, but we find we’re empathizing with a feelingless robot, and the experience is unsettling and feels “wrong.” Jamais Cascio has proposed that existly this kind of reaction may occur to transhumans with body modifications – so from that point of view among others, this phenomenon may be worth attending.

It’s interesting to contrast the case of the geminoid, though, with the experience of interacting with ELIZA, the AI psychotherapist created by Joseph Weizenbaum in 1966. In spite of having essentially no intrinsic intelligence, ELIZA managed to carry out conversations that did involve genuine empathic sharing on the part of its conversaton-partners. (I admit ELIZA didn’t do much for me even back then, but, I encountered it knowing exactly what it was and intrigued by it from a programming perspective, which surely colored the nature of my experience.)

Relatedly, some people today feel more empathy with their online friends than their real-life friends. And yet, I can’t help feel there’s something key lacking in such relationships.

One of the benefits of online social life is that one is freed from the many socio-psychological restrictions that come along with real-world interaction. Issues of body image and social status recede into the background – or become the subject of wild, free-ranging play, as in virtual worlds such as Second Life. Many people are far less shy online than in person – a phenomenon that’s particularly notable in cultures like Japan and Korea where social regulations on face-to-face communcation are stricter.

And the benefits can go far beyond overcoming shyness: for example, a fifty-year-old overweight trucker from Arkansas may be able to relate to others more genuinely in the guise of a slender, big-busted Asian girl with blue hair, a microskirt and a spiky tail... and in Second Life he can do just that.

On the other hand, there’s a certain falsity and emotional distance that comes along with all this. The reason the trucker can impersonate the Asian ingenue so effectively is precisely that the avenues for precise emotional expression are so impoverished in today’s virtual environments. So, the other fifty-year-old trucker from Arkansas whose purple furry avatar is engaged in obscene virtual acts with the Asian babe, has to fill in the gaps left by the simplistic technology – to a large extent, the babe he’s interacting with is a construct of his own mind, which improvises on the cues provided by the signals given by the first trucker.

Of course, all social interaction is constructive in this way: the woman I see when I talk to my wife is largely a construct of my own mind, and may be a different woman than I would see if I were in a different mood (even if her appearance and actions were precisely the same). But text-chat or virtual-world interactions are even more intensely constructive, which is both a plus and a minus. We gain the ability for more complete wish-fulfillment (except for wishes that are intrinsically tied to the physical ... though some people do impressively well as satisfying virtual satisfactions for physical ones), but we lose much of the potential for growing in new directions via empathically absorbing emotional experiences dramatically different from anything we would construct on our own based on scant, sketchy inputs.

It will be interesting to see how the emotional experience of virtual world use develops as the technology advances ... in time we will have the ability to toggle how much detail our avatars project, just as we can now choose whether to watch cartoons or live action films. In this way, we will be able to adjust the degree of constructive wish-fulfillment versus self-expanding experience-of-other ... and of course to fulfill different sorts of wishes than can be satisfied currently in physical or virtual realities.

As avatars become more realistic, they may encounter the uncanny valley themselves: it may be more rewarding to look at a crude, iconic representation of someone else’s face, than a representation that’s almost-there-but-not-quite ... just as with Ishiguro’s geminoids. But just as with the geminoids, the technology will get there in time.

The gaming industry wants to cross the uncanny valley by making better and better graphics. But will this suffice? Yes, a sufficiently perfected geminoid or game character will evoke as much empathy as a real human. But for a robot or game character controlled by AI software, the limitation will probably lie in subtleties of movement. Just like verbal language, the language of emotional gestures is one where it’s hard to spell out the rules exactly: we humans grok them from a combination of heredity and learning. One way to create AIs that people can empathize with will be to make the AIs themselves empathize, and reflect back to people the sorts of emotions that they perceive. Much as babies imitate adult emotions. Envision a robot or game character that watches a video-feed of your face and tailors its responses to your emotions.

Arguably, creating AIs capable of empathy has importance far beyond the creation of more convincing game characters. One of the great unanswered questions as the Singularity looms is how to increase the odds that once our AIs get massively smarter than we are, they still value our existence and our happiness. Creating AIs that empathize with humans could be part of the answer.

Predictably, AI researchers so far have done more with the inferential than the simulative side of empathic response. Selmer Bringsjord’s team at RPI got a lot of press earlier this year for an AI that controls a bot in Second Life, in a way that demonstrates a limited amount of “theory of mind”: the bot watches other characters with a view toward figuring out what they’re aware of, and uses this to predict their behavior and guide its interactions.

But Bringsjord’s bots don’t try to feel the feelings of the other bots or human-controlled avatars they interact with. The creation of AIs embodying simulative empathy seems to be getting very little attention. Rosalind Picard’s Affective Computing Lab at MIT has done some interesting work bringing emotion into AI decision processses but has stopped short of modeling simulative empathy. But I predict this is a subfield that will emerge within the next decade. In fact, it seems plausible that AI’s will one day be far more empathic than humans are – not only with each other but also with human beings. Ultimately, an AI may be able to internally simulate you better than your best human friend, and hence demonstrate a higher degree of empathy. Which will make our games more fun, our robots less eerie, and potentially help make the post-Singularity world a more human-friendly place.

Zarathustra, Plato, Saving Boxes, Oracle Machines and Pineal Antennae

Reading over the conversation I had (with Abram Demski) in the Comments to a prior blog post

http://multiverseaccordingtoben.blogspot.com/2008/10/are-uncomputable-entities-useless-for.html

I was reminded of a conversation I had once with my son Zarathustra when he was 4 years old.

Zar was defending his claim that he actually was omniscient, and explaining how this was consistent with his apparent ignorance on many matters. His explanation went something like this:

"I actually do know everything, Ben! It's just that with all that stuff in my memory, it can take me a really really long time to get the memories out ... years sometimes...."

Of course, Zar didn't realize Plato had been there before (since they didn't cover Plato in his pre-school...).

He also had the speculation that this infinite memory store, called his "saving box", was contained in his abdomen somewhere, separate from his ordinary, limited-scope memories in his brain. Apparently his intuition for philosophy was better than for biology... or he would have realized it was actually in the pineal gland (again, no Descartes in preschool either ;-p).

This reminded me of the hypothesis that arose in the conversation with Abram, that in effect all humans might have some kind of oracle machine in their brains.

If we all have the same internal neural oracle machine (or if, say, we all have pineal-gland antennas to the the same Cosmic Oracle Machine (operated by the ghost of Larry Ellison?)), then we can communicate about the uncomputable even though our language can never actually encapsulate what it is we're talking about.

Terrence McKenna, of course, had another word for these quasi-neural oracle machines: machine-elves ;-)

This means that the real goal of AGI should be to create a software program that can serve as a proper antenna 8-D

Just a little hi-fi sci-fi weirdness to brighten up your day ... I seem to have caught a bad cold and it must be interfering with my thought processes ... or messing up the reception of my pineal antenna ...

P.S.

perhaps some evidence for Zar's saving-box theory:

http://hubpages.com/hub/Cellular-Memories-in-Organ-Transplant-Recipients

Random Memory of a Creative Mind (Paul Feyerabend)

I had a brief but influential (for me: I'm sure he quickly forgot it) correspondence with the philosopher-of-science Paul Feyerabend when I was 19.

I sent him a philosophical manuscript of mine, printed on a crappy dot matrix printer ... I think it was called "Lies and False Truths." I asked him to read it, and also asked his advice on where I should go to grad school to study philosophy. I was in the middle of my second year of grad school, working toward my PhD in math, but I was having second thoughts about math as a career....

He replied with a densely written postcard, saying he wasn't going to read my book because he was spending most of his time on non-philosophy pursuits ... but that he'd glanced it over and it looked creative and interesting (or something like that: I forget the exact words) ... and, most usefully, telling me that if I wanted to be a real philosopher I should not study philosophy academically nor become a philosophy professor, but should study science and/or arts and then pursue philosophy independently.

His advice struck the right chord and the temporary insanity that had caused me to briefly consider becoming a professional philosopher, vanished into the mysterious fog from which it had emerged ...

(I think there may have been another couple brief letters back and forth too, not sure...)

(I had third thoughts about math grad school about 6 months after that, and briefly moved to Vegas to become a telemarketer and Henry-Miller-meets-Nietzsche style prose-poem ranter ... but that's another story ... and anyways I went back to grad school and completed my PhD fairly expeditiously by age 22...)


P.S.

Even at that absurdly young age (but even more so now), I had a lot of disagreements with Feyerabend's ideas on philosophy of science -- but I loved his contentious, informal-yet-rigorous, individualistic style. He thought for himself, not within any specific school of thought or tradition. That's why I wrote to him -- I viewed him as a sort of kindred maverick (if that word is still usable anymore, given what Maverick McCain has done to it ... heh ;-p)

My own current philosophy of science has very little to do with his, but, I'm sure we would have enjoyed arguing the issues together!

He basically argued that science was a social phenomenon with no fixed method. He gave lots of wonderful examples of how creative scientists had worked outside of any known methods.

While I think that's true, I don't think it's the most interesting observation one can make about science ... it seems to me there are some nice formal models you can posit that are good approximations explaining a lot about the social phenomenon of science, even though they're not complete explanations. The grungy details (in chronological order) are at:

• http://www.goertzel.org/dynapsyc/2004/PhilosophyOfScience_v2.htm (included in The Hidden Pattern)
• http://multiverseaccordingtoben.blogspot.com/2008/10/reflections-on-religulous-and.html.
• http://multiverseaccordingtoben.blogspot.com/2008/10/are-uncomputable-entities-useless-for.html

But, one thing I did take from Feyerabend and his friend/argument-partner Imre Lakatos was the need to focus on science as a social phenomenon. What I've tried to do in my own philosophy of science is to pull together the social-phenomenon perspective with the Bayesian-statistics/algorithmic-information perspective on science.... But, as usual, I digress!

hiccups on the path to superefficient financial markets

A political reporter emailed me the other day asking my opinion on the role AI technology played in the recent financial crisis, and what this might imply for the future of finance.

Here's what I told him. Probably it freaked him out so much he deleted it and wiped it from his memory, but hey...

There's no doubt that advanced software programs using AI and other complex techniques played a major role in the current global financial crisis. However, it's also true that the risks and limitations of these software programs were known by many of the people involved, and in many cases were ignored intentionally rather than out of ignorance.

To be more precise: the known mathematical and AI techniques for estimating the risk of complex financial instruments (like credit default swaps, and various other exotic derivatives) all depend on certain assumptions. At this stage, some human intelligence is required to figure out whether the assumptions of a given mathematical technique really apply in a certain real-world situation. So, if one is confronted with a real-world situation where it's unclear whether the assumptions of a certain mathematical technique really apply, it's a human decision whether to apply the technique or not.

A historical example of this problem was the LTCM debacle in the 90's. In that case, the mathematical techniques used by LTCM assumed that the economies of various emerging markets were largely statistical independent. Based on that assumption, LTCM entered into some highly leveraged investments that were low-risk unless the assumption failed. The assumption failed.

Similarly, more recently, Iceland's financial situation was mathematically assessed to be stable, based on the assumption that (to simplify a little bit) a large number of depositors wouldn't decide to simultaneously withdraw a lot of their money. This assumption had never been violated in past situations that were judged as relevant. Oops.

A related, obvious phenomenon is that sometimes humans assigned with the job of assessing risk are given a choice between:

1. assessing risk according to a technique whose assumptions don't really apply to the real-world situation, or whose applicability is uncertain
2. saying "sorry, I don't have any good technique for assessing the risk of this particular financial instrument"

Naturally, the choice commonly taken is 1 rather than 2.

In another decade or two, I'd predict, we'll have yet more intelligent software, which is able to automatically assess whether the assumptions of a certain mathematical technique are applicable in a certain context. That would avoid the sort of problem we've recently seen.

So the base problem is that the software we have now is good at making predictions and assessments based on contextual assumptions ... but it is bad at assessing the applicability of contextual assumptions. The latter is left to humans, who often make decisions based on emotional bias, personal greed and so forth rather than rationality.

Obviously, the fact that a fund manager shares more in their fund's profit than in its loss, has some impact in their assessments. This will bias fund managers to take risks, because if the gamble comes out well, they get a huge bonus, but if it comes out badly, the worst that happens is that they find another job.

My feeling is that these sorts of problems we've seen recently are hiccups on the path to superefficient financial markets based on advanced AI. But it's hard to say exactly how long it will take for AI to achieve the needed understanding of context, to avoid this sort of "minor glitch."

P.S.

After I posted the above, there was a followup discussion on the AGI mailing list, in which someone asked me about applications of AGI to investment.

My reply was:

1)
Until we have a generally very powerful AGI, application of AI to finance will be in the vein of narrow-AI. Investment is a hard problem, not for toddler-minds.

Narrow-AI applications to finance can be fairly broad in nature though, e.g. I helped build a website called stockmood.com that analyzes financial sentiment in news

2)
Once we have a system with roughly adult-human-level AGI, then of course it will be possible to create specialized versions of this that are oriented toward trading, and these will be far superior to humans or narrow AIs at trading the markets, and whomever owns them will win a lot of everybody's money unless the government stops them.

P.P.S.

Someone on a mailing list pushed back on my mention of "AI and other mathematical techniques."

This seems worth clarifying, because the line between narrow-AI and other-math-techniques is really very fuzzy.

To give an indication of how fuzzy the line is ... consider the (very common) case of multiextremal optimization.

GA's are optimization algorithms that are considered AI ... but, is multi-start hillclimbing AI? Many would say so. Yet, some multiextremal optimization algorithms are considered operations research instead of AI -- say, multistart conjugate gradients...

Similarly, backprop NN's are considered AI .. yet, polynomial or exponential regression algorithms aren't. But they pretty much do the same stuff...

Or, think about assessment of credit risk, to determine who is allowed to get what kind of mortgage. This is done by AI data mining algorithms. OTOH it could also be done by some statistical algorithms that wouldn't normally be called AI (though I think it is usually addressed using methods like frequent itemset mining and decision trees, that are considered AI).

Are Uncomputable Entities Useless for Science?

When I first learned about uncomputable numbers, I was profoundly disturbed. One of the first things you prove about uncomputable numbers, when you encounter them in advanced math classes, is that it is provably never possible to explicitly display any example of an uncomputable number. But nevertheless, you can prove that (in a precise mathematical sense) "almost all" numbers on the real number line are uncomputable. This is proved indirectly, by showing that the real number line as a whole has one order of infinity (aleph-one) and the set of all computers has another order of infinite (aleph-null).

I never liked this, and I burned an embarrassing amount of time back then (I guess this was from ages 16-20) trying to find some logical inconsistency there. Somehow, I thought, it must be possible to prove this notion of "a set of things, none of which can ever actually be precisely characterized by any finite description" as inconsistent, as impossible.

Of course, try as I might, I found no inconsistency with the math -- only inconsistency with my own human intuitions.

And of course, I wasn't the first to tread that path (and I knew it). There's a philosophy of mathematics called "constructivism" which essentially bans any kind of mathematical entity whose existence can only be proved indirectly. Related to this is a philosophy of math called "intuitionism."

A problem with these philosophies of math is that they rule out some of the branches of math I most enjoy: I always favored continuous math -- real analysis, complex analysis, functional analysis -- over discrete math about finite structures. And of course these are incredibly useful branches of math: for instance, they underly most of physics.

These continuity-based branches of math also underly, for example, mathematical finance, even though the world of financial transactions is obviously discrete and computable, so one can't possibly need uncomputable numbers to handle it.

There always seemed to me something deeply mysterious in the way the use of the real line, with its unacceptably mystical uncomputable numbers, made practical mathematics in areas like physics and finance so much easier.

Notice, this implicitly uncomputable math is never necessary in these applications. You could reformulate all the equations of physics or finance in terms of purely discrete, finite math; and in most real applications, these days, the continuous equations are solved using discrete approximations on computers anyway. But, the theoretical math (that's used to figure out which discrete approximations to run on the computer) often comes out more nicely in the continuous version than the discrete version. For instance, the rules of traditional continuous calculus are generally far simpler and more elegant than the rules of discretized calculus.

And, note that the uncomputability is always in the background when you're using continuous mathematics. Since you can't explicitly write down any of these uncomputable numbers anyway, they don't play much role in your practical work with continuous math. But the math you're using, in some sense, implies their "existence."

But what does "existence" mean here?

To quote former President Bill Clinton, "it all depends on what the meaning of the word is, is."

A related issue arises in the philosophy of AI. Most AI theorists believe that human-like intelligence can ultimately be achieved within a digital computer program (most of them are in my view overpessimistic about how long it's going to take us to figure out exactly how to write such a program, but that's another story). But some mavericks, most notably Roger Penrose, have argued otherwise (see his books The Emperor's New Mind and Shadows of the Mind, for example). Penrose has argued specifically that the crux of human intelligence is some sort of mental manipulation of uncomputable entities.

And Penrose has also gone further: he's argued that some future theory of physics is going to reveal that the dynamics of the physical world is also based on the interaction of uncomputable entities. So that mind is an uncomputable consequence of uncomputable physical reality.

This argument always disturbed me, also. There always seemed something fundamentally wrong to me about the notion of "uncomputable physics." Because, science is always, in the end, about finite sets of finite-precision data. So, how could these mysterious uncomputable entities ever really be necessary to explain this finite data?

Obviously, it seemed tome, they could never be necessary. Any finite dataset has a finite explanation. But the question then becomes whether in some cases invoking uncomputable entities is the best way to explain some finite dataset. Can the best way of explaining some set of, say, 10 or 1000 or 1000000 numbers be "This uncomputable process, whose details you can never write down or communicate in ordinary language in a finite amount of time, generated these numbers."

This really doesn't make sense to me. It seems intuitively wrong -- more clearly and obviously so than the notion of the "existence" of uncomputable numbers and other uncomputable entities in some abstract mathematical sense.

So, my goal in this post is to give a careful explanation of why this wrong. The argument I'm going to give here could be fully formalized as mathematics, but, I don't have the time for that right now, so I'll just give it semi-verbally/semi-mathematically, but I'll try to choose my words carefully.

As often happens, the matter turned out to be a little subtler than I initially thought it would be. To argue that uncomputables are useless for science, one needs some specific formal model of what science itself is. And this is of course a contentious issue. However, if one does adopt the formalization of science that I suggest, then the scientific uselessness of uncomputables falls out fairly straightforwardly. (And I note that this was certainly not my motivation for conceiving the formal model of science I'll suggest; I cooked it up a while ago for quite other reasons.)

Maybe someone else could come up with a different formal model of science that gives a useful role to uncomputable entities ... though one could then start a meta-level analysis of the usefulness of this kind of formal model of science! But I'll defer that till next year ;-)

Even though it's not wholly rigorous math, this is a pretty mathematical blog post that will make for slow reading. But if you have suitable background and are willing to slog through it, I think you'll find it an interesting train of thought.

NOTE: the motivation to write up these ideas (which have been bouncing around in my head for ages) emerged during email discussions on the AGI list with a large group, most critically Abram Demski, Eric Baum and Mark Waser.

A Simple Formalization of the Scientific Process

I'll start by giving a simplified formalization of the process of science.

This formalization is related to the philosophy of science I outlined in the essay http://www.goertzel.org/dynapsyc/2004/PhilosophyOfScience_v2.htm (included in The Hidden Pattern) and more recently extended in the blog post http://multiverseaccordingtoben.blogspot.com/2008/10/reflections-on-religulous-and.html. But those prior writing consider many aspects not discussed here.

Let's consider a community of agents that use some language L to communicate. By a language, what I mean here is simply a set of finite symbol-sequences ("expressions"), utilizing a finite set of symbols.

Assume that a dataset (i.e., a finite set of finite-precision observations) can be expressed as a set of pairs of expressions in the language L. So a dataset D can be viewed as a set of pairs


((d11, d12), (d21,d22) ,..., (dn1,dn2))

or else as a pair D=(D1,D2) where

D1=(d11,...,dn1)
D2=(d12,...,dn2)

Then, define an explanation of a dataset D as a set E_D of expressions in L, so that if one agent A1 communicates E_D to another agent A2 that has seen D1 but not D2, nevertheless A2 is able to reproduce D2.

(One can look at precise explanations versus imprecise ones, where an imprecise explanation means that A2 is able to reproduce D2 only approximately, but this doesn't affect the argument significantly, so I'll leave this complication out from here on.)

If D2 is large, then for E_D to be an interesting explanation, it should be more compact than D2.

Note that I am not requiring E_D to generate D2 from D1 on its own. I am requiring that A2 be able to generate D2 based on E_D and D1. Since A2 is an arbitrary member of the community of agents, the validity of an explanation, as I'm defining it here, is relative to the assumed community of agents.

Note also that, although expressions in L are always finitely describable, that doesn't mean that the agents A1, A2, etc. are. According to the framework I've set up here, these agents could be infinite, uncomputable, and so forth. I'm not assuming anything special about the agents, but I am considering them in the special context of finite communications about finite observations.

The above is my formalization of the scientific process, in a general and abstract sense. According to this formalization, science is about communities of agents linguistically transmitting to each other knowledge about how to predict some commonly-perceived data, given some other commonly-perceived data.

The (Dubious) Scientific Value of the Uncomputable

Next, getting closer to the theme of this post, I turn to consider the question of what use it might be for A2 to employ some uncomputable entity U in the process of using E_D to generate D2 from D1. My contention is that, under some reasonable assumptions, there is no value to A2 in using uncomputable entities in this context.

D1 and E_D are sets of L-expressions, and so is D2. So what A2 is faced with, is a problem of mapping one set of L-expressions into another.

Suppose that A2 uses some process P to carry out this mapping. Then, if we represent each set of L-expressions as a bit string (which may be done in a variety of different, straightforward ways), P is then a mapping from bit strings into bit strings. To keep things simple we can assume some maximum size cap on the size of the bit strings involved (corresponding for instance to the maximum size expression-set that can be uttered by any agent during a trillion years).

The question then becomes whether it is somehow useful for A2 to use some uncomputable entity U to compute P, rather than using some sort of set of discrete operations comparable to a computer program.

One way to address this question is to introduce a notion of simplicity. The question then becomes whether it is simpler for A2 to use U to compute P, rather than using some computer program.

And this, then, boils down to one's choice of simplicity measure.

Consider the situation where A2 wants to tell A3 how to use U to compute P. In this case, A2 must represent U somehow in the language L.

In the simplest case, A2 may represent U directly in the language, using a single expression (which may then be included in other expressions). There will then be certain rules governing the use of U in the language, such that A2 can successfully, reliably communicate "use of U to compute P" to A3 only if these rules are followed. Call this rule-set R_U. Let us assume that R_U is a finite set of expressions, and may also be expressed in the language L.

Then, the key question is whether we can have

complexity(U) < complexity(R_U)

That is, can U be less complex than the set of rules prescribing the use of its symbol S_U within the community of agents?

If we say NO, then it follows there is no use for A2 to use U internally to produce D2, in the sense that it would be simpler for A2 to just use R_U internally.

On the other hand, if we say YES, then according to the given complexity measure, it may be easier for A2 to internally make use of U, rather than to use R_U or something else finite.

So, if we choose to define complexity in terms of complexity of expression in the community's language L, then we conclude that uncomputable entities are useless for science. Because, we can always replace any uncomputable entity U with a set of rules for manipulating the symbol S_U corresponding to it.

If you don't like this complexity measure, you're of course free to propose another one, and argue why it's the right one to use to understand science. In a previous blog post I've presented some of the intuitions underlying my assumption of this "communication prior" as a complexity measure underlying scientific reasoning.

The above discussion assumes that U is denoted in L by a single symbolic L-expression S_U, but the same basic argument holds if the expression of U in L is more complex.

What does all this mean about calculus, for example ... and the other lovely uses of uncomputable math to explain science data?

The question comes down to whether, for instance, we have

complexity(real number line R) <>

If NO, then it means the mind is better off using the axioms for R than using R directly. And, I suggest, that is what we actually do when using R in calculus. We don't use R as an "actual entity" in any strong sense, we use R as an abstract set of axioms.

What would YES mean? It would mean that somehow we, as uncomputable beings, used R as an internal source of intuition about continuity ... not thus deriving any conclusions beyond the ones obtainable using the axioms about R, but deriving conclusions in a way that we found subjectively simpler.

A Postcript about AI

And, as an aside, what does all this mean about AI? It doesn't really tell you anything definitive about whether humanlike mind can be achieved computationally. But what it does tell you is that, if

• humanlike mind can be studied using the communicational tools of science (that is, using finite sets of finite-precision observations, and languages defined as finite strings on finite alphabets)
• one accepts the communication prior (length of linguistic expression as a measure of complexity)

then IF mind is fundamentally noncomputational, science is no use for studying it. Because science, as formalized here, can never distinguish between use of U and use of S_U. According to science, there will always be some computational explanation of any set of data, though whether this is the simplest explanation depends on one's choice of complexity measure.

Cosmic, overblown Grand Unified Theory of Development

In the 80's I spent a lot of time in the "Q" section of various libraries, which hosted some AI books, and a lot of funky books on "General Systems Theory" and related forms of interdisciplinary scientifico-philosophical wackiness.

GST is way out of fashion in the US, supplanted by Santa Fe Institute style "complexity theory" (which takes the same basic ideas but fleshes them out differently using modern computer tech), but I still have a soft spot in my heart for it....

Anyway, today when I was cleaning out odd spots of the house looking for a lost item (which I failed to find and really need, goddamnit!!) I found some scraps of paper that I scribbled on a couple years back while on some airline flight or another, sketching out the elements of a general-systems-theory type Grand Unified Theory of Development ... an overall theory of the stages of development that complex systems go through as they travel from infancy to maturity.

I'm not going to type in the whole thing here right now, but I made a table depicting part of it, so as to record the essence of the idea in some nicer, more permanent form than the fading dirty pieces of notebook paper....

The table shows the four key stages any complex system goes through, described in general terms, and then explained in a little more detail in the context of two examples: the human (or humanlike) mind as it develops from infancy to maturity, and the maturity of life from proto-life up into its modern form.

I couldn't get the table to embed nicely in this blog interface, so it's here as a PDF:

Table
of the Universal Stages of Development
as Manifested in Mind and Life

This was in fact the train of thought that led to two papers Stephan Bugaj and I wrote over the last couple years, on the stages of cognitive development of uncertain-inference based AI systems, and the stages of ethical development of such AI systems. While not presented as such in those papers, the stages given there are really specialized manifestations of the more general stages outlined in the above table.

Stephan and I are (slowly) brewing a book on hyperset models of mind and reality, which will include some further-elaborated, rigorously-mathematized version of this general theory of development...

Long live General Systems thinking ;-)

Parable of the Researcher and the Tribesman

I run an email discussion list on Artificial General Intelligence, which is often interesting, but lately the discussions there have been more frustrating than fascinating, unfortunately.

One recent email thread has involved an individual repeatedly claiming that I have not presented any argument as to why my designs for AGI could possibly work.

When I point to my published or online works, which do present such arguments, this individual simply says that if my ideas make any sense, I should be able to summarize my arguments nontechnically in a few paragraphs in an email.

Foolishly, I managed to get sufficiently annoyed at this email thread that I posted a somewhat condescending and silly parable to the email list, which I thought I'd record here, just for the heck of it....

What I said was:

In dialogues like this, I feel somewhat like a medical researcher talking to a member of a primitive tribe, trying to explain why he thinks he has a good lead on a potential drug to cure a disease. Imagine a dialogue like this:

• RESEARCHER: I'm fairly sure that I'll be able to create a drug curing your son's disease within a decade or so
• TRIBESMAN: Why do you believe that? Have you cured anyone with the drug?
• RESEARCHER: No, in fact I haven't even created the drug yet
• TRIBESMAN: Well, do you know exactly how to make the drug?
• RESEARCHER: No, not exactly. In fact there is bound to be some inventive research involved in making the drug.
• TRIBESMAN: Well then how the hell can you be so confident it's possible?
• RESEARCHER: Well I've found a compound that blocks the production of the protein I know to be responsible for causing the disease. This compound has some minor toxic effects in rats, but it's similar in relevant respects to other compounds that have shown toxic effects in rats, and then been minorly modified to yield variant compounds with the same curative impacts without toxic effects
• TRIBESMAN: So you're saying it's cured the same disease in rats?
• RESEARCHER: Yes, although it also makes the rats sick ... but if it didn't make them sick, it would cure them. And I'm pretty sure I know how to change it so as to make it not make the rats sick. And then it will cure them.
• TRIBESMAN: But my son is not a rat. Are you calling my son a rat? You don't seem to understand what a great guy my son is. All the women love him. His winky is twice as long as yours. What does curing a rat have to do with curing my son? And it doesn't even cure the rat. It makes him sick. You just want to make my son sick.
• RESEARCHER: Look, you don't understand. If you look at all the compounds in that class, you'll see there are all sorts of ways to modify them to avoid these toxic effects.
• TRIBESMAN: So you're saying I should believe you because you're a big important scientist. But your drug hasn't actually cured anyone. I don't believe it'll possibly work. People come by here all the time trying to sell me drugs and they never work. Those diet pill were supposed to make my wife 100 pounds thinner, but she still looks like a boat.
• RESEARCHER: I'm not responsible for the quacks who sold you diet pills
• TRIBESMAN: They had white lab coats just like yours
• RESEARCHER: Look, read my research papers. Then let's discuss it.
• TRIBESMAN: I can't read that gobbledygook. Do all the other researchers agree with you?
• RESEARCHER: Some of them do, some of them don't. But almost all of them who have read my papers carefully think I at least have a serious chance of turning my protein blocker into a cure. Even if they don't think it's the best possible approach.
• TRIBESMAN: So all the experts don't even agree, and you expect me to take you seriously?
• RESEARCHER: Whatever. I'll talk to you again when I actually have the cure. Have a nice few years.
• TRIBESMAN: We won't need your cure by then, Mr. Scientist. We're curing him with leeches already.

That just about sums it up....

The point is, the researchers's confidence comes from his intuitive understanding of a body of knowledge that the tribesman cannot appreciate due to lack of education.

The tribesman says "you haven't cured anyone, therefore you know nothing about the drug" ... but the researcher has a theoretical framework that lets him understand something about the drug's activity even before trying it on people.

Similarly, some of us working on AGI have a theoretical framework that lets us understand something about our AGI systems even before they're complete ... this is what guides our work building the systems. But conveying our arguments to folks without this theoretical framework is, unfortunately, close to impossible.... If I were to write some sort of popular treatment of my AGI work, the first 75% of it would have to consist of a generic explanation of background ideas (which is part of the reason I don't take the time to write such a thing ... it seems like an awful lot of work!!).

Obvious stuff, of course. I'm metaphorically kicking myself for burning half an hour in this sort of absurd email argument tonight ... gotta be more rigorous about conserving my time and attention, there's a lot of work to be done!!!

Reflections on "Religulous" ... and introducing the Communication Prior

I saw the documentary Religulous w/ my kids last night (well, the two who still live at home) ... it's a sort of goofball documentary involving comedian Bill Maher interviewing people with absurd religious beliefs (mostly focusing on Christians, Jews and Muslims, with a few other oddities like a Scientologist street preacher and an Amsterdam cannabis-worshipper) ...

This blog post records some of my random reactions to the movie, and then at the end gets a little deeper and presents a new theoretical idea that popped into my head while thinking about the difficulty of making a really sound intellectual refutation of common religious beliefs.

The new theoretical idea is called the Communication Prior ... and the crux is the notion that in a social group, the prior probability of a theory may be defined in terms of the ease with which one group member can rapidly and accurately communicate the theory to another. My suggestion is that the Communication Prior can serve as the basis for a pragmatic everyday interpretation of Occam's Razor (as opposed to the Solomonoff-Levin Prior, which is a formal-computer-science interpretation). This is important IMHO because science ultimately boils down to pragmatic everyday social phenomena not formal mathematical phenomena.

Random Reactions to Religulous

First a bit about Religulous, which spurred the train of thought reported here....

Some of the interviews in the movie were really funny -- for instance a fat Puerto Rican preacher named Jesus who claims to literally be the Second Coming of Christ, and to have abolished sin and hell ...

and as a whole the interviews certainly made Maher's point that all modern religions are based on beliefs that seem bizarre and twisted in the light of the modern scientific world-view ... the talking snake in the Garden of Eden ... Judgment Day when God comes to Earth and sorts the goodies from the baddies ... the notion that rapture will come only when the Muslims have finally killed all the Jews ... etc. etc. etc. etc. etc. ...

Some interesting historical tidbits were presented as well, e.g. the Egyptian figure Horus, who well predated Christ and whose life-story bears remarkable similarities to the Biblical tale of Jesus....

I've never been a huge fan of stand-up comedians; and among comedians Maher doesn't really match my taste that well ... he's not outrageous or absurd enough ... so I got a bit weary of his commentary throughout the film, but I felt the interviews and interspersed film and news snippets were well-done and made his point really well.

Of course, it's a damn easy point to make, which was part of his point: Of course all religions ancient and modern have been based on bizarre, wacky, impossible-for-any-sane-person-to-believe, fictional-sounding ideas...

One point that came up over and over again in his dialogues with religious folks was his difference with them over the basic importance (or lack thereof) of faith. "Why," he kept asking, "is faith a GOOD thing? Why is it a good thing to believe stuff that has no evidence in favor of it? Why is it a good thing to believe stuff that makes no sense and contradicts observation and apparent reality?"

The answer the religious folks invariably give him is something like "Faith is a good thing because it saved my life."

Dialogue like: "I used to be a Satan worshipper and wasted decades of my life on sex and drugs ... Getting saved by Jesus saved my life blahblaa..."


Religion and Politics: Egads!


Maher's interview with a religious fundamentalist US Senator is a bit disturbing. Indeed, to have folks who believe Judgment Day is nigh, in charge of running the most powerful country in the world, is, uh, scary....

And note that our outgoing President, W Bush, repeatedly invokes his religious beliefs in justifying his policies. He explicitly states that his faith in God is the cornerstone of his policies. Scary, scary, scary. I don't want to live in a society that is regulated based on someone's faith in a supernatural being ... based on someone's faith in the literal or metaphorical truth of some book a bunch of whacked-out, hallucinating Middle-Easterners wrote 2000 years ago....

As Maher points out, this is a completely senseless and insane basis for a modern society to base itself on....


Maher's Core Argument

I don't expect Maher's movie to un-convert a substantial number of religious folks...

Their natural reaction will be: "OK, but you just interviewed a bunch of kooks and then strung their kookiest quotes together."

Which is pretty much what he did ... and in a way that may well be compelling as a tool for helping atheists feel more comfortable publicly voicing their beliefs (which I imagine was much of his purpose) ...

And it has to be noted that a deep, serious, thorough treatment of the topic of religion and irrationality would probably never get into movie theaters.

Modern culture, especially US culture but increasingly world culture as well, has little time for deep rational argumentation. Al Gore made this book quite nicely in his book The Assault on Reason ... which however not that many people read (the book contained too much rational argumentation...).

So it's hard to fault Maher's film for staying close to the surface and presenting a shallow argument against religion ... this is the kind of argument that our culture is presently willing to accept most easily ... and if atheists restricted themselves to careful, thorough, reflective rational arguments, the result would be that even fewer people would listen to them than is now the case....

Maher's argument is basically: All religions have absurd, apparently-delusional, anti-scientific beliefs at their core ... and these absurd beliefs are directly tied to a lot of bad things in the world ... Holy Wars and so forth ....

He also, correctly, traces the bizarre beliefs at the heart of religions to altered brain-states on the part of religious prophets.

As he notes, if someone today rambled around telling everyone they'd been talking to a burning bush up on a hill, they'd likely get locked into a mental institution and force-fed antipsychotics. Yet, when this sort of experience is presented as part of the history of religion, no one seems to worry too much -- it's no longer an insane delusion, it's a proper foundation for the government of the world ;-p

What Percentage of the Population Has a World View Capable of Sensibly Confronting the Singularity?

One thing that struck me repeatedly when listening to Maher's interviews was:

Wow, given all the really HARD issues the human races during this period of rapidly-approaching Singularity ... it's pathetic that we're still absorbed with these ridiculous debates about talking snakes and Judgment Day and praying to supreme beings ... egads!!!

While a digression from this blog post, this is something I think about a lot, in the context of trying to figure out the most ethical and success-probable approach to creating superhuman AI....

On the one hand, due to various aspects of human psychology, I don't trust elitism much: the idea of a small group of folks (however gifted and thoughtful) creating a superhuman AI and then transforming the world, without broader feedback and dialogue, is a bit scary....

On the other hand, I've got to suspect that folks who believe in supreme beings, Judgment Day, jihad, reincarnation and so forth are not really likely to have much useful contribution to the actual hard issues confronting us as Singularity approaches....

Of course, one can envision a lot of ways of avoiding the difficulties alluded to in the prior two paragraphs ... but also a lot of ways of not avoiding them....

One hope is that Maher's movie and further media discourse legitimizing atheism will at least somewhat improve the intellectual level of broad public conversation ... so that, maybe, in a decade or so it won't be political suicide for a US Senatorial candidate to admit they're not religious or superstitious, for example...

On the other hand, it may well eventuate that this process of de-superstitionizing the world will be damn slow compared to the advent of technology ...

But, that's a topic for another lengthy blog post, some other weekend....


The Issues Posed by the "Problem of Induction" and the Philosophy of Science for the Argument Against Religion

Now I'll start creeping, ever so slowly, toward the more original intellectual content of this post, by asking: What might a more deeply reasoned, reflective argument against religion look like?

This topic is actually fairly subtle, because it gets at deep issues in the philosophy of science ... such as I reviewed in an essay a few years ago (included in my 2006 book The Hidden Pattern)...

Although Maher talks a lot about scientific evidence ... and correctly points out that there is no scientific evidence for the various kooky-sounding claims at the core of modern religions ... he doesn't seem to have thought much about the nature of scientific evidence itself. (Which is no surprise as he's a professional comedian and actor ... but of course, he's now a self-styled commentator on politics, science and religion, so....)

Evidence, in the sense of raw data, is not disputed that often among scientists -- and even religious folks don't dispute raw data collected by scientists that often. Statements like "this laboratory instrument, at this point in time, recorded this number on its dial" are not oft disputed. Sometimes argumentation may be made that not enough data were recorded to evaluate an empirical statement like the above (say, the temperature in the room, or the mind-state of the lab assistant, were not recorded): but this still isn't really an argument that the data are wrong, more an argument that the data are too incomplete to draw useful conclusions from them.

(The only area of research I know where raw data is routinely disputed is psi ... which I already addressed in a prior blog post.)

But the step from raw items of evidence to theory is a big one -- a bigger one than Maher or most naively-pro-science advocates care to admit.

This of course relates to the uncomfortable fact that the Humean problem of induction was never solved.

As Maher points out repeatedly in his film, we just don't really know anything for sure ... and it appears that by the basic logic of the universe and the nature of knowledge itself, we never can.

What he doesn't point out (because it's not that kind of movie) is that without making some kind of background assumptions (going beyond the raw evidence collected), we also can't really make probability estimates, or probabilistic predictions about the outcomes of experiments or situations.

Given a set of observations, can we predict the next observations we'll see? Even probabilistically? As Hume pointed out, we can do so only by making some background assumptions.

For instance, we can adopt the Occam's Razor heuristic and assume that there will be some simple pattern binding the past observations to the future ones.... But that begs the question: what is the measure of simplicity?

Hume says, in essence, that the relevant measure of simplicity is human nature.

But this conclusion may, initially, seem a bit disturbing in the context of the religion vs. science dichotomy.

Because, human nature in in many ways, not to put it too tactlessly, more than a bit fucked-up.

Maher doesn't review the evidence in this regard, but he does allude to it, e.g interviewing the discoverer of the "God gene" ... the point is: it seems to be the case that religious experience and religious delusions are deeply tied to intrinsic properties of the human brain.

What this suggests is that the reason religion is so appealing to people is precisely that it is assigned a high prior probability by their Humean "human nature" ... that our brain structure, which evolved in superstitious pre-civilized societies, biases us towards selecting theories that not only explain our everyday empirical observations, but also involve talking animals, voices speaking from the sky, tribalism, physical rewards or punishments for moral transgressions, and so forth...

So when Maher says that "it's time for us to grow up" and let go of these ancient religious superstitions and just be rational and scientific ... two big problems initially appear to arise, based on cursory consideration of the philosophy of science:

• There is no such thing as "just being rational" ... applying rationality to real observations always involves making some background assumptions
• The ancient religious superstitions are closely related to patterns wired into our brains by evolution ... which are naturally taken by us as background assumptions...
  

So when he asks folks to drop their religious beliefs, is Maher really asking folks to self-modify their brains so as not to apply prior distributions supplied by evolution (which has adapted our cognitive patterns to superstitious, tribal society), and to instead apply prior distributions supplied by the scientific and rationalist tradition...?

If so, that would seem a really tough battle to fight. If this were the case, then essentially, the transcendence of religious superstitions would require a kind of cognitive transhumanism.

Fortunately, though I don't think the situation is quite that bad. Cognitive transhumanism (which I define as the attempt to go beyond innately-human patterns of thinking) certainly can be a huge help in the transcendence of superstitions, but it's not strictly necessary.

It appears to me that it's enough "just" to get people to think more clearly about the relationship between their theories and ideas, their community, and their community's collective observations. If people understand this relationship clearly, then it's not actually necessary for them to transcend their various superstition-oriented human biases in order for them to go beyond naive religious ideas.

To elaborate on this point further I'll need to get technical for a moment and introduce a bit of Bayesian statistics and algorithmic information theory...

The Communication Prior

I'll now shift from philosophical babbling to basic math for a few paragraphs.

Recall the basics of Bayes Theorem... . Setting T for "theory" and E for "evidence", it says:

P(T|E) = P(T) P(E|T)/P(E)
... i.e., it says that a person's subjective probability that a theory T is true given that they receive evidence E, should be equal to their prior probability that T is true times the probability that they would receive evidence E if hypothesis T were true, divided by the probability of E (and the latter is usually found by summing over the weighted conditional probabilities given all potential theories).

It is critical to note that, according to Bayes rule, one's conclusion about the probability of theory T given evidence E depends upon one's prior assignment of probabilities.

Now, a real mind with computational limitations cannot always apply Bayes rule accurately ... so the best we can do is approximate.

(Some cognitive theorists, such as Pei Wang, argue that a real mind shouldn't even try to approximate Bayes rule, but should utilize a different logic specially appropriate for cognitive systems with severe resource limitations ... but I don't agree with this and for the purpose of this blog post will assume it's not the case.)

But even if a mind has enough computational resources to apply Bayes rule correctly, there remains the problem of how to arrive at the prior assignment of probabilities?

The most commonsensical way is to use Occam's Razor, the maxim stating that simpler hypotheses should be considered a priori more probable. But this also leads to some subtleties....

The Occam maxim has been given mathematical form in the Solomonoff-Levin universal prior, which says very roughly that the probability of a hypothesis is higher if the computer-programs for computing that hypothesis are shorter (yes, there's more to it, so look it up if you're curious).

Slightly more rigorously, Wikipedia notes that:

The universal prior probability of any prefix p of a computable sequence x is the sum of the probabilities of all programs (for a universal computer) that compute something starting with p. Given some p and any computable but unknown probability distribution from which x is sampled, the universal prior and Bayes' theorem can be used to predict the yet unseen parts of x in optimal fashion.

Note in the above quote that the probability of a program may be estimated as the probability that the program is found by randomly selecting bits in the program-defining section of the memory of a computer.

Anyway: That's very nice for mathematicians, but it doesn't help us much in everyday life ... because even if we wanted to apply this kind of formalization in everyday life (say, to decide an issue like evolution vs. creationism), the mapping of real-world situations into mathematical formalisms is itself highly theory-laden....

So what we really need is not just a mathematical formalization of a universal prior, but a commonsensical formalization of a prior that is helpful for everyday human situations (even if not truly universal).

One suggestion I have is to use Solomonoff's core idea here, but interpret it a bit differently, in terms of everyday human communicational operations rather than mathematical, abstracted machine operations.

Paraphrasing the above quoted text, I propose that

The communicational prior probability of any prefix p of a computable sequence x, relative to a social group G and a body of evidence E, is the sum of the communicational probabilities (calculated relative to G and E) of all programs that compute something starting with p.

But how then to compute the communicational probability of a program relative to a social group G and body of evidence E?

As the name indicates, this is defined, not in terms of bit-flipping, but in terms of communication within the group.

I define the communicational probability of a program p, as being proportional to the average amount of time it would take a randomly chosen member A of group G to communicate p to another randomly chosen member B of group G, with sufficient accuracy that G can then evaluate the outputs of p on randomly selected inputs drawn from E.

(The assumption is that A already knows how to evaluate the program on inputs drawn from E.)

One can also bake a certain error rate into this definition, so that G has to be able to correctly evaluate the outputs of p only on a certain percentage of inputs drawn from E.

This defines what I suggest to call the Communication Prior.

A variant would be the communication-and-testing probability of a program p, definable as being proportional to the average, for randomly chosen members A and B in the social group such that A already knows how to evaluate p on inputs in E, of

• the amount of time it would take A to communicate p to B, with sufficient accuracy that B can then evaluate the outputs of p on randomly selected inputs drawn from E
• the amount of time it actually takes B to evaluate p on a randomly selected element of E

(One can of course weight the two terms in this average, if one wants to.)

Taking a bit of terminological liberty, I will also group this communication-testing variant as being under the umbrella of the "Communication Prior."

Pragmatically, what does this mean about theories?

Roughly speaking, it means that the a priori probability of a theory (i.e. the "bias toward" a theory) has to do with ease of effectively communicating that theory within a social group ... and (in the communication-testing variant), the ease of effectively communicating how to efficiently apply the theory.

Of course, the a priori probability theory doesn't tell you how good a theory is. Communicating a theory may be very simple, but so what ... unless the theory explains something. But the "explanation" part is taken care of in Bayes Rule, in the P(E | T) / P(E) fraction. If the observed evidence is not surprisingly likely given the assumption of the theory, then this fraction will be small.

The Communication Prior is similar in spirit to the Solomonoff-Levin Universal Prior ... but it's not about formal, mathematical, theoretical systems, it's about real-world social systems, such as human communities of scientists. In terms of philosophy of science, this is sort-of a big deal, as it bridges the gap between formalist and social-psychology-based theories of science.

What's the Take-Away from All That Techno-babble?

So, roughly speaking, the nontechnical take-away from the above technical excursion should be the following suggestion:

A theory should be considered good within a social group, to the extent that it explains the evidence better than it would explain a bunch of randomly selected evidence -- and it's reasonably rapid to effectively communicate, to others in the group, information about how to efficiently apply the theory to explain the available evidence.

This may seem simple or almost obvious, but it doesn't seem to have been said before, in quite so crisp of a way.

(In my prior essay on philosophy of science, I left off without articulating any sort of specific simplicity measure: the Communication Prior fills in that gap, thus bringing the ideas in that essay closer to practical applicability.)

Consider for instance the evolution vs. creationism argument. For my new suggestion to favor evolution over creationism, what would have to be true?

Whether the simple essential core of creationism or evolution is easier to communicate within a human social group, really depends on the particular social group.

However, the simple essential core of creationism does an extremely bad job of explaining why the observed body of evidence (e.g. the fossil record) is more likely than a lot of other possible bodies of evidence.

To make a version of creationism that would explain why the observed body of evidence is particularly likely, one would need to add a heck of a lot of special-pleading-type explanations onto the essential core of creationism. This is because creationism does not effectively compress or compactify the body of observed data.

So, to get a version of creationism that is equally explanatory of the particulars of the evidence as evolution, one needs to make a version of creationism that takes a long time to communicate.

Conclusion: creationism is worse than evolution.

(OK, we don't really need to go through so much complexity to get to such an obvious conclusion! But I'm just using that example to make a more general point, obviously.)

Why Is Religion a Bad Idea?

Getting back to the initial theme of this overlong, overdiverse blog post, then: why is religion a bad idea?

Because we should judge our theories using Bayes rule with a communication prior ... or in other words, by asking that they explain the particulars of observed reality in a relatively rapidly communicable way.

There is a balance between success-at-detailed-explanation and rapid-communicability, and the exact way to strike this balance is going to be subtle and in some cases subjective. But, in the case of religious beliefs, the verdict is quite clear: the religious world view, compared to the scientific world view, fails miserably at explaining the particulars of observed reality in a relatively rapidly communicable way.

The key point here is that, even if people want to stick with their evolutionary-legacy-based inductive biases (which make them intuitively favor superstitious explanations), the failure of religious theories to explain the particulars of observed reality is now so drastic and so obvious, that anyone who really carefully considers the evidence should reject these religious theories anyway.

Maher's film points out sensationalistically silly aspects of religious belief systems. But these aren't really the right anti-religion argument to use, in terms of philosophy of science and the theory of rationality. After all, are the Big Bang and Big Crunch and the evolution of humans from apes really any less everyday-ishly wacky than Judgment Day and the talking snake in the Garden of Eden?

The right argument to use is that, if one assumes Bayes rule plus a Communication Prior (or any other sensible, everyday-reality-based prior), then religious theories fail miserably.

Of course, almost no one on the planet can understand the previous sentence, though ... which is why his approach of dramatically emphasizing the most absurdly wacky religious beliefs and believers is probably a way more effective PR strategy!


The Emotion Prior

Finally, another suggestion I have regarding the popularity of religious beliefs has to do with something my ex-wife said to me once, shortly after her religious conversion to Buddhism, a topic about which we had numerous arguments (some heated, some more rational and interesting, none usefully conclusive nor convincing to either of us). What she said was: "I believe what I need to believe in order to survive."

She didn't just mean "to survive physically" of course ... that was never at issue (except insofar as emotional issues could have threatened her physical survival) ... what she meant was "to survive emotionally" ... to emotionally flourish ...

My (rather uncontroversial) suggestion is that in many cases religious people -- and others -- have a strong bias toward theories that they enjoy believing.

Or in other words: "If believing it feels good, it can't be wrong!"

This is probably the main issue in preaching atheism: one is asking people to

• adopt (some approximant of) Bayes rule with a Communication Prior (or similar)
• actually carefully look at the evidence that would be used in Bayes rule

... rather than to, on the other hand,

• avoid looking at evidence that might disconfirm one's theory
• utilize an Emotion Prior when evaluating various theories that might explain the evidence

The question is then whether, in each individual case,

• the Emotion Prior outweights the Communication Prior (or similar)
• the sociopsychological pressure to look at evidence outweighs the sociopsychological pressure to ignore it

Ignoring evidence gets harder and harder as the Internet broadcasts data to everyone, all the time....

To study these choices in an interesting way, one would need to model the internals of the believer's mind more subtly that has been done in this post so far....

But anyway ... the evidence of the clock in front of me is that I have spent too much time amusing myself by writing this blog post, and now have more useful things to do ... so, till next time!

P.S. Thanks to my wife Izabela for discussions leading to the introduction of the communication-testing variant of the Communication Prior, after the more basic version had already been formulated....

Another Transhumanist Nightmare

Some anonymous freak wrote this story, a piece of transhumanist/absurdist fantasy which includes me in a minor role ... it's childish, but I have to say, mildly amusing...

Another
Transhumanist
Nightmare

The End of the American Era!! (Not)

I'm not generally a very political person ... my thinking and my life-decisions are pretty strongly focused on the "big picture": superhuman AI, the Singularity, transhumanism and all that.

I was deeply into politics as a teen (largely because my parents raised me to be), but as I realized that utopian political dreams were likely to founder on the intrinsic biological perversity of human nature, I drifted away from the political sphere and started thinking more about how to improve or transcend human nature itself....

However, every now and then some piece of political stupidity gets on my nerves sufficiently that I wind up burning time thinking about it.

One of these cases has occurred recently: I've become annoyed by a large number of people proclaiming that "the American era is finally ending." No empire rules forever, and blah blah blah.

I've been hearing this sort of talk for a while, but all the more intensely given the recent week's American banking crisis.

So I decided to write a blog post to get my thoughts on the topic out of my head!

I've never been noted for my patriotism: I really don't care, at a fundamental level, about nations or other related manifestations of contemporary human society. I'll be happy to see them all go away once human nature is fundamentally reformed via radical technological advances.

I've also spent enough time living and traveling outside the US, to get some feel for the strengths and weaknesses of the good/bad old US of A.

My considered opinion of the "end of the American era" meme is that it's pretty much bullshit.

I also seem to look at the current financial crisis a little differently than most others (big surprise there, huh?).

The issues that investment banks, insurance companies and related institutions have recently experienced have been widely attributed to greed, poor government regulation, and so forth. These attributions are surely correct -- but any real event has multiple causes ("cause" being essentially a creature of subjective theory rather than physical reality anyway). And one cause is not being commented on enough, which is the phenomenal practical creativity involved in all the recondite financial instruments (credit default swaps, mortgage strips and the like) underlying the recent woes.

There is some really cool math underlying these financial devices, and this math was largely invented and pragmaticized by American entrepreneurial thinkers. American quants have developed many new fields of financial math, and brought these into the real world, thus moving the global economy to a whole new level of complexity and efficiency.

Innovation always carries risks ... and we've seen that in the markets over the last weeks and months. But let's not forget how amazing the innovations are, and what tremendous positive potential they have.

I agree that exotic derivatives should be regulated more carefully. On the other hand, I also agree with their advocates that they add significant efficiency to the financial markets, and hence are a major asset to the world economy.

Of course, one can theoretically envision socioeconomic systems in which efficiency would be achieved by other, less perverted and convoluted means. But, as history shows, theoretically-envisioned socioeconomic systems are difficult to translate into realities, because of the subtleties of human psychology and culture.

And it's precisely these "subtleties of psychology and culture" that led America to invent quantitative finance ... and so many other amazing technological and scientific developments ... which is exactly why I tend to doubt the "American era" is at its end.

My contention, and it's not a terribly original one (but I may have a somewhat original slant), is that compared to other countries on the planet right now, the USA has a combination of cultural psychology and socioeconomic institutions that is uniquely well-suited to fostering practical creativity.

Note the compound of terms: "practical" and "creativity."

I don't think the US has any kind of monopoly on creativity itself. There are brilliant, creative minds everywhere. Some cultures foster creativity more than others ... and the US is pretty good at this ... but I'm not sure it's uniquely good.

And I don't think the US has any kind of monopoly on practicality, either. Although historically this has been a US characteristic, there are surely other nations that are currently more down-to-earth and practical than the US (as a generalization across various aspects of life).

However, the US seems to be uniquely good at taking creative new ideas and finding the first ways to give them practical implementations -- an art that requires a great deal of creativity in itself, of course.

What is it about the US that fosters practical creativity? It's no one thing. It's a synergetic combination of culture and institutions. The institutions help keep the culture alive, and the culture helps keep the institutions alive. Practical creativity is something that pervades many aspects of US life -- government, research, education and industry, for example. Precisely because of its pervasive and systemic nature, the memeplex that constitutes practical creativity in the US is difficult for other nations to copy, even if they have a genuine desire to.

To see what I mean more concretely, think about three examples: the Internet, the Human Genome Project, and the personal computer. How did these come about?

The history of the PC embodies many classic stories of American entrepreneurism, including the creation of Apple and Microsoft by young nerdy entrepreneurs out of nowhere. But it also tells you something about the flexibility of large US corporations relative to similar institutions elsewhere: it was IBM striking a deal with Bill Gates, some young nerd from nowhere with no real business experience, that set the PC industry on its modern path. Not to mention the freewheeling US corporate research lab culture of the time (Xerox PARC and all that). And the government research funding establishment played its role behind the scenes, for instance in funding the creation of mainframes that Bill Gates played with (often breaking the rules to do so) in high school and college, before starting Microsoft....

The Internet began as a project of ARPA (now DARPA), a US government research funding agency that has its strengths and weaknesses, but is notable for its chaotic approach to funding. DARPA program managers cycle in and out every 4 years so that no individual has too much power over resource allocation decisions. There are certainly "old boy networks" involved, and I've personally been fairly unhappy with DARPA's funding choices in my own research field of AI. However, it's interesting to compare the DARPA funding approach with the approach of, say, the Japanese government. Historically, the Japanese have had a tendency to fund huge, comprehensive, nationwide research programmes: e.g. the Fifth Generation computing initiative (which funded a large number of researchers to work on logic-based AI), or the current focus on robotics technology. As a crude approximation, it seems the Japanese funding system tends to push researchers to "all work on the same sort of thing at the same time", whereas the American research funding system is more chaotic, leading to a greater diversity of ideas getting explored simultaneously. We still are overly trend-following and narrow-focused in the US, from my point of view: for instance, AI funding has focused on narrow-AI, logic-based systems and neural net systems for far too long; and the biology community is taking way too long to wake up to the importance of systems biology. But, compared to the rest of the world, the US research funding system is a hotbed of creative chaos.

And then, once the Internet escaped the clutches of ARPA (due to the legislative action of folks like Al Gore, who famously bragged he "invented the Internet" due to his role in this political process), it spread through the collective activity of masses of software entrepreneurs. The Web was initially developed in Europe, but what made it a huge phenomenon was American entrepreneurship, pushed on by the relative ease of securing angel and venture funding in the US. I lived in Australia in the late 1990's but when I wanted to start a software business I had to return to the US because it was so hard to secure investment for an oddball software startup anywhere else (not that it was easy in the US, but it was a bit less painfully difficult...).

The Human Genome Project (which has ushered in a completely new era of genetics and medical research) was began as a US government initiative, involving a network of university labs. And note that the US graduate education system is still by far the best in the world. Our elementary and high schools are generally pathetic compared to those of other developed nations, though there are many exceptionally good schools out there too (the US being a big, diverse place) ... but by the time one gets to grad school, the US is the place to be. Top undergrads from around the world vie to get into our grad schools, and top PhDs vie for postdoc positions at our universities.

But what accelerated the Human Genome Project was the entry of Celera Genomics into the picture -- a venture-funded entrepreneurial attempt to outdo the government genome sequencing project. The new ideas Celera introduced (shotgun sequencing) accelerated the government sequencing project as well, helping the latter to complete ahead of schedule and under budget. (Now Craig Venter, who founded Celera, is involved with a number of projects, some commercial and some nonprofit within government-funded labs ... including a far-out attempt to create the first artificial genome.)

In each of these three cases -- and I could have chosen many others -- we see a complex combination of individual scientific and entrepreneurial initiative, and the spontaneously coordinated, somewhat chaotic and happenstance interaction of government, commercial and educational institutions. This combination isn't planned in detail, and doesn't always make sense, and makes a lot of really stupid decisions (such as not funding advanced AI research much more amply), but it also does a lot of smart things ... and it interpenetrates with subtle, hard-to-describe aspects of American culture in ways that no one has yet been able to document.

Part of the story, of course, is the incredible diversity of the American population: our scientists and engineers, especially, come from all over the world ... and increasingly our business leaders do too. So American culture isn't exactly American culture: it's really world culture, but with an American slant. And this is one among many major differences between America and other contemporary nations, which is closely linked to the "practical creativity" memeplex. I can't see anywhere in Asia, or anywhere in Europe (except possibly England), adopting the "melting pot" aspect of American culture ... but without this melting-pot aspect, it seems to me that practical creativity will have a lot harder time really flourishing. The diversity of ideas and approaches that comes from welcoming and then chaotically blending cultures and outlooks from all over the world, is a major source of practical creativity.

The move from a manufacturing and service economy to a knowledge economy has become famous. The next step, I suggest, is going to be a gradual shift from a knowledge economy to a creativity economy. As knowledge work becomes commoditized, the really precious thing will be creativity work: but not abstract creativity-work detached from the everyday world ... practical-creativity work, aimed at moving the real world forward in unexpected directions. Because of this, I suspect the US will maintain its cultural and economic leadership role in the world for quite some time.

And we'd damn well better, because with all the debt we're racking up, we're basically placing a huge BET that we're going to dramatically increase our productivity via technological efficiency improvements of various sorts. It's a fairly large gamble, but calculated risks are part of the American way ... as recent events on Wall Street show, this approach definitely has its risks ... but my guess is that this gamble will ultimately pan out just fine.

Getting back to my futurist preoccupations: My best guess is that the bulk of the work of creating the Singularity is going to be centered in America. This work will surely be international -- my own current work on advanced AGI technology involves a team with members in South America, Europe, Australia, New Zealand and Asia as well as the US (no Antarcticans yet...). But there's a reason my company Novamente LLC is centered in the US and not these other countries, beyond historical happenstance ... the US is the place where businesses and nonprofit agencies are most willing to seriously consider the practical value of way-out-there technologies. So long as this doesn't change, the American era is going to keep on rolling ... at least that's my best guess at the moment ...

AGI Intelligence Testing

I spent a while this weekend thinking about what might be the right approach for testing the intelligence of early-stage AGI systems that are aimed at human-level, roughly human-like general intelligence (either as an end goal or an intermediate developmental milestone).

Some of my thoughts are summed up in an essay I posted at

http://goertzel.org/agiq.pdf

I’ll quote the first few paragraphs here:

One of the many difficult issues arising in the course of research on human-level AGI is that of “evaluation and metrics” – i.e., AGI intelligence testing.

It’s not so hard to tell when you’ve achieved human-level AGI — though there is some subtlety here, which I’ll discuss below. However, assessing the quality of incremental progress toward human-level AGI is a much subtler matter. In this essay I’ll present some thoughts on this issue, culminating in a couple specific proposals:

1) Online School Tests, in which AGIs are tested via their ability to succeed in existing online educational fora

2) of more immediate interest, a series of tests called the AGI Preschool Tests (AIP Tests, for short, pronounced “ape tests”), based on the notion of “multiple intelligences” and also on some novel ideas regarding learning-based intelligence testing.

The AIP Tests suggested here are specifically intended for AGI systems that control agents embodied in 3D worlds resembling the everyday human world, via either physical robots or virtually embodied agents. Very differently embodied AGI systems (e.g. systems to be initially taught purely via text without any simulated human-like or animal-like body) would potentially need qualitatively different testing methdologies.

On the Preservation of Goals in Self-Modifying AI Systems

I wrote down some speculative musings on the preservations of goals in self-modifying AI systems, a couple weeks back; you can find them here:

http://www.goertzel.org/papers/PreservationOfGoals.pdf

The basic issue is: what can you do to help mitigate against the problem of "goal drift", wherein an AGI system starts out with a certain top-level goal governing its behavior, but then gradually modifies its own code in various ways, and ultimately -- through inadvertent consequences of the code revisions -- winds up drifting into having different goals than it started with. I certainly didn't answer the question but I came up with some new ways of thinking about the problem, and formalizing the problem, that I think might be interesting....

While the language of math is used in the paper, don't be fooled into thinking I've proved anything there ... the paper just contains speculative ideas without any real proof, just as surely as if they were formulated in words without any equations. I just find that math is sometimes the clearest way to say what I'm thinking, even if I haven't come close to proving the correctness of what I'm thinking yet...

An abstract of the speculative paper is:

Toward an Understanding of the Preservation of Goals
in Self-Modifying Cognitive Systems


Ben Goertzel

A new approach to thinking about the problem of “preservation of AI goal systems under repeated self-modification” (or, more compactly, “goal drift”) is presented, based on representing self-referential goals using hypersets and multi-objective optimization, and understanding self-modification of goals in terms of repeated iteration of mappings. The potential applicability of results from the theory of iterated random functions is discussed. Some heuristic conclusions are proposed regarding what kinds of concrete real-world objectives may best lend themselves to preservation under repeated self-modification. While the analysis presented is semi-rigorous at best, and highly preliminary, it does intuitively suggest that important humanly-desirable AI goals might plausibly be preserved under repeated self-modification. The practical severity of the problem of goal drift remains unresolved, but a set of conceptual and mathematical tools are proposed which may be useful for more thoroughly addressing the problem.

Playing Around with the Logic of Play

On the AGI email list recently, someone asked about the possible importance of creating AGI systems capable of playing.

Like many other qualities of mind, I believe that the interest in and capability for playing is something that should emerge from an AGI system rather than being explicitly programmed-in.

It may be that some bias toward play could be productively encoded in an AGI system ... I'm still not sure of this.

But anyway, in the email list discussion I formulated what seemed to be a simple and clear characterization of the "play" concept in terms of uncertain logical inference ... which I'll recount here (cleaned up a slight bit for blog-ification).

And then at the end of the blog post I'll give some further ideas which have the benefit of making play seem a bit more radical in nature ... and, well, more playful ...

Fun ideas to play with, at any rate 8-D

My suggestion is that play emerges (... as a consequence of other general cognitive processes...) in any sufficiently generally-intelligent system that is faced with goals that are very difficult for it .

If an intelligent system has a goal G which is time-consuming or difficult to achieve ... it may then synthesize another goal G1 which is easier to achieve

We then have the uncertain syllogism


Achieving G implies reward

G1 is similar to G

|-

Achieving G1 implies reward

(which in my Probabilistic Logic Networks framework would be most naturally modeled as an "intensional implication.)

As links between goal-achievement and reward are to some extent modified by uncertain inference (or analogous process, implemented e.g. in neural nets), we thus have the emergence of "play" ... in cases where G1 is much easier to achieve than G ...

Of course, if working toward G1 is actually good practice for working toward G, this may give the intelligent system (if it's smart and mature enough to strategize) or evolution impetus to create additional bias toward the pursuit of G1

In this view, play is a quite general structural phenomenon ... and the play that human kids do with blocks and sticks and so forth is a special case, oriented toward ultimate goals G involving physical manipulation

And the knack in gaining anything from play (for the goals that originally inspired the play) is in appropriate similarity-assessment ... i.e. in measuring similarity between G and G1 in such a way that achieving G1 actually teaches things useful for achieving G.

But of course, play often has indirect benefits and assists with goals other than the ones that originally inspired it ... and, due to its often stochastic, exploratory nature it can also have an effect of goal drift ... of causing the mind's top-level goals to change over time ... (hold that thought in mind, I'll return to it a little later in this blog post...)

The key to the above syllogism seems to be similarity-assessment. Examples of the kind of similarity I'm thinking of:

• The analogy btw chess or go and military strategy
• The analogy btw "roughhousing" and actual fighting

In logical terms, these are intensional rather than extensional similarities

So for any goal-achieving system that has long-term goals which it can't currently effectively work directly toward, play may be an effective strategy...

In this view, we don't really need to design an AI system with play in mind. Rather, if it can explicitly or implicitly carry out the above inference, concept-creation and subgoaling processes, play should emerge from its interaction w/ the world...

Note that in this view play has nothing intrinsically to do with having a body. An AGI concerned solely with mathematical theorem proving would also be able to play...

Another interesting thing to keep in mind when discussing play is subgoal alienation

When G1 arises as a subgoal of G, nevertheless, it may happen that G1 survives as a goal even if G disappears; or that G1 remains important even if G loses importance. One may wish to design AGI systems to minimize this phenomenon, but it certainly occurs strongly in humans.

Play, in some cases, may be an example of this. We may retain the desire to play games that originated as practice for G, even though we have no interest in G anymore.

And, subgoal alienation may occur on the evolutionary as well as the individual level: an organism may retain interest in kinds of play that resemble its evolutionary predecessors' serious goals, but not its own!

Bob may have a strong desire to play with his puppy ... a desire whose roots were surely encoded in his genome due to the evolutionary value in having organisms like to play with their own offspring and those of their kin ... yet, Bob may have no desire to have kids himself ... and may in fact be sterile, dislike children, and never do anything useful-to-himself that is remotely similar to his puppy-playing obsession.... In this case, Bob's "purely playful" desire to play with his puppy is a result of subgoal alienation on the evolutionary level. On the other hand, it may also help fulfill other goals of his, such as relaxation and the need for physical exercise.

This may seem a boring, cold, clinical diagnosis of something as unserious and silly as playing. For sure, when I'm playing (with my kids ... or my puppy! ... or myself ... er ... wait a minute, that doesn't work in modern English idiom ;-p) I'm not thinking about subgoal alienation and inference and all that.

But, when I'm engaged in the act of sexual intercourse, I'm not usually thinking about reproduction either ... and of course we have another major case of evolution-level and individual-level subgoal alienation right there....

In fact, writing blog entries like this one is largely a very dry sort of playing! ... which helps, I think, to keep my mind in practice for more serious and difficult sorts of mental exercise ... yet even if it has this origin and purpose in a larger sense, in the moment the activity seems to be its own justification!

Still, I have to come back to the tendency of play to give rise to goal drift ... this is an interesting twist that apparently relates to the wildness and spontaneity that exists in much playing. Yes, most particular forms of play do seem to arise via the syllogism I've given above. Yet, because it involves activities that originate as simulacra of goals that go BEYOND what the mind can currently do, play also seems to have an innate capability to drive the mind BEYOND its accustomed limits ... in a way that often transcends the goal G that the play-goal G1 was designed to emulate....

This brings up the topic of meta-goals: goals that have to do explicitly with goal-system maintenance and evolution. It seems that playing is in fact a meta-goal, quite separately from the fact of each instance of playing generally involving an imitation of some other specific real-life goal. Playing is a meta-goal that should be valued by organisms that value growth and spontaneity ... including growth of their goal systems in unpredictable, adaptive ways....

w0000000t!!!!

Machine Consciousness (report from the Nokia Workshop)

I just got finished with the two-day Workshop on Machine Consciousness that Pentti Haikonen organized at Nokia Research, in Helsinki.

I probably wouldn't have come to Finland just for this gathering, but it happened I was really curious to meet the people at RealXTend, the Finnish open-source-virtual-worlds team Novamente has been collaborating with (with an aim toward putting our virtual pets in RealXTend) ... so the workshop plus RealXTend was enough to get me on a plane to Helsinki (with a side trip to Oulu where RealXTend is located).

This blog post quasi-randomly summarizes a few of my many reactions to the workshop....

Many of the talks were interesting, but as occurs at many conferences, the chats in the coffee and meal breaks were really the most rewarding part for me...

I had not met Haikonen personally before, though I'd read his books; and I also met a lot of other interesting people, both Finnish and international....

I had particularly worthwhile chats with a guy named Harri Valpola, a Finnish computational neuroscience researcher who is also co-founder of an AI company initially focused on innovative neural-net approaches to industrial robotics.

Harri Valpola is the first person I've talked to who seems to have originally conceived a variant of my theory of how brains may represent and generate abstract knowledge (such as is represented in predicate logic using variables and quantifiers). In brief my theory is that the brain can re-code a neural subnetwork N so that the connection-structure of N serves as input to some other subnetwork M. This lets the brain construct "higher order functions" as used in combinatory logic or Haskell, which pose an equivalent mathematical alternative to traditional predicate logic formulations. Harri's ideas did not seem exactly identical to this, but he did have the basic idea that neural nets can generate abstraction via having subnets take-as-input aspects of the connection structure of other nets.

Once again I was struck by the way different people, from totally different approaches, may arrive at parallel ideas. I arrived at these particular ideas via combinatory logic and then sought a neuroscience analogue to combinatory logic's higher-order-functions, whereas Harri arrived at them via a more straightforward neuroscience route. So our approaches have different flavors and suggest different research directions ... but ultimately they may well contribute the same core idea.

I don't have time to write summaries of the various talks I saw or conversations I had, so I'll just convey a few general impressions of the state of "machine consciousness" research that I got while at the conference.

First of all, I'm grateful to Pentti Haikonen for organizing the workshop -- and I'm pleased to see that the notion of working on building conscious, intelligent machines, in the near term, has become so mainstream. Haikonen is a researcher at a major industry research lab, and he's explicitly saying that if the ideas in his recent book Conscious Robots are implemented, the result will be a conscious intelligent robot. Nokia does not seem to have placed a tremendous amount of resources behind this conscious-robot research program at present, but at least they are taking it seriously, rather than adopting the skeptical attitude one might expect from talking to the average member of the AAAI. (My own view is that Haikonen's architecture lacks many ingredients needed to achieve human-level AGI, but could quite possibly produce a conscious animal-level intelligence, which would certainly be a very fascinating thing....)

The speakers were a mix of people working on building AI systems aimed at artificial consciousness, and philosophers investigating the nature of consciousness in a theoretical way. A few individuals with neuroscience background were present, and there was a lot of talk about brains, but the vast majority of speakers and participants were from the computer science, engineering or philosophy worlds, not brain science. The participants were a mix of international speakers, local Finns with an interest in the topic (largely from local universities), and Nokia Research staff (some working in AI-related areas, some with other professional foci but a general interest in machine consciousness).

Regarding the philosophy of consciousness, I didn't feel any really new ground was broken at the workshops, though many of the discussants were insighful. As a generalization, there was a divide betwen participants who felt that essentially any machine with a functioning perception-action-control loop was conscious, versus those who felt that a higher level of self-reflection was necessary.

My own presentation from the workshop is here ... most of it is cut and pasted from prior presentations on AGI but the first 10 slides are so are new and discuss the philosophy of consciousness specifically (covering content previously given in my book The Hidden Pattern and various blog posts). I talked for half an hour and spent the first half on philosophy of consciousness, and the second half on AGI stuff.

I was the only vocal panpsychist at the workshop ... i.e. the only one maintaining that everything is conscious, and that it makes more sense to think of the physical world as a special case of consciousness (Peirce's "Mind is matter hide-bound with habit") than to think of consciousness as a special case of the physical world. However, one Finnish philosopher in the audience came up to me during a coffee break and told me he thought my perspective made sense, and that he was happy to see some diversity of perspective at the workshop (i.e. to see a panpsychist there alongside all the hard-core empiricists of various stripes).

My view on consciousness is that raw consciousness, Peircean First, is an aspect of everything ... so that in a sense, rocks and numbers are conscious, not just mice and people. However, different types of entities may have qualitatively different kinds of consciousness. For instance, systems that are capable of modeling themselves and intelligently governing their behavior based on their self-models, may have what I call "reflective consciousness." This is what I have tried to model with hypersets, as discussed in my presentation and in a prior blog post.

Another contentious question was whether simple AI systems can display consciousness, or whether there's a minimal level of complexity required for it. My view is that reflective consciousness probably does require a fairly high level of complexity -- and, furthermore, I think it's something that pretty much has to emerge from an AI system through its adaptive learning and world-interaction, rather than being explicitly programmed-in. My guess is that an AI system is going to need a large dynamic knowledge-store and a heck of a lot of experience to be able to usefully infer and deploy a self-model ... whereas, many of the participants in the workshop seemed to think that reflective consciousness could be created in very simple systems, so long as they had the right AI architecture (e.g. a perception-action-control loop).

Since my view is that

• consciousness is an aspect of everything
• enabling the emergence of reflective consciousness is an important part of achieving advanced AGI

my view of machine consciousness as a field is that

• the study of consciousness in general is part of philosophy, or general philosophical psychology
• the study of reflective consciousness is an important part of cognitive science, which AGI designers certainly need to pay attention to
  

One thing we don't know, for example, is which properties of human reflective consciousness emanate from general properties of reflective consciousness itself, and which ones are just particular to the human brain. This sort of fine-grained question didn't get that much time at the workshop, and I sorta wish it had -- but, maybe next year!

As an example, the "7 +/- 2" property of human short-term memory seems to have a very big impact on the qualitative nature of human reflective consciousness ... and I've always wondered the extent to which it represents a fundamental property of STM versus just being a limitation of the brain. It's worth noticing that other mammals have basically the same STM capacity as humans do.

(I once speculated that the size of STM is tied to the octonion algebra (an algebra that I discussed in another, also speculative cog-sci context here), but I'm not really so sure about that ... I imagine that even if there are fundamental restrictions on rapid information processing posed by algebraic facts related to octonions, AI's will have tricky ways of getting around these, so that these fundamental restrictions would be manifested in AI's in quite different ways than via limited STM capacity.)

However, it's hard to ever get to fine-grained points like that in broad public discussions of consciousness ... even among very bright, well-intentioned expert researchers .. because discussion of consciousness seems to bring up even worse contentious, endless, difficult arguments among researchers than the discussion of general intelligence ... in fact consciousness is a rare topic that is even harder to discuss than the Singularity!! This makes consciousness workshops and conferences fun, but also means that they tend to get dominated by disagreements-on-the-basics, rather than in-depth penetration of particular issues.

It's kind of hard for folks who hold different fundamental views on consciousness -- and, in many cases, also very different views on what constitute viable approaches to AGI -- to get into deep, particular, detailed discussions of the relationship between consciousness and particular AI systems!

In June 2009 there will be a consciousness conference in Hong Kong. This should be interesting on the philosophy side -- if I go there, I bet I won't be the only panpsychist ... given the long history of panpsychism in various forms in Oriental philosophy. I had to laugh when one speaker at the workshop got up and stated that, in studying consciousness, he not only didn't have any answers, he didn't know what were the interesting questions. I was tempted to raise my hand and suggest he take a look at Dharmakirti and Dignaga, the medieval Buddhist logicians. Buddhism, among other Oriental traditions of inquiry, has a lot of very refined theory regarding different states of consciousness ... and, while these traditions have probably influenced some modern consciousness studies researchers in various ways (for example my friend Allan Combs, who has sought to bridge dynamical systems theory and Eastern thought), they don't seem to have pervaded the machine-consciousness community very far. (My own work being an exception ... as the theory of mind on which my AI work is based was heavily influenced by Eastern cognitive philosophy, as recounted in The Hidden Pattern.)

I am quite eager to see AI systems like my own Novamente Cognition Engine and OpenCogPrime (and Haikonen's neural net architecture, and others!!) get to the point where we can study the precise dynamics by which reflective consciousness emerges from them. Where we can ask the AI system what it feels or thinks, and see which parts of its mind are active relevant to the items it identifies as part of its reflective consciousness. This, along with advances in brain imaging and allied advances in brain theory, will give us a heck of a lot more insight....

Me on "The Future and You" podcast

I did an interview recently on "The Future and You" podcast ... interested parties may, er, feast their ears at

http://www.thefutureandyou.libsyn.com/index.php?post_id=368080

Topics include AGI, uploading, the nature and ultimate immateriality of the self, continuity of consciousness, and all manner of other expectable rambling blablabla ;-)

In Memoriam Leo Zwell

History is a damn dim candle
over a damn dark abyss
-- W.S. Holt

My grandfather Leo Zwell died last week at age 92, so I thought I'd write a blog post (inadequately) commemorating his existence and lamenting his passing.

Leo and me, 1967

What a really exceptional person he was, and how glad I am to have known him.

He was a crystallographer by profession, and a really outstanding grandfather, but most of all I'll remember him as two things (in no particular order):

• an inquisitive, careful, always-processing, generally-interested mind
• a caring, loving human who always wanted to help, and to see that others were doing well

His attitude toward humanity was a subtle one, alluded to by a phrase I remember from a poem I wrote for him and my grandmother when I was 18 or so: "Cynically and with innocence." Of course, this is a rather typically Eastern-European-Jew sort of almost-but-not-quite-self-contradictory attitude, but he manifested it in a uniquely warm and thoughtful way.

He saw humans as hopelessly flawed, screwed-up animal creatures, dealt a mixed hand by evolution (a frequent saying of his was, "We're really just animals. Considering that we're really just animals like all the other animals, we're really not so bad ... we've actually come a long way.") -- yet he was dedicated to getting the most out of the flawed human-animal mind by understanding the world around him and encouraging others to do the same ... and to helping others nudge their flawed human existences in the direction of satisfaction and growth and cognizance rather than suffering and foolishness....

He thought people sometimes expected too much of humanity, given that after all we're just animals with mildly hyperdeveloped craniums. But then, he was also prone to push people to think more and more, to consider perspectives and avenues beyond what their culture or personality or habitual mind-set presented them with. It wasn't exactly "hope for the best, expect the worst" -- more like "expect the mixed-up and confused, but keep pushing for the better and better."

In conversations and in his own thinking on everyday or scientific topics, he was always willing to approach the world simply, in the manner of a child, just looking at reality without preconceived notions ... yet was also extremely interested in accumulating knowledge, and critical of people who govern their thinking and living in an ignorant way.... He was always great with children (at least until his very last years, when he occasionally became impatient), and one of the things he liked about kids was their natural inclination to be scientists and observers: he was always concerned to carefully and openly observe the world around him (and within him) and see what was actually there.

On a personal level, he was extremely important to me in two specific ways (beyond just being a loving and helpful family member), connected to the two qualities. I mentioned above...

Firstly: He was the one who got me into hard science, in the first place. Both of my parents were into social change and social science and such, and the general milieu of my early youth was left-wing hippie nonconformists, not hard scientists concerned with understanding physical or mathematical reality. Whenever we visited Leo (and we moved near him when I was 6, so I saw him often after that) he lectured me on physics, chemistry, biology, mathematics and what-not ... and he had an endless reserve of stories about experiments he'd done, famous and non-famous scientists he'd worked with, and so forth.

Leo taught me some practical things about science and math, such as tricks for doing mental arithmetic ... and simple trigonometry, Newton's method, the basics of X-ray diffraction and so forth -- but that wasn't really the main point ... I could learn that stuff from books ... the main point was his enthusiasm for science and his immersion in the culture of science ... in the culture of thinking, learning, and communicating with a goal of incrementally understanding more and more about the world.... The cognitive/adaptive/communicational attitude was something he applied to every aspect of existence, including everyday human life, but he saw the scientific sphere as the place where thought and in-depth communication could really flourish.

The other major gift he gave me was: As a child, he was the only example I saw, up-close, of a man who had gotten really deeply into taking care of his kids. My father Ted Goertzel was a very good father and I learned an awful lot from engaging with him in wide-ranging intellectual conversations throughout my childhood ... we also traveled a lot together and played sports together and did many other rewarding things ... but, in our house as a child it was my mom who did the vast bulk of childcare. Leo on the other hand had taken on a large amount of the responsibility for taking care of my mother and her brother, himself -- and I could see this, even as a child myself, in the nature of his relationship with his adult children. I thought this was pretty interesting, and I could see that both he and his children had gained a huge amount from this (at the time, quite unusual) pattern.

Seeing the relationship between Leo and my mom was a lot of what imprinted me with the idea that caring for my kids was something I was supposed to do myself (another thing pushing me in this direction, was probably my mother's own radical-feminist emphasis on male-female equality) ... an idea that continues to absorb a significant portion of my life, as I have ~50% shared-with-my-ex-wife custody of my two still-under-18 kids (my oldest, Zar, is 18 and now a junior in university: damn that makes me feel old!!).

It is impossible for me to estimate the amount of personal reward I've gotten from following Leo down the child-caring path ... or the amount to which my thinking about human and AI cognition has been influenced by carefully observing and partaking of the mental and emotional development of my 3 kids....

These days it is not that shocking for a father to take an intensive hands-on role, but for Leo's generation it was anomalous, and -- while there were some specific situational reasons that pushed him in this direction, such as some temporary health problems on his wife's part -- I'm sure much of the the reason he took on this role as intensively as he did was just his overall passion for being helpful. His kids needed and valued his help, so from his view, it would have been unnatural not to provide as much help as was sensibly possible. He could have left it to his wife, but as a rule, he never really was one to leave things to others (another meme I've adopted from him: I too like to Get Things Done, and have a strong tendency toward getting them done myself rather than relying on others...).

I only knew Leo for the last 41/92 of his life, and I'm not going to try to convey nearly all that I knew of him, but I hope the observations I'll make here will transmit some meaningful (if tiny) fraction of the essence of the person.

This blog post will be sort of disorganized (it already is, and I'm just getting started!) ... I'm going to jump around in time a bit ... but that's the way memory works ... and anyway, as Leo and I discussed a few times, modern physics tells us the directionality of time is projected by the perceiving mind...

The end was distressing but, at least, not marked by great suffering. Leo's body faded gradually during the last few years, with an especially fast decline in the last 6 months after a surgery ... his death itself was about as humane as deaths get, with his daughter (my mother, who did an incomprehensibly great job of caring for him during his last years, all while holding down a very demanding more-than-full-time job running a social service program) at his side. He spent much of his last few hours listening to family members bid him farewell over the phone, and then he curled up in a fetal position and reluctantly let his worn-out body shut down.

One of the most interesting things about the last few years was what an astute observer he was about his own mental and physical deterioration. As each aspect of his functioning declined (arithmetic ability, memory for faces, memory for names, sense of direction), he would note this and analyze the particular nature of the deterioration. And he worried to the end about imposing a burden on his family, to which we would always reply that he had helped us a huge amount in the earlier part of his life, so we were more than happy to return the favor.

Indeed, helping others was -- as I already hinted above -- one of the biggest themes of Leo's life. The receptionist at the old folks' home where he lived for his last few years (Martin's Run) likes to tell a story about the day that Leo felt she looked bored and hungry sitting at her desk by the entrance, so he went to the kitchen and brought her some soup. Walking was slow for him at that time, but, he thought it was imperative that the receptionist be treated well. I don't remember any of the other residents at Martin's Run paying nearly so much attention to the well-being of the hired staff. But this meme went way back in Leo's life, to his roots growing up in Brooklyn during the Great Depression, and then his political involvement in socialist organizations a little later on.

(His interest in socialism was never an abstract or theoretical thing; while, as a scientist, he appreciated very much the value of good theories with explanatory power, he never took much stock in social theories, considering them largely mumbo-jumbo. His interest in socialism was always very practical: he saw the world as full of people who needed help, and he thought that society should be organized in such a way that they got the help they needed. After the truth about the Soviet Union became clearer in the 1970's and 1980's he backed away from the more Marxist variants of socialism, but remained strongly attached to the caring-oriented values at the heart of the democratic socialist tradition.)

The (woefully inadequate) obituary in the Philadelphia Inquirer, after his death, read as follows:

Leo Zwell, 92, a retired scientist with the Joint Committee of X-Ray Powder Diffraction Standards in Swarthmore, died of heart failure on Wednesday at Martin's Run, a retirement community in Media where he had lived since 2002.

Mr. Zwell was a physicist with the Jet Propulsion Laboratory at the California Institute of Technology, U.S. Steel, and the U.S. Bureau of Standards before joining the Swarthmore firm in 1972.

He was a 1934 graduate of Brooklyn College. "He graduated at the age of 19," said his son, Michael, chief executive officer of his own human resources firm in Chicago.

"He was working full time" while in college "to put himself through school and contribute to his family's income," his son said. In the depths of the Depression, there was no money for advanced studies.

The Joint Committee, Michael Zwell said, is a publisher of research about X-raying of materials "to identify what their atomic structure was."

Besides his son, he is survived by his daughter, Carol Goertzel; sisters Gladys Berman and Priscilla Endler; four grandchildren; and a step-grandchild.

Mr. Zwell's wife of 54 years, Etta, died in 1994.

A memorial service will be held at 12:30 p.m. today at Martin's Run, near Route 320 and Paxon Hollow Road outside Media.

A spokeswoman for the Humanity Gifts Registry said Mr. Zwell had donated his body to science.

(Small digressive note: Perhaps the most peculiar thing about this obituary is its failure to mention that my mother Carol Goertzel runs a very successful social service program in the Philadelphia area. Apparently some obituary-writers don't do much homework.)

me and my mom Carol (Leo's daughter),
kickin' it old school back in the good old days ...
1968 or so, Eugene, Oregon,
in the midst of all the hippy madness...
but I was more concerned with my box
than science or revolt or the Vietnam war...

Leo graduated high school at 14 (so he wound up entering college at 15, just as I did ... though he was a younger 15) and went to Brooklyn College, where he majored in chemistry. Unlike me (whew!) he worked full-time while in college, helping out in his father's parking garage (which he found rewarding though at times exhausting ... and one of his favorite topics of discourse was the incredible ethical and intellectual characteristics of his father, Pop Charlie ... Charlie Zwagilski before immigrating to the US from Eastern Europe and getting his name mutated).

Pop Charlie (holding my sister Rebecca)
and Grandma Sarah (holding me)

Pop Charlie before I knew him, back in the day

Not too long after graduating college Leo joined the WWII war effort, stationed in the lab rather than the battlefield, working on various projects related to creating better metallic compounds for use in missiles and other military devices. Like many others of that time, he found his innate tendency to pacifism called into question by the Nazi threat. All or nearly all of his relatives who had not emigrated to the US, were wiped out in one or another anti-Jewish purge in Eastern Europe.

After the war ended, he spent some time as a researcher at the Jet Propulsion Laboratory in California, but eventually had to leave there due to McCarthyism: he was actively involved in efforts to politically organize scientists and engineers, and was hence falsely accused of un-Americanism. This episode had a lasting impact on his psyche, imprinting him with a cynicism about human nature and society that never quite left him after that. But, he went on to a very successful research career at US Steel in Pittsburgh, a major industrial research lab of its time ... and then after "retiring" from US Steel, spent 20 years working at the Joint Committee of X-Ray Powder Diffraction Standards in Swarthmore PA, where he pored over the crystallography research literature and figured out which of the many results published there seemed solid enough to enter into JCPDS's extensive databases.

(I briefly had a job helping him him with some of his JCPDS work during the summer of 1982, after my first year of college, but my duties consisted largely of photocopying and I confess I rapidly quit the job even though the small amount of pay was useful to me ... I did not, and still don't, have the patience to enjoy that sort of work....)

Leo during the JCPDS period

Had he been born a little later (a complicated counterfactual, but let's roll with it for the moment...) he would surely have gotten a PhD; but in his day you could do serious science without one ... and he surely did a lot of it. One trait of his that I did not inherit was his surpassing modesty: though he was brighter and more knowledgeable than many of the more famous and well-recognized scientists in the labs he worked in, he genuinely did not envy their greater glory and external recognition. Rather, he was contented to work in the background, analyzing peoples' data for them, solving the tough problems that no one else could solve, and pushing science ahead by helping others with their work as best he knew how.

Another excellent trait of his that I did not inherit was his incredible carefulness. He would review a body of data again and again, with never-failing concentration and precision, looking for subtle patterns, or subtle clues that some sort of error or irregularity might have occurred. There were various commonly-used formulas in the crystallographic literature that he considered inaccurate because the data from which they were derived had various issues; and I have little doubt he was correct. While I am not cognitively suited to be as careful as he was, watching his approach to understanding data was extremely educational for me, and throughout my career I have sought to work with people who are careful in the same way that he was, knowing that this is a virtue I lack.

He also taught me the value of teams in science. My inclination is to be a lone wolf and seek to range far and wide from others trying to solve the hardest problems by idiosyncratic means -- but he showed me that in science, an awful lot of things can only be achieved via different people with different strengths working together. What I'm doing now in my own career exemplifies this: in my thinking I remain a pretty-far-out-there lone wolf, but I'm happy to be working with a diverse, well-integrated R&D team that probably shares many characteristics with the teams he worked in at US Steel.

One theme that he often returned to in his ruminations and (extensive, sometimes rather excessive!) storytelling was that of Generalists versus Specialists. He considered himself more of a generalist, as he was always seeking to synthesize knowledge from different areas, and look for overall patterns of organization. On the other hand he also had deep specialized knowledge of particular areas such as X-ray diffraction data analysis. Only by integrating generalist and specialist traits, he felt, was it possible to really make profound scientific progress. He saw too many scientists as being generalists or specialists only, and felt that for this reason a lot less progress was made than could otherwise be the case.

In part, it was probably his very humbleness that allowed him to be so helpful to so many other scientists, during the course of his career. As they knew he didn't care to compete with them, they were comfortable sharing their doubts, questions, ideas and hard problems with him. He was always interested in arguing intellectual points -- with anyone, be they a famous scientist or a two year old child -- but rarely was there any rancor involved ... it was very much passionate, abstract argumentation in the Greek tradition. Ideas meant a lot to him, yet it was rare for him to hold someone's bad ideas against them as a human being. He had friends with whom he profoundly disagreed.

I recall, when my sister Rebecca (now a school principal) was 6, Leo was lecturing her on the need to avoid fashionable clothing and such, because making fashion statements was foolish and involved emphasizing the wrong things. (He liked to say how he'd worn the same clothes all his life, and watched them go in and out of style.) Rebecca argued back that being determinedly unfashionable was itself a kind of fashion statement, so should also be avoided according to his principles. He laughed and agreed with her, quite willing to take-lightly his own studied unfashionableness, and complimented her on her thoughtfulness. (Of course, Rebecca ultimately grew up not to be the kind of person who reads fashion magazines....)

One of Leo's and my biggest, long-running arguments regarded the future of humanity.

He believed human nature to be fundamentally flawed, and figured that all attempts to reform society and improve human nature were doomed to fail, based on the fundamentally screwed-up essence of human nature. He saw the scientific process as having a greater perfection than human nature, but still being deeply flawed in various ways due to the underlying flaws in the humans doing the science (when led to, for example, wrong formulas being perpetuated because the individuals advocating them were famous or people were simply too lazy to study the underlying data correctly).

When I argued to him that science could in fact be used to improve human nature, by modifying the brain or by uploading people into computers, he basically laughed off the idea. Not that he felt it was impossible, but that he felt he didn't have the conceptual background to really think about it thoroughly. I really wish I had discussed these topics with him when he was 55 or 60 rather than 75-92, but it just didn't occur to me, mostly because our conversations were more dominated by his own (interesting) interests. The most evocative thing he ever said on this topic was something like: "Fine, Ben, but by the time you modify the human brain to remove all the ethical problems and the foolishness, what you'll have won't be human anymore."

In other words, he saw screwed-up-ness as essential to the nature of humanity ... and he understood himself as inextricably part of this human web of screwed-up-ness ... but nevertheless he felt compelled to devote himself to gaining more and more understanding and helping all the other screwed-up humans as much as possible.

Still, I always had the feeling that if we argued long enough, I might have been able to bring him around to a point of view closer to my own radical futurism. Another of his excellent qualities was his ability to change his ideas and attitudes -- no matter how deep-seated -- based on evidence and reflection. This was evident in his scientific work, and also in his personal life: for instance he was raised with relatively sexist and homophobic attitudes (by modern standards), but gradually revised these over time as he observed they really did not explain what he saw around him, nor accord with his desire for general human happiness.

I also remember arguing with him that it might, one day, be possible to resurrect the dead in a scientific way. My argument was: If quantum theory is correct then all the information about everything that has ever happened is encoded in the perturbations of particles in the universe now, so that in principle dead people could be reconstituted from this information, if a being were smart and powerful enough to collect it and do the appropriate nanoengineering. On the other hand, who knows if quantum theory is correct ... and there are, er, some engineering difficulties in this plan. Leo certainly found this train of thought amusing -- but he was not of the emotional cast to draw any hope from it. As far as he was concerned, once he was dead, he was gone, and that was the end of it ... remote possibilities regarding far-future weirdball engineering feats didn't really enter his emotional world. He did not want to die, but he accepted it, and had no patience for superstitions or wishful thinking.

Another comment he frequently made was that he was "never bored." He claimed not to really understand the concept of boredom. "I always have my own mind," he said. "How can I be bored? There's always so much to think about and to wonder about."

Indeed when your attitudes are "Question Authority!" and furthermore "Question Everything!" (two attitudes that came down to me from both sides of my family with pretty overwhelming force), boredom is hard to come by, because there are always so many things to be questioning....

Even at the end, in fact, when his powers of thought were a fraction of their previous, he STILL was never-bored, and was always thinking and trying to understand things. At the very end, when his memory was gone, he was continually asking questions about the objects in the hospital room: what is that? What is that for? Who put that there?

In his last few hours, right before he lost the energy to talk, he was counting ... he counted from 1 to 36, slowly and carefully, as if to be sure all the numbers were still there, as if by attaching himself to the Platonic realm of numbers he was connecting with a reality more substantial than his fading body and memory ... as a non-religious person, he had no delusions of heavens or hells, but beyond our own personal worries, concerns and attachments, there is always the more permanent and perfect world of the Numbers ...

He had a great love of measurement as a way of understanding the world, and when we emptied out his drawers, boxes and closets after his death, we found a remarkable number of rulers, yardsticks, protractors, compasses, calculators and slide rules. (At a certain point, in Swarthmore, he adopted the habit of asking each visitor to his apartment, as soon as they walked in the door, to name their height ... and then proceeding to measure them, taking his daily dose of kindly schadenfreude from observing how nearly all males tend to overestimate their own heights.) Perhaps his most prized personal possession was his watch: when in the hospital the nurses took his watch from him, he felt completely at a loss until it was returned. I was reminded of the historical theory that the main reason Western civilization advanced so much further than others (such as the Chinese) was the invention, in the middle ages, of quantification, of precise empirical measurement. His career was based on measuring things and recognizing patterns in these measurements, and he was concerned with this till the end.

Well ... having written all that, it still seems pathetically inadequate, and there is so much more to say. Most of all I have left off the very, very long list of people whom Leo and his wife Etta helped in various ways during their lives -- going beyond family and colleagues, comprising a remarkable assemblage of individuals whom they encountered in one random way or another and tried their best to help on their ways through life.

I loved all 4 of my grandparents (the others died some time ago) ... and my other grandfather, Victor Goertzel, was also an accomplished scientist (a psychologist) with whom I had considerable intellectual interaction (for instance, when I was in my early 20's, Victor and my grandmother Mildred and my father and I co-authored a biography of Linus Pauling together) ... but I have to say that Leo is the only grandparent whom I really internalized, to the point of view where I sometimes feel like I have a miniature Leo Zwell homunculus living in some obscure corner of my brain, pointing out to me when someone needs help, and pointing out to me when some point on a data chart is likely to be an outlier, and urging me to doubt all my beliefs and ideas, especially the ones that are most important to me.

One of my favorite phrases was taught to me by my friend Bruce Klein, founder of the Immortality Institute and my collaborator in Novamente LLC: "To abolish the plague of involuntary death."

Indeed: few goals are as important. So sad that it did not happen in time for Grandpa Leo.

He had a good life and a very useful one (not always happy, there were bouts of depression and the usual real-life troubles, but overall a richly rewarding human existence) ... but even at 92 years, I can't help thinking that this excellent person's life was far, far too short.

P.S. The photos included in this blog post are among the many we took from Leo's apartment after his death.

ISO a non-religious foundation for the process of "taking responsibility"

This is a re-post (with light edits) of a post I made last week, that got disappeared due to some IT difficulties regarding poor communication btw blogger.com and my Web hoster.

These were some late-night thoughts about the conceptual logic of morality and responsibility, written to the tune of Charlie Parker's glorious "Au Privave" (hot on the heels of "Step into the Realm" by the Roots, one of the few hip-hop bands I like at all...).

I guess what I am inching toward here is some sort of cognitive theory of moral responsibility ... but I'm really inching there, one teeny little piece at a time.... (Well, some topics lend themselves to speed better than others....)

I'll start with will, and then move on to my main topic of taking personal "moral responsibility" for one's actions.

Don't worry, I haven't turned into a preacher yet (though I haven't shaved for a while and am sporting a fairly spiffy Jesus-like beard, though I've been planning to shave for a few days and just haven't found the time), my basic orientation on these topics is one of systems theory....

So, for starters: Anyone with any sense knows by now that the intuitive feeling of "free will" we have is illusory. Our unconscious decides for us, before any conclusion is derived by the process of conscious ratiocination that feels like it's making a decision.

So why bother with the decision process at all? Why not just go with the flow of the non-ratiocinative unconscious? Because we know that the intensely-conscious decision process helps dynamically restructure our long-term memory in a way that will help our unconscious make better decisions in the future.

Next: Anyone with any sense knows that the notion of "moral responsibility" is, to a large extent, a hanger-on from obsolete religious belief systems.

And, the notion of "taking personal responsibility" for one's actions has -- in most particular instances -- questionable empirical grounding. After all, anything any one of us does, is to a large extent caused by our social and physical context -- as Saddam famously said in the South Park movie: "It's not my fault that I'm so evil ... it's society ... society...." Of course, it really IS society ... that cute little pseudo-Saddam wasn't lying ... and in any particular case, none of us really has the information to tease out the internal from the external causes underlying any of our actions ... but yet, this is a poor perspective to take, in spite of the element of truth underlying it.

(Causality, in the end, is not really a scientific concept anyway: it's a tool that minds use to understand the world. A causes B, from the perspective of mind M, if

• A precedes B
• The probability of events in class B is differentially higher, given the prior and correlated occurrence of events in class A.
• M can fairly confidently analogize that, if it were to carry out some action similar to A, then some event similar to B might be likely to follow

But that's a topic for another blog post, another day ... and is covered to some extent in the last chapter of my co-authored book Probabilistic Logic Networks, which Springer is supposed to publish this month...)

Sooo .... Why bother with the process of "taking moral responsibility" at all? Because we know that doing so helps us structure our long-term memory in a way that will help our unconscious take better actions in the future.

When we do something we wish we hadn't done, the act of assigning "responsibility" to ourselves causes us to insert a "correction signal" into our unconscious, which then modifies the structure of our internal declarative/procedural knowledge base in a way that makes it less likely we'll do similar regret-worthy things in the future. This is the case even though we (i.e. the deliberative, ratiocinative, "decision process" aspect of ourselves) don't know that much (rationally or intuitively) about how the unconscious works, and can't really untangle the various causal threads weaving through our minds and our worlds and leading to our actions.

The "ordinary waking state of consciousness" that most people occupy most of the time, involves a coupling of ratiocinative-decision-making with the free-will illusion, and a coupling of moral-responsibility-taking with some semi-religious notion of moral-agency. But it's possible to get into a state of mind where you carry out ratiocinative-decision-making and moral-responsibility-taking without any significant illusions attached to them ... simply because these cognitive dynamics tend to lead to effective overall system functioning.

Now, when I say "it's possible to get into a state of mind where X holds", am I saying that "by exercising one's free will, one can cause oneself to get into a state of mind where X holds" ?

Not really. What I'm saying is that "Sometimes the self-organizing dynamics of a mind coupled with an environment will result in that mind getting into a state of mind where X holds."

And what's the point of me telling you this? Well, some states of mind want to spread from one mind to another....

The basic idea is: If one does not internally take responsibility for one's own actions, one will never send those necessary correction-signals to one's own unconscious. Then one will just keep on doing those regrettable things.

Removing the obsolete, flawed quasi-religious concepts of blame, shame and so forth from one's inner mental landscape is an important step toward becoming a rational and self-aware, fully-realized person; but, once they are removed, they need to be replaced with something else ... they need to be replaced with a recognition of the mind as a holistic, complex dynamical system; and with a recognition of the role of the deliberative, ratiocinative aspect of mind as modulating the complex nonlinear dynamics of the unconscious.

None of us can control ourselves, none of us is fully aware of the dynamics by which we operate (in part because of basic information-theoretic limitations on the extent to which any finite system can understand itself; in part because of information-theoretically unnecessary limitations posed by the human brain architecture, which did not evolve in situations where acute self-awareness and mental self-control were key aspects of the fitness function). But "we" (the deliberative, ratiocinative "phenomenal self" portions of our minds) can modulate the dynamics of the other portions of our minds, via doing things like rational-decision-processes and responsibility-taking....

Our Lovable Species

I thought you might enjoy this email that I (and some others) received this morning. It is fairly typical of its genre, but particularly amusingly worded. I guess it doesn't need much comment; it kinda speaks for itself.

What scares me a little is that this sort of attitude is probably far more common than the transhumanist/Singularitarian/futurist attitude of most of the people I regularly communicate with ... and would probably be even more common if more people were aware of the kind of work that futurist-minded scientists and engineers are doing!

Clearly we need some more Hollywood flicks in which (preferably really hot-looking) AGI's save the world....


From: ********
Date: Sat, Aug 9, 2008 at 3:23 AM
Subject: I hope you realize.....
To: ****

I hope you people realize the immense stupidity, not intelligence, of what youre undertaking.

Anyone with any common sense would not allow this kind of research to reach its natural conclusion: the destruction of our humanity, such as what the transhumanists want to do.

Implants which are used to communicate across networks, or AI's so powerful that they literally take over normal human responsiblities. AI's would conclude we are incapable of caring for ourselves without help if they got advanced enough...

Take Kevin Warwiks work at the University of Reading. Absolutely monstrous. Everything he does, and everything you people do creates major issues in ethics and bioethics.

Trying to make people better with machines...We already are better then any cyborg you can make.

Anyone with any kind of common sense would reject these sort of things. Keep computers in a sense below where we are at.

Anybody who even thought about this, would say hey isnt there a problem with sticking machines in people, first it will be the disabled at first, sure, then normal people because somehow its cool and then alot of other horrible things...

Our technology is outpacing our wisdom, our humanity, our ability to comprehend the ethical boundaries beyond this, when you start getting into the term super intelligence.

The super intelligent thing would be to shutdown implants that react directly with the human mind, exceptions being people that medically benefit, such as alzheimers, the crippled, etc, but they should never be in perfectly healthy human beings.

If God had intended that machines be a part of us, he would have made us like that. But who knows if you guys even believe in Him.

No matter, just be aware of the dangerous waters you swim in. Our technology is so far ahead of our spirituality, that it represents a singularity of its own. A black hole that will suck us into oblivion unless you take a step back and realize the dangers of all this heady research.

So stop a bit, smell the roses, then reflect on this whole madness, Heres to You Mr Warwick of Reading, God help us all.

Self is to Long-Term Memory as Awareness is to Short-Term Memory

This is a brief addendum to a post I made a while back where I gave a casual but mathematical discussion of will and reflective awareness in terms of self-referential structures ("hypersets").

There, the following recursive definitions are given:

"S is conscious of X" is defined as: The declarative content that {"S is conscious of X" correlates with "X is a pattern in S"}

"S wills X" is defined as: The declarative content that {"S wills X" causally implies "S does X"}

Funky, huh? Chew on that for a while!

My point here is to posit a similar definition for that strange beast called the "phenomenal self" (and for a gloriously, Germanically thorough treatment of this entity, please read Thomas Metzinger's masterwork Being No One):

"X is part of S's self" is defined as: The declarative content that {"X is a part of S's self" correlates with "X is a persistent pattern in S over time"}

One thing that's nice about this definition is the relationship that it applies between self and awareness. In a formula:

Self is to long-term memory as awareness is to short-term memory

Elegant, huh?

Your self is nothing more or less than the awareness of your persistent being.

Your momentary awareness is nothing more or less than the self of your instantaneous being.

(Time-span makes a big difference! Indeed, time is almost equivalent to "difference." But that's a subject for another post, for another late-night fueled by too much green tea and too many weird ideas...)

AGI in Xiamen ... and some rambling on the "creativity economy"

I just returned from 2.5 weeks in the Orient ... a week in Japan, doing biz meetings, going to a virtual worlds conference, seeing an awesome guitarist at a weird bar called BarTube, visiting an old friend, and hanging out w/ my son who is staying there for a month studying Japanese and playing Go ... a day or so in Seoul (visiting a humanoid robotics research group and a virtual pets company) ... a few days in Hong Kong giving a talk at WCCI's Human-Level AI session (on how to make a human-level NLP system by partly cheating, see the paper at http://novamente.net/papers) ... and a week in Xiamen, which is in China right across the water from Taiwan ... here's the beautifully situated Xiamen University ...

My friend Hugo de Garis (inventor of evolvable computing, prophet of the Artilect War and all around creative thinker)

(there's me and Hugo and his wife) ... is now a prof at Xiamen University and he's pulling together a humanoid robotics team, whose goal is to spend 4 years making an intelligent computer brain for a Nao humanoid robot:

Hugo and I have been plotting a way to make a clever Nao via using his evolved neural nets for perception and action, and the OpenCog system for cognition and overall system control. The Xiamen folks seem to like the plan and we're discussing the possibility of me spending a couple months there each summer to collaborate, and them funding some students to work on the OpenCog side of the project. If someone follows ahead with the idea I've been selling, of integrating a robot simulator (like Gazebo) with a virtual world (like OpenSim), this could synergize really nicely with Novamente's AI-in-virtual-worlds stuff....

And Xiamen would be a very nice place to spend summers...

I've been fascinated by China since youth, probably due to my mother doing grad work in Chinese history and philosophy back then. She gave me a bunch of Chinese history books and stories and poetry to read, which made me fascinated with the culture. When I was 17, halfway thru my 3rd year of university, I applied for a scholarship to go to China for a year and do research relating non-well-founded sets (hypersets) to Buddhist cognitive philosophy. But the scholarship required me to know Chinese and I didn't, so I didn't go.... (I've never been able to put much energy into learning languages... too much other interesting stuff to study and think about ... and I find it hard to pick up languages via immersion because of my habit of not paying attention to what anyone is saying or doing around me ... so I'm rarely actually immersed in anything but my own thoughts ;-O ) ...

ANYWAYS ... I met F2F Novamente & OpenCog's Chinese contributors, which was very nice... here we have (back: Lian Ruiting, Guo Junfei, Chen Shuo, Rui Liu, me)

Very smart, interested, ambitious people!

Hugo is convinced that China is the country of the future and America is already obsolete. He foresees a coming century of reverse brain drain, where China recruits smart scientists and engineers from Western nations....

It might happen -- I don't rule it out. Of course, unlike Hugo, I think some sort of technological Singularity is very likely by mid-century and maybe sooner -- but let's ignore that for the moment ... talking just in conventional political/cultural terms, it's not obvious to me that he's right.

No doubt China has very many very smart and ambitious and hardworking people (like the ones pictured above!) ... but the cultural differences w/ the West are profound and I don't think any of us understands what they mean in terms of the future of science and engineering.

One observation I like to make is as follows. People talk about the knowledge economy ... where manual work has long been outsourced to 3rd world countries, leaving 1st world countries increasingly consumed w/ knowledge work.

And more and more so, the US becomes a pragmatic knowledge integration economy -- specialized knowledge like programming and science gets farmed out to 3rd world countries, but the task of integrating together various pieces of knowledge for practical purposes is still done in America. Even in Novamente, which is a damn international company, we do programming and science and project management overseas, but the figuring-out of what programming and science needs to be done to serve business goals, is largely done in the US. Because the US is where our customer companies are -- even if their work is largely done overseas, the high-level staff defining their vision are mostly here. The matching-up of technology and business, where Novamente is concerned, occurs mainly within the arena of US culture. (We do have overseas customers, but they are either run by Americans or following business models that closely copy American ones.)

The next step, I think, is the creativity economy. Even integrative knowledge will become commoditized. Creation of new ideas will be the LAST thing to get commoditized. But this is exactly where America excels. No nation on Earth fosters creativity as well as the USA. And for this reason, I'm not so sure that America's period of dramatic success is over. The more science and technology accelerate, the more critical creativity becomes -- and, lame as American culture and institutions are, they seem better than most alternatives at fostering wide-ranging creativity. (The only cultures I've known that seemed maybe more creativity-friendly were Australia, New Zealand and Hungary. But those are small places, population-wise.)

There is loads of creativity in China, for instance, on a personal level. Very creative people. But I'm not sure the culture fosters creativity in the way that US culture does. Oriental culture seems to favor obedience a lot more than US culture, and creativity is often not compatible with obedience.... The US is probably the most anarchic major developed country -- which has its downsides, especially for those below the poverty line in the US -- but, it seems that anarchy and creativity are inextricably entwined.

If China evolves a culture of creativity, then Hugo will be proved right and this will become the Chinese century ... and maybe the Singularity will get launched in China (hey, maybe it will get launched there anyway via Hugo's and my collaboration!!!)..... But that's a big "if", I suppose. Yet one feature of Chinese history is its tendency toward sudden, radical changes of one sort or another. Time will tell.

Anyway I look forward to returning to Xiamen and other parts of China when my schedule permits (hopefully for a couple months next summer, and a couple weeks in the fall or winter) ... there is a definite energy there that I don't find in developed countries these days, nor in 3rd world countries ... there is a feeling of "waking up" and progress that is exciting...

And, more importantly, there is a possibility of creating a thinking machine and doing other amazing technology projects there more rapidly than in other parts of the world, due to the availability of brilliant scientists and engineers at a relatively low cost (esp. outside the tier 1 cities). Whether or not China develops a culture of creativity allowing it to "own" the next century, there are loads of opportunities for international collaboration ... like what Hugo and I are trying to set up....

But anyway. Enough rambling. I've been sleep-deprived since returning from China, due to jet lag issues ... tonight I'll go to sleep "early" (i.e. maybe by 1AM) and hopefully actually get a full night of sleep.. (yah right...)

Eureeka!! -- The Underlying Logic Unifying Quantum Theory and General Relativity, Revealed Over a Plate of Sour Fish Consumed Over South China

Eureeka!! -- The Underlying Logic Unifying Quantum Theory and General Relativity, Revealed Over a Plate of Sour Fish Consumed Over South China; Plus Long Digressions on Mark Twain, the Pathetic Woes of Middle Age and the Good Old Mongolian Skin-Peeler

[I wrote this post 2 weeks ago, but didn't get around to posting it due to being in China, with a slow Net connection..]

En route from Seoul to Hong Kong, exhausted from a 5 hour night's sleep following a 4 hour night's sleep, over-jazzed by too much strong coffee (which I rarely drink), stomach-sickened by ordering and consuming random dishes in a Korean restaurant via pointing at random hieroglyphs on the menu and hoping vainly for the best ... head full of Mark Twain's wacky biography which I just finished ... irrationally nervous due to having left my oldest son in Japan to tour around on his own for a week before his Japanese class in Kanazawa starts (yes, he's mature enough to handle himself ... and Japan is a damnably safe place, aside from the risk of spending all your money ... but even I the ultimate anarchist parent can't help a bit of worry) ... dulled almost but not quite to a stupor by a relentless series of software technology oriented business meetings (all with wonderful and interesting people, but still, there's only so much meeting I can take) ... I picked up Lee Smolin's book on quantum gravity, which I bought for my physician-cum-maverick-physics-theorist father-in-law years ago but never read myself ... and while reading a totally irrelevant passage and eating the oddly sour fish that passes for food on Korean Air, some very simple and obvious ideas popped into my mind, and I realized to my surprise that, via converging together several streams that have been tumbling through my head for years, I'd happened upon what appeared to be the correct probabilistic logic of unified quantum gravity.

I'm eager to write up this logic in a paper, but, I've promised myself not to undertake anything serious -- except tasks critical for Novamente as a business, or in order to fulfill obligations already incurred -- until the OpenCog Prime (http://www.opencog.org) wiki pages are done (maybe another 20-30 hours of work, but hours for concentrated writing/editing work are very hard to come by these days due to the combination of business obligations and ongoing research projects needing supervision and/or feedback).... But I'll indulge myself in a brief blog post on the topic as a stopgap - partly to ensure the idea doesn't escape from my mind tonight when I finally slip into the deep sleep my body's been craving for 72 hours or so....

Twain's bio was a fascinating read, by the way. Three things among many others struck me, viewing his life-story from a selfish view in terms of its potential lessons for my own life. One is the way he spent a load of his time on stuff other than writing -- business of various sorts, as well as lecturing, traveling and so forth. But these "distractions" didn't seem to detract from his productivity as a writer as much as I would have thought -- they filled his head with stimulation and ideas, and no doubt made his writing more interesting than if he'd just sat home writing all day. Second is the romance he found in business pursuits ... which reminded me a bit of Rimbaud, who gave up poetry as a very young man after too few years as a writer, and wasted his twenties chasing African gold, ultimately dying from poisoning attained via wearing gold under his undies to hide it from thieves. Rimbaud, due to his premature death among other issues, failed to transform his digressive life experiences into art. I can see in my own psychology the excitement that the business world held for these people: it does stimulate parts of the mind that creative art and science don't touch. Finally I'm struck by the amount of real trash literature Twain produced. I'm reminded of Danilo Kis's (a truly great Serbian writer -- thx to Predrag Janicic for waking me up to him) comment that he didn't write his complete works, only his selected works. Twain was not like that. Twain's best work was awesome, his worst work was terrible. He could have omitted a good 50% of his production and his legacy would be greater not less. Philip K. Dick had the same property: there's Ubik ... and then there's Dr. Futurity.... The lesson for me is, I suppose, not to worry too much about spending time on apparently digressive pursuits (like writing this blog post, um) -- so long as they're feeding the creative engine one way or another -- and given the limited time I have for creative pursuits, to try hard to be more like Kis than Twain or Dick, and filter out crappier works before I take the time to produce them.

Another striking thing about Twain was the way he foresaw the power of machinery to alleviate human suffering. A lesson that seems obvious these days but was surprisingly poorly understood in his times, even though the industrial revolution was in full swing and new mechanical inventions of all sorts were pouring out of human minds at an amazing rate. If you haven't read it, his Connecticut Yankee in King Arthur's Court -- arguably the first American SF novel -- is a hilarious and deeply insightful premonition of the promise and peril of advanced technology. As a time travel fable, it's got Back to the Future beat by a long shot, without need of paradoxical absurdities beyond those intrinsic to human nature.

And now... what about quantum gravity...

Three threads need to be drawn together, into a single mathematical formalism. But there really seems no obstacle to doing so. (Except time of course, which is a distressingly rare commodity for me these days. Must confess to a bit of jealousy of my son as he wandered off from Tokyo to Kyoto, with no specific plan for spending his days, other than to amuse himself. I never intended to accumulate so many obligations -- keep companies running, organize conferences, pay other peoples' college tuition, close a mortgage, finish a book, pay child support each month, drive the kids to and from school, blah blah blah blah blah ... I once actually thought I'd live the life of the "free and easy wanderer" from the Chuang Tzu (on my mind as the flight I'm on approaches China), or maybe of Paul Erdos who freeloaded off one friend after another as he spent his life journeying around the world doing mathematics and taking drugs ... I never envisioned taking on all this responsibility for other people (kids, wife, ex-wife) and organizations (companies, non-profits, egads!) ... yet it's all wonderful, interesting stuff ... people and ideas I really care about ... so I'd really be an ass to complain ... it's a fantastic time to be alive ... yet not quite as fantastic as a few decades hence will likely be, when minds will be far more fully liberated from the horrifying/stultifying constraints of legacy human physiology ... but, well, anyway...)

Quantum gravity!

Thread one was invented by Saul Youssef, and I've written about it before. Check out the lovely bibliography he's assembled at 

http://physics.bu.edu/~youssef/quantum/quantum_refs.html

Brilliant, brilliant man. A hero of our time! Someone give that man a muffin!!

The observation here is that if you're willing to take the step of assuming probabilities are complex rather than real numbers, the basic rules of quantum theory fall right out. This is one of those things that seems shocking and weird at first, and then seems tremendously obvious after you read through the math. Three cheers for Saul Youssef!!

Thread two is something I came up with a couple years ago, and wrote up in a paper which I'm in the middle of submitting for publication. I sent the paper to a journal and they sent it back asking me to provide names and mailing addresses of eight referees able to review the paper. I've been lagging on that task along with a huge amount of other stupid paperwork that's accumulated during the last N years. I guess the editor couldn't think of anyone to send it to. The idea, anyway, is infinite order probability.

An ordinary probability is a probability of an event. A probability distribution is a function that assigns a probability to each one of a set of mutually exclusive outcomes of some event (the different values assigned to different exclusive outcomes must sum to one). A second-order probability is a probability distribution over probability distributions ... it's a function that assigns a probability to each one of a set of probability distributions. A third-order probability ... etc.

An infinite-order probability is a function that assigns a probability to each one of a set of infinite-order probability distributions. Sounds odd, but it's a mathematically consistent idea, as I showed in my paper. I also showed that these oddball entities are closely related to some much more familiar and intuitive mathematical entities, Markov matrices.

The third thread is causal networks. A foundational notion in general relativity is causality. The causal network of events, in relativity, tells you for any pair (A, B) of events, which ones have the property that A is causal for B. This has to do with the finitude of the speed of light: if A and B are too close in time and too distant in space, there may be no way for A and B to causally affect each other.

If A and B are not causally related, there may still be some event C so that C is causal for both A and B. In that case we may say that, probabilistically speaking, A and B are independent conditional on C. That is,

P(A & B | C) = P(A | C) P(B |C)

The causal network gives us a set of independence assumptions on the space of events.

General relativity is in essence a dynamic on causal networks: it tells you how a causal network at one time (plus some extra information) gives rise to a different, related causal network at a subsequent time.

Finally let's reflect on what Smolin (see Three Roads to Quantum Gravity) calls the "strong holographic principle." His reasoning for this principle is subtle and involves the Bekenstein bound and related results, which state that all the information about the interior of some physical region, may actually be thought of as being contained on the surface of that regions. (He explains this better than I could, so I'll just refer you to his book.)

What the principle says is: a la Nietzsche, there are only surfaces. Re-read Nietzsche's Twilight of the Idols and you'll see that he presaged quantum gravity here, in a similar way to how he presaged quantum theory proper in his vision (in The Will to Power) of the universe as composed of a dancing swarm of discrete interacting quanta. Kant posited phenomena and noumena, Nietzsche saw only noumena. Smolin also. Smolin views the universe as a collection of surfaces, each one defined as a relationship among other surfaces. Put in words like this, it sounds mystical and fuzzy, but there's math to back it up -- the words just hint at the mathematical reality.

But is each of these Smolin surfaces definitively known? No. Each one is probabilistically known. And if each of these surfaces is to be thought of as a relationship between other surfaces, then this means each of these surfaces is most directly modeled as a hyperset (see my prior blog posts on these mathematical constructs). (This is not how Smolin models things mathematically, but I doubt he'd be opposed, as he's used equally recondite math structures such as topoi.) So these surfaces should be modeled as probabilistic hypersets -- aka infinite-order probability distributions.

But what kinds of probabilities should be involved in these distributions? Clearly, Youssef has taught us, these should be complex probability distros -- or in my variation, infinite-order complex probability distributions.

The inescapable conclusion is: The physical universe is a dynamically evolving causal network defined on an infinite-order complex probability distribution.

You read it first here, folks.. ;-O

Or, to put it a bit more conservatively: A useful, perhaps critical language for modeling quantum gravity phenomena is the logic of causal networks on infinite-order complex probability distributions.

There are fun connections here with the psychology of self-awareness and free will, as I've discussed in a couple previous blog posts (follow the links). According to those blog posts, a good way to model reflective awareness would be using infinite-order real probability distributions; and a good way to model will would be using causal networks on these distributions. What quantum theory introduces is the complex-number probability aspect, which makes everything counterintuitive and weird.

I hope I can really find time -- amidst the manifold obligations of middle age plus the not incidental life-task of creating superhuman AI, plus other distractions like bioinformatics and fiction and music and what-not ... and family and the occasional personal entertainment -- to write these ideas up carefully, because I really do think they have deep potential.

There seem to be more connections lurking here: the logic of causal networks seems somehow inescapably tied up with Clifford algebras, providing a tie-in with my algebra of multi-boundary forms (my only publication in a physics journal so far, but it's really a math paper). Presumably one can go from causality somehow straight to discrete Clifford algebras using some kind of axiomatic derivation, and from there to the various beautiful algebraic symmetries underlying modern physics ... Gell-Mann's "Eightfold Path" and its kin ... but anyway, the flight's about to land and the stewardess wants me to put away my laptop, so the blog post is gonna end .. I'll post it online when I get back to the hotel assuming there's functional internet there ... the inimitable Yan King Yin (famous on various AI email lists) is picking me up at the airport and I'm curious to meet him, although I'm so worn out I'm not sure I'll be lucid enough to milk the occasion's potential for lively AI discussions....

How about the bloody Yverse (see previous blog post on this)? Each Smolin surface ... each relation in the network of interdefining hyperrelations ... defines its own multiverse: a quantum multiverse relative to its own perspective. The network of surfaces (aka relationships) is then a Yverse. QED.

Another day, another dozen digressions ... it's SOOOOO tempting to take a few days and formalize the logic of causal nets over complex infinite-order distros, but instead (inbetween biz meetings and AI research meetings and conference speeches and meals with AI colleagues) I'll spend my "spare" hours in the next few weeks on the OpenCogPrime documents ... a very tedious matter of taking about 50 wiki pages from the Novamente wiki site and editing them down into OpenCogPrime rather than Novamente Cognition Engine pages... yecch...

(Any wealthy patrons out there want to hire me a secretary, a housecleaner and a scientific assistant? I can't promise the Singularity will be accelerated by a few years but it's a definite possibility. For sure a lot more fascinating math, art and science would be generated were I to be thus endowed. (And, getting back to Twain, I wonder what additional great works he would have produced if some of his rich friends had decided to fund him, sparing him the financial anxiety that led him to waste years of his life on various harebrained business schemes. Yeah, they provided grist for his creative mill ... but there's such a thing as too much grist and not enough time to mill.) But I can't complain too much (er, OK, wait, I guess I am...) ... whenever I get TOO frustrated at the realization that 50% of my really good ideas and creations will remain forever unarticulated or un-worked-out-in-detail because I've failed to be born rich or become rich (so far), I remind myself of my favorite Haruki Murakami character, the Mongolian Skin-Peeler ... a World War II torture artist who tortured Chinese prisoners of war by slowly peeling their skin off ... as I see Ulaanbaator on the video screen of the plane as it approaches Hong Kong (haha, I'm a bad boy and failed to shut off my laptop when instructed ... how very non-Oriental of me!!) it's hard for me not to feel thankful that I'm not one of his victims ... I've got my epidermis attached to my dermis, woo hoo! ... and I at least have time to work out a nontrivial percentage of the cool ideas and creations that course through my overheated brain...)

(While you're at it, imaginary patron, recruiting Novamente LLC a CEO with lots of game or virtual world industry experience would be nice. I think I'm doing a decent job as CEO, with help from Bruce and Cassio and Wendy and my other wonderful colleagues, but it would be nice to have a sufficiently complete management team that I could spend 80% of my Novamente-time on science rather than business. And you may as well recruit us a kick-ass project manager too, so Cassio can help me out with research and retire from project management. (Ok, dream on, Ben.... And remember the Mongolian Skin-Peeler....). And while you're at it, throw in maybe $1M per year so that I can actually fund a team of kick-ass programmers to build a thinking machine ... in case you haven't heard I have a pretty detailed and well-argued design for one, but it's getting built bloody fucking slowly due to lack of funding, and because it's not the sort of thing where partial progress yields exciting incremental results, any more than building 30% of a human brain would yield a 30% functional human... but I dididididididigress ;_)

(I think my wife is really, really tired of hearing about the Mongolian Skin-Peeler. He seems to occupy an unjustifiably prominent role in my emotional topography. Read The Wind-Up Bird Chronicle.)

Mark Twain, I add, would have been a hell of a blogger; far more entertaining than me. He wrote a dozen letters each day back then in the pre-digital dark ages.

Time to get off the plane.

Open-Source Robots + Robot Simulators + Virtual Worlds + AI = ???

I’ve been reading up on the iCub open-source humanoid robot lately, and I think it’s pretty exciting. Given what open source has done for Web browsers, bioinformatics tools and other sorts of software, the possibility of harnessing the same development methodology for robot hardware and software development seems almost irresistably exciting.

I’m no roboticist, but I do know something about the AI software that robots need to understand the world and act in it – and I’ve been doing a lot of work lately on the use of AI to control simulated agents in virtual worlds. In this vein, this blog entry contains some follow-up thoughts about the possibility of building connections between the iCub and various other relevant open-source software systems relevant to AI and virtual worlds.

For starters: What if someone made a detailed simulation of iCub in Gazebo, an open-source 3D robot simulation platform? Then folks around the world could experiment with iCub without even building a robot, simply via writing software and experimenting with the simulation. Experiments with other robots and Gazebo have shown that the simulation generally agrees very closely with real-world robotic experience.

And what if someone integrated Gazebo with OpenSim, the up-and-coming open-source virtual-world platform (which uses an improved version of Second Life’s user interface, but features a more sophisticatedly architected and flexible back end, and best of all it’s free)?

Furthermore, work is underway to integrate OpenSim with OpenCog, an open-source AI platform aimed at advanced machine cognition (yes, I’m one of the organizers of OpenCog); and OpenSim could similarly be integrated with OpenCyc, OpenNARS, and a host of other existing open-source AI platforms. Throngs of diversely customized, simulated iCubs controlled by various AI algorithms could mill around OpenSim, interacting with human-controlled avatars in the simulated world, learning and sharing their knowledge with each other. The behaviors and knowledge learned by the robots in the virtual world could then be transferred immediately back to their physically embodied brethren.

What stands between us and this vision is “just” some software integration work ... but of course, this kind of work isn’t easy and takes time and expertise. For various economic and cultural reasons, this sort of work has not been favored by any of the world’s major R&D funding sources – but the open-source approach seems to have increasingly high odds of getting it done. It seems at least plausible that iCub won’t go the way of OpenPINO and other prior attempts at open-source robotics, and will instead combine with other open-source initiatives to form a key part of a broadly-accepted, dynamically evolving platform for exploring physical and virtual humanoid robotics.

Artificial Wisdom (... episodic memory, general intelligence, the Tao of John Coltrane, and so forth)

Every now and then, someone suggests to me that, alongside the pursuit of Artificial Intelligence, we should also be pursuing "Artificial Wisdom."

I always figured the "artificial wisdom" idea was probably just a bunch of useless English-language wordplay -- but one night last week, while watching Idiocracy with the kids for the second time (great movie exploring a non-Singularity-based future by the way ... highly recommend it!), I spent a while surfing the Web on my laptop refreshing my memory on how others have construed the "wisdom" concept and musing on what it might mean for AI.

Surprisingly enough, this led in some moderately interesting directions -- nothing revolutionary, but enough to justify the couple hours spent musing about it (and another 90 minutes or so synthesizing and writing up my glorious conclusions).

My main conclusion was a perspective in which wisdom is viewed as one of three core aspects of intelligence, associated with three distinct types of memory:

• cleverness, associated with declarative memory (and the ability to manipulate abstract, certain or uncertain declarative knowledge)
• skillfulness, associated with procedural memory (and the ability to effectively learn and adapt new procedures based on experience)
• wisdom, associated with episodic memory (and insightful drawing of large-scale conclusions therefrom)

This being a blog post, though, rather than just presenting my conclusion, I'll start out by recounting some of the winding and mostly irrelevant path that led me there ;-)

Classical Conceptions of Wisdom

I started out with the dictionary, and as usual found it close to useless....

A typical dictionary definition of "wisdom," which is not a heck of a lot of help, is from Wiktionary, which tells us that

wisdom (plural wisdoms)

means

1. An element of personal character that enables one to distinguish the wise from the unwise.
2. A piece of wise advice.
3. The discretionary use of knowledge for the greatest good.
4. The ability to apply relevant knowledge in an insightful way, especially to different situations from that in which the knowledge was gained.
5. The ability to make a decision based on the combination of knowledge, experience, and intuitive understanding.
6. (theology) The ability to know and apply spiritual truths.

and furthermore that

wise

means

Showing good judgement or the benefit of experience.

Hoo haw.

These definitions don't give us any particularly interesting way of distinguishing "wisdom" from "intelligence." Essentially they define wisdom as either intelligence, spiritual insight, or the application of intelligence for ethical ends. Nothing new here.

Wikipedia is slightly more useful (but only slightly). Firstly it notes that

A standard philosophical, (philos-sophia: literally "lover of wisdom"), definition says that wisdom consists of making the best use of available knowledge.

It then notes some psychological research demonstrating that in popular culture, wisdom is considered as different from intelligence. Psychological researchers are quoted as saying that though "there is an overlap of the implicit theory of wisdom with intelligence, perceptiveness, spirituality and shrewdness, it is evident that wisdom is a distinct term and not a composite of other terms."

More interestingly, Wikipedia notes, Erik Erikson and other psychologists have argued that it is, in large part, the imminence of death that gives older human beings wisdom.

The knowledge of imminent death is seen as focusing the mind on concerns beyond its own individual well-being and survival, thus inducing a broader scope of understanding and an identification with the world at large, which are associated with the concept of wisdom.

This is interesting from a transhumanist perspective in that it suggests that the death of death would be the death of wisdom! I have seen some evidence for that in the incredible, shallow-minded selfishness of a certain subset of the transhumanist community -- people who are dead-set on having their own selves live forever, without any real thought as to why this might be valuable or what this might mean in a larger perspective. But of course, I don't really think death is the only or ultimate source of wisdom, though in a human context I can believe it's one of the main forces nudging us toward wisdom.

Paul Graham on Wisdom

One of the more interesting theories of wisdom I've run across (I found it a while ago for some random reason I've forgotten, and dug it up again last week) came from a contemporary blogger, Paul Graham:

http://paulgraham.com/wisdom.html

who distinguishes wisdom from intelligence in the following way:


"Wise" and "smart" are both ways of saying someone knows what to do. The difference is that "wise" means one has a high average outcome across all situations, and "smart" means one does spectacularly well in a few.

This explanation also suggests why wisdom is such an elusive concept: there's no such thing. "Wise" means something—that one is on average good at making the right choice. But giving the name "wisdom" to the supposed quality that enables one to do that doesn't mean such a thing exists. To the extent "wisdom" means anything, it refers to a grab-bag of qualities as various as self-discipline, experience, and empathy

Graham considers wisdom as partly a kind of de-biasing and cleansing of the mind, a notion that has some resonance with the modern notion of "Bayesian calibration" of the mind:

Recipes for wisdom, particularly ancient ones, tend to have a remedial character. To achieve wisdom one must cut away all the debris that fills one's head on emergence from childhood, leaving only the important stuff. Both self-control and experience have this effect: to eliminate the random biases that come from your own nature and from the circumstances of your upbringing respectively. That's not all wisdom is, but it's a large part of it. Much of what's in the sage's head is also in the head of every twelve year old. The difference is that in the head of the twelve year old it's mixed together with a lot of random junk.

Provocatively, Graham also posits that intelligence is quite different from wisdom, in that it has to do with accentuating rather than avoiding biases:

The path to intelligence seems to be through working on hard problems. You develop intelligence as you might develop muscles, through exercise. But there can't be too much compulsion here. No amount of discipline can replace genuine curiosity. So cultivating intelligence seems to be a matter of identifying some bias in one's character -— some tendency to be interested in certain types of things—- and nurturing it. Instead of obliterating your idiosyncrasies in an effort to make yourself a neutral vessel for the truth, you select one and try to grow it from a seedling into a tree.

To avoid confusion, from here on I'll sometimes refer to Graham's interpretation of these concepts as Graham-style wisdom and Graham-style intelligence, respectively.

There is an unclarity in Graham's essay as to the extent to which he thinks the kind of focusing and bias-accentuation that's part of Graham-style intelligence has to involve irrationality. My own view is that Graham-style intelligence definitely does NOT require an individual to be irrational, in the sense of making suboptimal judgments about a particular problem given the resources devoted to thinking about the problem. However, a finite system in a complex environment is always going to be irrational to some measure, due to not having enough resources to make a fully analysis of any complex situation. To the extent that Graham-style intelligence involves heavy focus on some particular set of topic areas, it's going to drain resources from other areas, thus making the mind less intelligent regarding these other areas.

So, in Graham's view, intelligence has to do with focusing loads of resources on processing in a handful of narrow domains that match one's innate biases, whereas wisdom has to do with evenly distributing processing across all the different domains in one's environment.

Along these lines Graham also notes (correctly, I think) that:

The wise are all much alike in their wisdom, but very smart people tend to be smart in distinctive ways.

As Graham conceives it, wisdom is basically equivalent to general intelligence: it's intelligence averaged across a variety of situations. In mathematics there exist various sorts of averages, some of which weight extreme values more heavily than others (these are p'th power averages). Graham's view would be that "wisdom" and "intelligence" are both estimates of general intelligence (defined as intelligence averaged over different domains/tasks), but with different sorts of averaging: in the case of intelligence, an averaging that pays especial attention to extremes (say a p-power average with p=5, or whatever); and in the case of wisdom, a more typical arithmetic averaging.

This is all sort of nice, but (as will become clear as the essay unfolds) I don't really think it gets at the crux of the matter.


Wisdom Goes Beyond the Individual

Another interesting perspective (that I also think doesn't get at the crux of the matter) is given in the paper "Meaning generation and artificial wisdom" with abstract

We propose an interpretation of wisdom in terms of meaning generation in social groups. Sapient agents are able to generate useful meanings for other agents beyond their own capability of generation of self-meanings. This makes sapient agents specially valuable entities in agent societies because they provide interagent reliable third-person meaning generation that provides some functional redundancy that contributes to enhance individual and social robustness and global performance.

Here wisdom is identified with the ability to generate meaning in the social group, going beyond meaning that is perceptible by the individual doing the meaning-generating. This harks back to Erikson's understanding of wisdom as related to identification with the world at large, beyond the mind/body.

This view also reminds me vaguely of Aldous Huxley's Perennial Philosophy, an attempt to distill the "wisdom teachings" of all the world's religions. In the Perennial Philosophy, wisdom teaches that the individual self is an illusion and all of us are one with the universe (and yet in a sense still distinct and individual.)

Mulling over all this, none of it really satisfied me. Of course, a folks concept like "wisdom" can't be expected to have a crisp and sensible formalistic definition ... but it still seemed to me that all the attempts at systematization and formalization I'd read about were missing some really essential aspects of the folk concept.

Wisdom, Cleverness and Skillfulness

And so, I came up with a totally different idea....

After a fair bit of musing, my mind kept drifting to the familiar distinction between declarative, procedural and episodic memory (drawn from textbook cognitive psych).

Remember:

• Declarative knowledge = knowledge of facts, conjectures, hypotheses (abstract or concrete)
• Procedural knowledge = knowledge of how to do things (could be physical, mental, social, etc.)
• Episodic knowledge = knowledge of stories that have occurred in the history of intelligent beings (oneself, others one knows, others one has heard about,...)

One interesting thought that popped into my head is: The concept of wisdom, in its folk-psychology sense, has a lot to do with the ability to solve problems that are heavily dependent on context, using intuition that's based on large-scale analysis of one's episodic-memory store.

Or, less geekily: Wisdom consists of making intelligent use of experience.

A subtlety here is that this need not be one's own experience. Direct experience may be the best way to acquire wisdom (and surely this is part of the reason that wisdom is commonly associated with age) but some rare folks are remarkably gifted at absorbing wisdom from the experience of others -- absorbed via observation, via reading, or conversation, or whatever.

More broadly, this train of thought leads me to a sort of fundamental trinity of aspects of intelligence: cleverness, skillfulness and wisdom.

There's cleverness, which is the ability to appropriately manipulate, create and absorb declarative knowledge toward one's goals. This declarative knowledge may be abstract, or it may be concrete facts. Declarative knowledge is largely symbolic in nature, and cleverness is largely founded on adeptness at symbol-manipulation.

There's skillfulness, which is the ability to effectively do stuff in service of one's goals. This covers physical skills but also highly abstract mental skills like writing an essay, proving a theorem, or closing a business deal.

In some domains skillfulness can exist in the total absence of cleverness. The vast majority of shred metal guitarists would seem to fit in this category (to choose a somewhat random example based on what's playing in my headphones at the moment). These guys are so damn skilled, yet there's not much adept manipulation of meaning in their solos, or compositions. Compare the typical shred guitarist to Yngwie Malmsteen or Buckethead, who are also massively skilled (and in similar ways) -- but who are also highly clever in their symbolic manipulation of the abstract patterns characterizing the concrete sonic forms they're so skilled at producing.

In other domains, it's really hard for cleverness and skillfulness to emerge in any way except exquisitely intercombined. Mathematics is an example. Procedural knowledge at doing proofs is needed for fully understanding complex proofs -- because so many steps are left out in proofs as typically written down, if you don't know how to do proofs, you won't be able to fill in all the gaps in your head when you read a proof, so you'll never get more than a general understanding. On the other hand, it's even more obvious that deep declarative understanding and manipulation-ability regarding mathematical content is necessary to do mathematical proofs. Math is a domain where procedural and declarative intelligence have got to work in extremely tight synergy.

Finally, there's wisdom, which as I'm conceiving it here is the ability to intelligently draw conclusions from a vast repository of data regarding specific situations.

Human minds tend to organize data regarding specific situations using story-like, "narrative" structure, so that in human practice, wisdom often takes the form of the ability to mine appropriate abstract patterns from a vast pool of remembered stories.

Of course, the operation of human episodic memory is largely constructive -- we don't actually grab experiential data out of some sort of neurological database; rather, we synthesize stories from fragmentary images, stories, and such. Wisdom is about synthesizing appropriate stories from large databases of partially-remembered, ambiguous, fractional stories -- and then, as appropriate, using these stories to guide the creation of declarative or procedural knowledge.

In mathematics, wisdom is closely related to what's called "mathematical maturity" ... the general sense of how mathematics is done. Mathematical maturity guides the mind to interesting problems and interesting concepts ... and helps you choose an overall proof strategy (whereas it's cleverness and skillfulness that help you carry out the proof).

The transition from {cleverness + skillfulness} to wisdom in music is epitomized to me by the mid-to-late John Coltrane ... the Coltrane of "My Favorite Things" and "A Love Supreme." These are the solos of a man who has listened so much and played so much that he's disassembled thousands of different musical narratives and reassembled them to tell different kinds of stories, like no one ever told before. So much richer than the merely clever, skillful and emotionally moving solos of the early Coltrane. Certain works of great art manage to be intensely personal and dramatically universal at the same time, and
this often results from wisdom in the sense I'm defining it here.

Note that a mature mathematician or a world-changing jazz soloist need not be "wise" in the sense of a Taoist sage. The classical conception of wisdom has to do with making intelligent judgments based on large stores of experience in everyday human life. In the old days this was pretty much the only experience there was -- everyday human life plus various shamanic and psychedelic experiences.... But now the human world has become far more specialized, and it's possible to have a specialized wisdom, because it's possible to have a huge and rich store of episodic knowledge that's restricted to some special domain, like music or mathematics, or even a sufficiently complex game like Go or chess.

This vision of wisdom would seem to contradict Graham's, cited above -- he views wisdom as related to the ability to achieve goals over a broad variety of domains, in contract to intelligence which he conceives as a more narrowly domain-specialized intelligence.

But I don't think the contradiction is total.

I think that within a sufficiently rich and complex domain, one requires wisdom as I've defined it in order to achieve a really high level of intelligence. Learning skills and manipulating symbols is not enough. Direct and intelligent mining of massive experience-stores is needed.

I also think that wisdom, even if achieved initially and primarily within a certain domain, has a striking power to transcend domains. There are a lot of universal patterns among large stores of stories, no matter what the domain.

But even if the wisdom achieved by a great mathematician or chess player or jazz soloist helps that person to intuitively understand the way things work in other domains, this won't necessarily lead them to practical greatness in these other domains -- great achievement seems to require a synthesis of wisdom with either cleverness or skillfulness, and in some domains (like math or jazz improvisation) all three.

Defined-Problem versus Contextual Intelligence

Next, what does all this have to do with artificial intelligence?

One of the lessons learned in the last few decades of AI practice is that there is a pretty big difference between:

1. Defined-problem intelligence: Problem-solving that occurs "after a crisply-defined problem statement has been identified", versus
2. Contextual intelligence: problem-solving that is mainly concerned with interpreting general goals in the context of a complex situation, and, "figuring out what the context-specific problem is, in the first place" -- i.e. figuring out what crisply-defined problem, if solved in the relevant context, is likely to work toward the general goals at hand

I think this might be a more useful and more precise distinction than the "narrow AI" versus "general AI" distinction that I've often made before. It's ultimately getting at the same thing, but it's putting the point in a better way, I think.

What's narrow about "narrow AI" systems like chess-playing programs and medical diagnostic expert systems isn't merely that they're focused on specific, narrow domains. It's the fact that they operate based on defined-problem intelligence. It happens, though, that in some sufficiently specialized domains, defined-problem intelligence is enough to yield ass-kicking performance. In other domains it's not -- because in these other domains, figuring out what the problem is, is basically the problem.

I suggest that defined-problem intelligence is focused on declarative and procedural knowledge: i.e. it consists of cleverness or skillfulness or some combination thereof.

Logical reasoning systems, for example, are focused on declarative knowledge, and possess in some cases great facility at manipulating declarative knowledge.

Evolutionary learning systems and neural nets, on the other hand, are mainly focused on procedural knowledge -- on learning how to do stuff, without need for symbolic representations or symbol manipulations.

On the other hand: Contextual intelligence, I suggest, is a matter of knowing how to synthesize declarative and procedural knowledge, that representing problem-statements and problem-solutions, out of the combination of general goals and real-world situations.

I suggest that powerful contextual intelligence always relies upon powerful use of episodic memory, and associated mechanisms for storing, accessing, manipulating and analyzing sets of stories.

Or, briefly getting less geeky again: contextual intelligence requires wisdom.

Not at the level of the Taoist sage, John Coltrane or Riemann ... but at a way higher level than possessed by any currently operational AI system.

Note that defined-problem intelligence may sometimes draw on a wide body of background knowledge -- but it uses this background knowledge in a manner constrained by certain well-defined declarative propositions, or practical constraints on procedure-learning. It uses the background knowledge in a manner that doesn't require the background knowledge to be organized or accessed episodically -- rather, it uses background knowledge as a set of declarative facts, or data items, or constraints on actions, or procedures for doing specific things in specific types of situations.

"How to make a lot of money in Russia" is a problem that requires intense contextual as well as defined-problem intelligence. Whereas, "how to make a lot of money by trading oil futures on the Russian stock exchange" is more heavily weighted toward calculational intelligence, though it could be approached in a contextual-intelligence-heavy manner as well.

For instance, in the domain of bioinformatics, figuring out a rule that can diagnose a disease based on a gene expression microarray dataset, is a well-defined problem -- a problem that can be solved via focusing strictly on a small set of reasonably well-encapsulated information items. Declarative and/or procedural focused AI works well here ... much better than human intelligence.

On the other hand, figuring out which datasets are likely to be reliable, and figuring out how to normalize these datasets in a reasonable way based on the experimental apparatus described in the associated research paper, are tasks that require much more understanding of context, more milking of subtle patterns in episodic memory. I.e., I'm suggesting, more wisdom.

In the current practice of bioinformatic data analysis, human wisdom is needed to craft well-defined problems to feed into the superior (in this domain) declarative and procedural intelligence of narrow-AI bioinformatic data-analysis systems like the ones we've created at Biomind LLC.

Doing Time in the Universal Mind

Getting back to some of the ideas introduced at the start of this essay ... it seems all this ties in moderately closely with Erikson's definition and the Perennial Philosophy definition of "wisdom."

These definitions conceive wisdom as related to an understanding of life situations in a broader context than that of the individual body and mind. Wisdom as these thinkers conceive it, is a higher level of contextual intelligence than average humans display -- an ability to conceive daily situations in a broader-than-usual context.

This corresponds, really, to relying on a kind of collective episodic memory store, rather than just the episodic memory store corresponding to one's own life. By the time one is old, one is reviewing a longer life, and reviewing the past and future lives of one's children and grandchildren, and thinking about the whole scope of stories all these people may be involved in. A much richer contextuality.

Another ingredient of the Perennial Philosophy notion of wisdom is self-understanding, and I think that ties in here very closely too. One's own self is always part of the context, and to carry out really deep contextual understanding or problem-solving, one needs to appreciate how one's own history, knowledge and biases are affecting the situation and affecting one's own judgments. Powerful contextual intelligence -- unlike powerful calculational intelligence -- requires deep and broad self-understanding.

Wrapping Up

Sooo ... if we conceive wisdom as contextual intelligence powered by rich analysis of episodic memory, then it is clear that wisdom is a key aspect of general intelligence -- and is precisely the aspect that the AI research field has most abjectly ignored to date.

And it is also clear that ethical judgment is richly bound up with wisdom, as here conceived. Ethical judgment, in real life, is all about contextual understanding. It's not about following logical principles of ethics -- even when such principles are agreed-upon, real-life application always comes down to tricky context-specific intuitive judgments. Which comes down to understanding a vast pool of different situations, different episodes, that have existed in the lives of different human being and groups.

Defined-problem intelligence can be useful for ethical judgments. For instance in cases where scarce resources need to be divided fairly among a large number of parties with complex interrelationships and constraints, one has a well-defined problem of figuring out the optimally ethical balance, or a reasonable approximation thereof. But this actually seems an exceptional case, and the default case of ethical judgment seems to be to rely much more heavily on contextual than defined-problem intelligence.

Just to be clear: I'm not claiming that the conception of "wisdom" I've outlined here thoroughly captures all aspects of the natural-language/folk-psychology term "wisdom." Like "mind", "intelligence" and so forth, "wisdom" is a fuzzy term that amalgamates various different overlapping meanings ... it's not the kind of thing that CAN be crisply defined and analyzed once and for all.

What I hope to have done is to extract from the folks concept of wisdom some more precise, interesting and productive ideas, that closely relate to this folk concept but don't pretend to exhaust it.

In short...

• General intelligence = defined-problem intelligence + contextual (problem-defining) intelligence
• Calculational intelligence = cleverness (declarative intelligence) + skillfulness (procedural intelligence)
• Contextual intelligence = in the human context, highly reliant on large-scale analysis of episodic memory
• Wisdom = interestingly interpreted as contextual intelligence
• Ethics = heavily reliant on wisdom

In this view, not surprisingly, the pursuit of Artificial Wisdom emerges as a subtask of the pursuit of Artificial General Intelligence. But what's interesting is it emerges as a complementary subtask to the one that most of the AI community is working on at the moment -- narrow-AI, or artificial defined-problem intelligence.

There is a bit of work in the AI community on narrative and story understanding. But most of this work seems, well, overly artificial. It has to do with formalistic systems for representing story structure. That is just not how we do things, in our human minds, and I suspect it's not an effective path at all.

I don't at the moment know any way to give an AGI system a rich understanding of episodes in the world than to actually embed it in the world and let is learn via experiencing. Virtual worlds may be a great start, given the amount of rich social interaction now occurring therein.

Thus I conclude that an excessive focus on narrow-AI research is, well, un-wise ;-)

And physically or virtually embodied AGI may potentially be a wise approach...

And I return again to the apparent wisdom of integrative AI approaches. Cleverness, skillfulness and wisdom are, I suggest, separate aspects of intelligence, which are naturally implemented in an AI system as separate modules -- but modules which must be architected for close inter-operation, because the real crux of general intelligence is the synergetic fusion of the three.

Buckets of Crumbs!!!

I just posted a way deeper and more interesting blog post a couple hours ago (using multiverse theory and Occam's Razor to explain why voting may often be rational after all), but I decided to post this sillier one tonight too because I have a feeling I'll forget if I put it off till tomorrow (late at night I'm willing to devote a little time to blogging in lieu of much-needed sleep ... tomorrow when I wake up there will be loads of work I'll feel obliged to do instead!)

This blog post just re-"prints" part of a post I made to the AGI email list today, which a couple people already asked me if they could quote.

It was made in response to a poster on the AGI list who made the argument that AGI researchers would be more motivated to work on building superhuman AGI if there were more financial gain involved ... and that, in fact, desire for financial gain MUST be a significant part of their motivation ... since AGI researchers are only human too ...

What I said is really simple and shouldn't need to have been said, but still, this sort of thing seems to require constant repetition, due to the nature of the society we live in...

Here goes:



Singularitarian AGI researchers, even if operating largely or partly in the business domain (like myself), value the creation of AGI far more than the obtaining of material profits.




I am very interested in deriving $$ from incremental steps on the path to powerful AGI, because I think this is one of the better methods available for funding AGI R&D work.




But deriving $$ from human-level AGI really is not a big motivator of mine. To me, once human-level AGI is obtained, we have something of dramatically more interest than accumulation of any amount of wealth.




Yes, I assume that if I succeed in creating a human-level AGI, then huge amounts of $$ for research will come my way, along with enough personal $$ to liberate me from needing to manage software development contracts or mop my own floor. That will be very nice. But that's just not the point.





I'm envisioning a population of cockroaches constantly fighting over crumbs of food on the floor. Then a few of the cockroaches -- let's call them the Cockroach Robot Club -- decide to spend their lives focused on creating a superhuman robot which will incidentally allow cockroaches to upload into superhuman form with superhuman intelligence. And the other cockroaches insist that the Cockroach Robot Club's motivation in doing this must be a desire to get more crumbs of food. After all, just **IMAGINE** how many crumbs of food you'll be able to get with that superhuman robot on your side!!! Buckets full of crumbs!!!


(Perhaps after they're resurrected and uploaded, the cockroaches that used to live in my kitchen will come to appreciate the literary inspiration they've provided me! For the near future though I'll need to draw my inspiration elsewhere as Womack Exterminators seems to have successfully vanquished the beasties with large amounts of poisonous gas. Which I can't help feeling guilty about, being a huge fan of the film Twilight of the Cockroaches ... but really, I digress...)

I'm also reminded of a meeting I was in back in 1986, when I was getting trained as a telephone salesman (one of my lamer summer jobs from my grad school days ... actually I think that summer I had given up on grad school and moved to Las Vegas with the idea of becoming a freelance philosopher ... but after a couple months of phone sales, which was necessary because freelance philosophers don't make much money, I reconsidered and went back to grad school in the fall). The trainer, a big fat scary guy who looked and sounded like a meaner version of my ninth grade social studies teacher, was giving us trainee salespeople a big speech about how everyone wanted success, and he asked us how success was defined. Someone in the class answered MONEY and the trainer congratulated him and said: "That's right, in America success means money, and you're going to learn to make a lot of it!" The class cheered (a scene that could have been straight out of Idiocracy ... "I like money!"). Feeling obnoxious (as I usually was in those days), I raised my hand and asked the trainer if Einstein was successful or not ... since Einstein hadn't been particularly rich, I noted, that seemed to me like a counterexample to the principle that had been posited regarding the equivalence of success and financial wealth in the American context. The trainer changed the subject to how the salesman is like a hammer and the customer is like a nail. (By the way I was a mediocre but not horrible phone salesman of "pens, caps and mugs with your company name on them." I had to use the name "Ben Brown" on the phone though because no one could pronounce "Goertzel." If you were a small business owner in summer 1986 and got a phone call from an annoying crap salesman named Ben Brown, it was probably the 19 year old version of me....)

Why Voting May Not be Such a Stupid Idea (A Multiversal Argument)

I haven't voted in any election for a heck of a long time ... but, in some conversations a couple years ago, an argument came up that actually seems like a reasonable argument why voting might be a good idea.

I'm not sure why I never blogged this before ... but I didn't ... so here goes ...


Why might voting be worthwhile, even though the chances that your vote breaks a tie in the election are vanishingly small?

Consider this: Would you rather live in a branch of the multiverse where the people like you vote, or where the people like you don't vote?

Obviously, if there are a lot of people like you, then you'll be better off in a branch where the people like you vote.

So: You should vote so as to be sure you're in one of those branches.

But, wait a minute. How do you know you won't end up in a branch where most of the people like you DON'T vote, but you vote anyway?

Well, you can't know that for sure. But, the question to ask is, which of the two swaths of possible universes are more probable overall:

Type 1) Ones in which everyone like you votes

Type 2) Ones in which most people like you don't vote, but you're the exception

Adopting an "Occam prior" that favors simpler possible universes over more complex ones, you arrive at the conclusion that Type 1 universes are more probable.

Now, this isn't an ironclad, universal argument for voting. If you're such a freak that all the people like you voting wouldn't make any difference, then this argument shouldn't convince you to vote.

Another counterargument against the above argument is that free will doesn't exist in the multiversal framework. What the heck does it mean to "decide" which branch of the multiverse to go down? That's not the kind of thing you can decide. Your decision process is just some dynamics that occurs on some branches and not others. It's not like your decision process steps out of the branching-process governing the multiverse and chooses which routes you follow....

But the thing is, deciding still feels like deciding from within your own human mind -- whether or not it's REALLY deciding in any fundamental physical sense.

So, I'm not telling you to decide anything. I'm merely (because it's what my internal dynamics are doing, in this branch of the multiverse that we're in) typing in some words that my internal dynamics believe may encourage you to carry out some of your own internal dynamics that may feel to you like you're deciding something. Right? Because, this is simply the way the universe is happening ... in this branch of the multiverse....

Don't decide anything. Just notice that these words are making you reflect on which branch of the multiverse you'd rather be in -- the one where everyone like you votes, or the one where they don't....

And of course it's not just about voting. It's really about any ethical behavior ... any thing such that we'd all be better off if everyone like us did that thing.

It's about compassion, for that matter -- we'd all be better off if everyone was more compassionate.... Would you rather be in the branch of the multiverse where everyone like you is compassionate, or....

Well, you get it.

But am I voting in this year's Presidential elections?

Out of all the candidates available, I'd definitely support Obama ... but nah, I think I'll probably continue my long tradition of lame citizenship and not vote.

I just don't think there are that many people like me out there ;-)

But if I read enough other blog posts like this one, I'd decide there was a large enough population of similar people out there, and I WOULD vote....

Quantum Voodoo in "Classical" Systems?

Way way back in the dark ages, when I was 19 years old and in my second year of grad school, I wrote a paper called "Holistic Indeterminacy" and submitted it to the journal Mind.

The basic idea was that, in some cases, very complex "classical" physical systems might literally display the same kind of indeterminacy associated with quantum systems.

The paper was printed out crappily on a dot matrix printer with dimly printed ink, and written in a not terribly professional way. It got rejected, and I've long since lost the thing. Furthermore, I never since found time to write up the ideas in the paper again. (Had there been a Web back then I would have posted the thing on my website, but this was the mid 1980's ... if I recall correctly, I hadn't even sent an email yet, at that point. I might actually have the paper on some old floppy disk in the basement, but odds are the data's long corrupted even if the disk is still around...).

But anyways ... please pardon these reminisces of an old man!! ... these old ideas of mine came up today in a conversation I was having with a friend over lunch, so I figured I'd take a few minutes to type them into a blog post (way less work than a paper!).

In fact these ideas are far more topical now than in the 1980's, as quantum computing is these days finally becoming a reality ... along with macroscopic quantum systems and all sorts of other fun stuff....

Partly because of these advances, and partly because the ideas have had decades to pervade my brain, I think I can now express the idea a bit more crisply than I did back then.

Still, it's a freaky and speculative train of thought, which I am not fully convinced makes any sense.

But at very least, it's some amusing hi-fi sci-fi.....

The basic idea is as follows.

Premise: Quantum logic is the logic of that which, in principle, cannot be observed. Classical logic is the logic of that which can, in principle, be observed.

The above may sound odd but it's not my idea -- it's the conclusion of a lot of work in quantum physics and the quantum theory of measurement, by serious physicists who understand such things far better than I do. It's way clearer now than it was in the mid 80's, though it was known to all the cool people even then....

Now is where things start to get weird. I want to make the above premise observer-dependent in a manner different from how quantum theory does it. Namely, I want to introduce an observer who, himself, has a finite capacity for understanding and observation -- a finite Kolmogorov complexity, for example.

This leads to my

Modest proposal: An observing system should use quantum logic to reason about anything that it, as a particular system, cannot in principle observe.

There are some things that a worm cannot observe, because it is just a worm; but I can observe. From the perspective of the worm, I suggest, these things should be reasoned about using quantum logic.

Similarly, there are some things that I cannot observe, in principle, because I am just a little old me.

Yes, I could potentially expand myself into a dramatically greater being. But, then that wouldn't help ME (i.e., my current self) to observe these things ... it would just help {some other, greater guy who had evolved out of me} to observe these things.

Of course, you can't step into the same river once ... and there is not really any ME that is persistent beyond an individual moment (and there are no individual moments!). But you can talk about a class of systems, and you can say that some observables are simply NOT observable by any system within that class. So systems within that class need to reason about these observables using quantum logic.

Where does complexity come into the picture? Well, among the things I can't in principle observe, are patterns of more complexity than can fit in my brain.

And among the things my deliberatively conscious mind can't in principle observe, are patterns of more complexity than can fit within its own very limited capacity.

So, if we interpret "quantum logic is the logic of things that can't in principle be observed" subjectively, as applying to particular real-world observing systems (including subsystems like the deliberatively conscious component of a human brain), then we arrive at the funky conclusion that maybe we should reason about each others' minds using quantum logic ... or maybe even, that we should reason about our own unconscious using quantum logic....

Funny idea, hmmm?

Way back when I wrote down some mathematics embodying these notions, but I don't feel like regenerating that right now. Although I'm a bit curious to see whether it had any validity or not ;-)

What made me think of this today was a discussion about consciousness, and the possibility (raised by the friend I was talking to) that some sort of wacky quantum voodoo is necessary to produce consciousness.

Maybe so. On the other hand, it could also be that any system complex enough to display the kind of rich deliberative consciousness we humans do, is complex enough that humans need to reason about it using quantum logic ... because in principle we cannot observe its dynamics (without becoming way more complex than we are, hence losing our self-ness...).

Ahhh... well I'll get back to doing the final edits on the Probabilistic Logic Networks book now ...

A New, Improved, Completely Whacky Theory of Evolution

This blog posts presents some really weird, speculative science, that I take with multiple proverbial salt-grains ... but, well, wouldn't it be funky if it were true?

The idea came to mind in the context of a conversation with my old friend Allan Combs, with whom I co-edit the online journal Dynamical Psychology.

It basically concerns the potential synergy between two apparently radically different lines of thinking:

• Rupert Sheldrake's idea of "morphic fields"
• The notion of EDA's (Estimation of Distribution Algorithms) in computer science

Morphic Fields

The basic idea of a morphic field is that, in this universe, patterns tend to continue -- even when there's not any obvious causal mechanism for it. So that, for instance, if you teach thousands of rats worldwide a certain trick, then afterwards it will be easier for additional rats to learn that trick, even though the additional rats have not communicated with the prior one.

Sheldrake and others have gathered a bunch of evidence in favor of this claim. Some say that it's fraudulent or somehow subtly methodologically flawed. It might be. But after my recent foray into studying Ed May's work on precognition, and other references from Damien Broderick's heartily-recommended book Outside the Gates of Science (see my previous blog posts on psi), I'm becoming even more willing than usual to listen to data even when it goes against prevailing ideas.

Regarding morphic fields on the whole, as with psi, I'm still undecided, but interested. The morphic field idea certainly fits naturally with my philosophy that "the domain of pattern is primary, not the domain of spacetime"

Estimation of Distribution Algorithms

EDA's, on the other hand, are a nifty computer science idea aimed at accelerating artificial evolution (that occurs within software processes)

Evolutionary algorithms are a technique in computer science in which, if you want to find/create a certain object satisfying a certain criterion, you interpret the criterion as a "fitness function" and then simulate an "artificial evolution process" to try to evolve objects better and better satisfying the criterion. A population of candidate objects is generated at random, and then, progressively, evolving objects are crossed-over and mutated with each other. The fittest are chosen for further survival, crossover and mutation; the rest are discarded.

Google "genetic algorithms" and "genetic programming" if this is novel to you.

This approach has been used to do a lot of practical stuff -- in my own work, for example, I've evolved classification rules predicting who has cancer or who doesn't based on their genetic data (see Biomind); evolved little programs controlling virtual agents in virtual worlds to carry out particular tasks (see Novamente); etc. (though in both of those cases, we have recently moved beyond standard evolutionary algorithms to use EDA's ... see below...)

EDA's mix evolutionary algorithms with probabilistic modeling. If you want to find/create an object satisfying a certain criterion, you generate a bunch of candidates -- and then, instead of letting them cross over and mutate, you do some probability theory and figure out the patterns distinguishing the fit ones from the unfit ones. Then you generate new babies, new candidates, from this probability distribution -- throw them into the evolving population; lather, rinse, repeat.

It's as if, instead of all this sexual mating bullcrap, the Federal gov't made an index of all our DNA, then did a statistical study of which combinations of genes tended to lead to "fit" individuals, then created new individuals based on this statistical information. Then these new individuals, as they grow up and live, give more statistical data to throw into the probability distribution, etc. (I'd argue that this kind of eugenics is actually a plausible future, if I didn't think that other technological/scientific developments were so likely to render it irrelevant.)

Martin Pelikan's recent book presents the idea quite well, for a technical computer science audience.

Moshe Looks' PhD thesis presents some ideas I co-developed regarding applying EDA's to automated program learning.

There is by now a lot of mathematical/computational evidence that EDA's can solve optimization problems that are "deceptive" (hence very difficult to solve) for pure evolutionary learning. To put it in simple terms, there are many broad classes of fitness functions for which pure neo-Darwinist evolution seems prone to run into dead ends, but for which EDA style evolution can jump out of the dead ends.

Morphic Fields + EDA's = ??

Anyway -- now how do these two ideas fit together?

What occurred to Allan Combs and myself in an email exchange (originating from Allan reading about EDA's in my book The Hidden Pattern) is:

If you assume the morphic field hypothesis is true, then the idea that the morphic field can serve as the "probability distribution" for an EDA (allowing EDA-like accelerated evolution) follows almost immediately...

How might this work?

One argument goes as follows.

Many aspects of evolving systems are underdetermined by their underlying genetics, and arise via self-organization (coupled to the environment and initiated via genetics). A great example is the fetal and early-infancy brain, as analyzed in detail by Edelman (in Neural Darwinism and other writings) and others. Let's take this example as a "paradigm case" for discussion.

If there is a morphic field, then it would store the patterns that occurred most often in brain-moments. The brains that survived longest would get to imprint their long-lasting patterns most heavily on the morphic field. So, the morphic field would contain a pattern P, with a probability proportional to the occurrence of P in recently living brains ... meaning that occurrence of P in the morphogenetic field would correspond roughly to the fitness of organisms containing P.

Then, when young brains were self-organizing, they would be most likely to get imprinted with the morphic-field patterns corresponding to the most-fit recent brains....

So, if one assumes a probabilistically-weighted morphic field (with the weight of a pattern proportional to the number of times it's presented) then one arrives at the conclusion that evolution uses an EDA ...

Interesting to think that the mathematical power of EDA's might underly some of the power of biological evolution!

The Role of Symbiosis?

In computer science there are other approaches than EDAs for jumping out of evolutionary-programming dead ends, though -- one is symbiosis and its potential to explore spaces of forms more efficiently than pure evolution. See e.g. Richard Watson's book from a couple year back --

Compositional Evolution: The Impact of Sex, Symbiosis, and Modularity
on the Gradualist Framework of Evolution

and, also, Google "symbiogenesis." (Marginally relevantly, I wrote a bit about Schwemmler's ideas on symbiogenesis and cancer , a while back.)

But of course, symbiosis and morphic fields are not contradictory notions.

Hypothetically, morphic fields could play a role in helping organisms to find the right symbiotic combinations...

But How Could It Be True?

How the morphic fields would work in terms of physics is a whole other question. I don't know. No one does.

As I emphasized in my posts on psi earlier this year, it's important not to reject data just because one lacks a good theory to explain it.

I do have some interesting speculations to propound, though (I bet you suspected as much ;-). I'll put these off till another blog post ... but if you want a clue of my direction of thinking, mull a bit on

http://www.physics.gatech.edu/schatz/clocks.html

Brief Report on AGI-08

Sooo....

The AGI-08 conference (agi-08.org) occurred last weekend in Memphis...!

I had hoped to write up a real scientific summary of AGI-08, but at the moment it doesn't look like I'll find the time, so instead I'll make do with this briefer and more surface-level summary...

Firstly, the conference went VERY well. The tone was upbeat, the discussions were animated and intelligent, and all in all there was a feel of real excitement about having so many AGI people in one place at one time.

Attendance was good: We originally anticipated 80 registrants but had 120+.

The conference room was a futuristic setting called "The Zone" that looked sorta like the Star Trek bridge -- with an excellent if mildly glitchy video system that, during Q&A sessions, showed the questioner up on a big screen in front of the room.

The unconventional format (brief talks followed by long discussion/Q&A) sessions was both productive and popular. The whole thing was video-ed and at some point the video record will be made available online (I don't know the intended timing of this yet).

The proceedings volume was released by IOS Press a few weeks before the conference and is a thick impressive-looking tome.

The interdisciplinary aspect of the conference seemed to work well -- e.g. the session on virtual-worlds AI was chaired by Sibley Verbeck (CEO of Electric Sheep Company) and the session on neural nets was chaired by Randal Koene (a neuroscientist from Boston University). This definitely made the discussions deeper than if it had been an AI-researchers-only crowd.

Plenty of folks from government agencies and large and small corporations were in attendance, as well as of course many AI academics and non-affiliated AGI enthusiasts. Among the AI academics were some highly-respected stalwarts of the AI community, alongside the new generation...

There seemed to be nearly as many Europeans as Americans there, which was a pleasant surprise, and some Asians as well.

The post-conference workshop on ethical, sociocultural and futurological issues drew about 60 people and was a bit of a free-for-all, with many conflicting perspectives presented quite emphatically and vociferously. I think most of that discussion was NOT captured on video (it took place in a different room where video-ing was less convenient), though the workshop talks themselves were.

The media folks in attendance seemed most energized by the section on AI in virtual worlds, which is because in this section the presenters (me, Andrew Shilliday, and Martin Magnusson) showed movies of cute animated characters doing stuff. This gave the nontechnical observers something to grab onto, which most of the other talks did not.

As at the earlier AGI-06 workshop, one of the most obvious observations after listening to the talks was that a lot of AGI research programs are pursuing fairly similar architectures and ideas but using different languages to describe what they're doing. This suggests that making a systematic effort at finding a common language and really understanding the true overlaps and differences of the various approaches, would be very beneficial. There was some talk of organizing a small, invitation-only workshop among practicing AGI system architects, perhaps in Fall 2008, with a view toward making progress in this direction.

Much enthusiasm was expressed for an AGI-09, and it was decided that this will likely be located in Washington DC, a location that will give us the opportunity to use the conference to help energize various government agencies about AGI.

There was also talk about the possibility of an AGI online technical journal, and a group of folks will be following that up, led by Pei Wang.

An "AGI Roadmap" project was also discussed, which would involve aligning different cognitive architectures currently proposed insofar as possible, but also go beyond that. Another key aspect of the roadmap might be an agreement on certain test environments or tasks that could be used to compare and explore various AGI architectures in more of a common way than is now possible.

Lots of ideas ... lots of enthusiasm ... a strong feeling of community-building ... so, I'm really grateful to Stan Franklin, Pei Wang, Sidney DeMello and Bruce Klein and everyone else who helped to organize the conference.

Finally, an interesting piece of feedback was given by my mother, who knows nothing about AGI research (she runs a social service agency) and who did not attend the conference but read the media coverage afterwards. What she said is that the media seems to be taking a far less skeptical and mocking tone toward AGI these days, as opposed to 7-10 years ago when I first started appearing in the media now and then. I think this is true, and it signifies a real shift in cultural attitude. This shift is what allowed The Singularity Is Near to sell as many copies as it did; and what encouraged so many AI academics to come to a mildly out-of-the-mainstream conference on AGI. Society, including the society of scientists, is starting to wake up to the notion that, given modern technology and science, human-level AGI is no longer a pipe dream but a potential near-term reality. w00t! Of course there is a long way to go in terms of getting this kind of work taken as seriously as it should be, but at least things seem to be going in the right direction.

Balancing concrete work on AGI with community-building work like co-organizing AGI is always a tricky decision for me.... But in this case, the conference went sufficiently well that I think it was worthwhile to deviate some time from the R&D to help out with it. (And now, back to the mass of other work that piled up for me during the conference!)

Yet More Rambling on Will (Beyond the Rules vs. Randomness Dichotomy)

A bit more on this nasty issue of will ... complementing rather than contradicting my previously-expressed ideas.

(A lot of these theory-of-mind blog posts are gonna ultimately get revised and make their way into The Web of Pattern, the sequel to The Hidden Pattern that I've been brewing in my mind for a while...)

What occurred to me recently was a way out of the old argument that "free will can't exist because the only possibilities are RULES versus RANDOMNESS."

In other words, the old argument goes: Either a given behavior is determined, or it's random. And in either case, where's the will? Granted, a random coin-toss (quantum or otherwise) may be considered "free" in a sense, but it's not willed -- it's just random.

What occurred to me is that this dichotomy is oversimplified because it fails to take two factors into account:

1. A subjectively experienced moment occurs over a fuzzy span of time, not at a single physical moment
2. "Random" always means "random with respect to some observer."

To clarify the latter point: "S is random to system X" just means "S contains no patterns that system X could identify."

System Y may be able to recognize some patterns in S, even though X can't.

And, X may later evolve into X1, which can recognize patterns in S.

Something that was random to me thirty years ago, or thirty seconds ago, may be patterned to me now.

Consider the perspective of the deliberative, rational component of my mind, when it needs to make a choice. It can determine something internally, or it can draw on an outside source, whose outcome may not be predictable to it (that is, it may make a "random" choice). Regarding outside sources, options include

1. a random or pseudorandom number generator
2. feedback from the external physical world, or from another mind in the vicinity
3. feedback from the unconscious (or less conscious) non-deliberative part of the mind

Any one of these may introduce a "random" stimulus that is unpatterned from the point of view of the deliberative decision-maker.

But of course, options 2 and 3 have some different properties from option 1. This is because, in options 2 or 3, something that appears random at a certain moment, may appear non-random a little later, once the deliberative mind has learned a little more (and is thus able to recognize more or different patterns).

Specifically, in the case of option 3, it is possible for the deliberative mind to draw on the unconscious mind for a "random" choice, and then a half-moment later, import more information from the unconscious that allows it to see some of the patterns underlying the previously-random choice. We may call this process "internal patternization."

Similarly, in the case of option 2, it is possible for the deliberative mind to draw on another mind for a "random" choice, and then a half-moment later, import more information from the other mind that allows it to see some of the patterns underlying the previously random choice. We may call this process "social patternization."

There's also "physical patternization" where the random choice comes from an orderly (but initially random to the perceiving mind) process in the external world.

These possibilities are interesting to consider in the light of the non-instantaneity of the subjective moment. Because, the process of patternization may occur within a single experienced moment.

The subjective experience of will, I suggest, is closely tied to the process of internal patternization. When we have the feeling of making a willed decision, we are often making a "random" choice (random from the perspective of our deliberative component), and then immediately having the feeling of seeing some of the logic and motivations under that choice (as information passes from unconscious to conscious). But the information passed into the deliberative mind is of course never complete and there's always still some indeterminacy left, due to the limited capacity of deliberative mind as compared to unconscious mind.

So, what is there besides RULES plus RANDOMNESS?

There is the feeling of RANDOMNESS transforming into RULES (i.e. patterns), within a single subjective moment.

When this feeling involves patterns of the form "Willing X is causing {Willing X plus the occurrence of S}", then we have the "free will" experience. (This is the tie-in with my discourse on free will and hypersets, a few blog posts ago.)

That is, the deliberative content of recursive willing is automatized and made part of the unconscious, through repeated enaction. It then plays a role in unconscious action determination, which is perceived as random by the deliberative mind -- until, toward the tail end of a subjective moment, it becomes more patterned (from the view of the deliberative mind) due to receiving more attention.

Getting practical for a moment: None of this, as I see it, is stuff that you should program into an AGI system. Rather it is stuff that should emerge within the system as a part of its ongoing recognition of patterns in the world and itself, oriented toward achieving its goals. In this particular case the dynamics of attention allocation is key -- the process by which low-attention items (unconscious) can rapidly gain attention (become intensely deliberatively conscious) within a single subjective moment, but can also have a decisive causal impact prior to this increase in attention. The nonlinear dynamics of attention, in other words, is one of the underpinnings of the subjective experience of will.

What I'm trying to do here is connect phenomenology, cognitive science and AGI design. It seems to work, conceptually, in terms of according with my own subjective experience and also with known data on human brain/mind and my intuition/experience with AGI design.

Characterizing Consciousness and Will in Terms of Hypersets

This is another uber-meaty blog post, which reports a train of thought I had while eating dinner with my wife last night, which appears to me to provide a new perspective on two of the thorniest issues in the philosophy of mind: consciousness and will.

(No, I wasn't eating any hallucinogenic mushrooms for dinner; just some grilled chicken with garlic and ginger and soy sauce, on garlic naan. Go figure.)

These are of course very old issues and it may seem every possible perspective on them has already been put forth, without anything fundamentally being resolved.

However, it seems to me that the perspectives on these topics explored so far constitute only a small percentage of the perspectives that may sensibly be taken.

What I'm going to do here is to outline a new approach to these issues, which is based on hyperset theory -- and which ties in with various things I've written on these topics before, inspired by neuropsychology and systems theory and probabilistic logic and so on and so forth.

(A brief digressive personal comment: I've been sooooo overwhelmingly busy with Novamente-related business stuff lately, it's really been a pleasure to take a couple hours to write down some thoughts on these more abstract topics! Of course, no matter what I'm doing with my time as I go through my days, my unconscious is constantly churning on conceptual issues like the ones I talk about in this blog post -- but time to write down my thoughts on such things is so damn scant lately.... One of the next things to get popped off the stack is the relation of the model of will given here with ethical decision-making, as related to the iterated prisoner's dilemma, the voting problem, and so forth. Well, maybe next week ... or next month.... I certainly feel like working toward making a thinking machine for real, is more critical than exploring concepts in the theory of mind; but on a minute-by-minute basis, I have to admit I find the latter more fun....)

Hypersets

One of the intuitions underlying the explorations presented here is that possibly it's worth considering hypersets as an important part of our mathematical and conceptual model of mind -- and consciousness and will in particular.

A useful analogy might be the way that differential equations are an important part of our mathematical and conceptual model of physical reality. Differential equations aren't in the world; and hypersets aren't in the mind; but these sorts of mathematical abstractions may be extremely useful for modeling and understanding what's going on.

In brief, hypersets are sets that allow circular membership structures, e.g. you can have

A = {A}

A = {B,{A}}

and so forth. It follows that you can have functions that take themselves as arguments, and lots of other stuff that doesn't work according to the standard axioms of set theory.

While exotic, hypersets are well-defined mathematical structures, and in fact simple hypersets have fewer conceptual conundrums associated with them than the real number system (which is assumed in nearly all our physics theories).

The best treatment of hypersets for non-mathematicians that I know of is the book The Liar, which I highly recommend.

Anyway, getting down to business, let's start with consciousness, and then after that we'll proceed to will.

Disambiguating Consciousness

Of course the natural language term "consciousness" is heavily polysemous, and I'm not going to try to grapple with every one of its meanings. Specifically, I'm going to focus on the meaning that might be specified as "reflective consciousness." Which is different from the "raw awareness" that, arguably, worms and bugs have, along with us bigger creatures.

Raw awareness is also an interesting topic, though I tend toward a kind of panpsychism, meaning that I tend to believe everything (even a rock or an electron) possesses some level of raw awareness. Which means that raw awareness is then just an aspect of being, rather than a separate quality that some entities possess and not others.

Beyond raw awareness, though, it's clear that different entities in the universe manifest different kinds of awareness. Worms are aware in a different way than rocks; and, I argue, dogs, pigs, pigeons and people are aware in a different way from worms. What I'll (try to) deal with here is the sense in which the latter beasts are conscious whereas worms are not -- i.e. what might be called "reflective consciousness." (Not a great term, but I don't know any great terms in this domain.)

Defining Reflective Consciousness

So, getting down to business.... My starting-point is the old cliche' that

Consciousness is consciousness of consciousness

This is very nice, but doesn't really serve as a definition or precise characterization.

In hyperset theory, one can write an equation

f = f(f)

with complete mathematical consistency. You feed f, as input, f; and you receive, as output, f.

It seems evident, though, that while this sort of anti-foundational recursion may be closely associated with consciousness, this simple equation itself doesn't tell you much about consciousness. We don't really want to say

Consciousness = Consciousness(Consciousness)

I think it's probably more useful to say:

Consciousness is a hyperset, and consciousness is contained in its membership scope

Here by the "membership scope" of a hyperset S, what I mean is the members of S, plus the members of the members of S, etc.

This is no longer a definition of consciousness, merely a characterization.

What is says is that consciousness must be defined anti-foundationally as some sort of construct via which consciousness builds consciousness from consciousness -- but it doesn't specify exactly how.

Next, I want to introduce the observation, which I made in The Hidden Pattern (and in an earlier essay) that the subjective experience of being conscious of some entity X, is correlated with the presence of a very intense pattern in one's overall mind-state, corresponding to X. This idea is also the essence of neuroscientist Susan Greenfield's theory of consciousness (but in her theory, "overall mind-state" is replaced with "brain-state").

Putting these pieces together (hypersets, patterns and correlations), we arrive at the following working definition of consciousness:

"S is conscious of X" is defined as: The declarative content that {"S is conscious of X" correlates with "X is a pattern in S"}

In other words: Being conscious of a pig, means having in one's mind declarative knowledge of the form that one's consciousness of that pig is correlated with that pig being a pattern in one's overall mind-state.

Note that this declarative knowledge must be expressed in some language such as hyperset theory, in which anti-foundational inclusions are permitted. But of course, it doesn't have to be a well-formalized language -- just as pigeons, for instance, can carry out deductive reasoning without having a formalization of the rules of Boolean or probabilistic logic in their brains. All that is required is that the conscious mind has an internal informal language capable of expressing and manipulating simple hypersets.

To make this formal, one requires also a definition of pattern, which I've supplied in The Hidden Pattern.

OK, so much for consciousness. Now, on to our other old friend, will.

Defining Will

The same approach, I suggest, can be used to define the notion of "will," by which I mean the sort of willing process that we carry out in our minds when we subjectively feel like we are deciding to make one choice rather than another.

In brief:

"S wills X" is defined as: The declarative content that {"S wills X" causally implies "S does X"}

To fully explicate this is slightly more complicated than in the case of consciousness, due to the need to unravel what's meant by "causal implication." This is done in my forthcoming book Probabilistic Logic Networks in some detail, but I'll give the basic outline here.

Causal implication may be defined as: Predictive implication combined with the existence of a plausible causal mechanism.

More precisely, if A and B are two classes of events, then A "predictively implies B" if it's probabilistically true that in a situation where A occurs, B often occurs afterwards. (Yes, this is dependent on a model of what is a "situation", which is assumed to be part of the mind assessing the predictive implication.)

And, a "plausible causal mechanism" associated with the assertion "A predictively implies B" means that, if one removed from one's knowledge base all specific instances of situations providing direct evidence for "A predictively implies B", then the inferred evidence for "A predictively implies B" would still be reasonably strong. (In a certain logical lingo, this means there is strong intensional evidence for the predictive implication, along with extensional evidence.)

If X and Y are particular events, then the probability of "X causally implies Y" may be assessed by probabilistic inference based on the classes (A, B, etc.) of events that X and Y belong to.

In What Sense Is Will Free?

But what does this say about the philosophical issues traditionally associated with the notion of "free will"?

Well, it doesn't suggest any validity for the idea that will somehow adds a magical ingredient beyond the familiar ingredients of "rules" plus "randomness." In that sense, it's not a very radical approach. It fits in with the modern understanding that free will is to a certain extent an "illusion."

However, it also suggests that "illusion" is not quite the right word.

The notion that willed actions somehow avoid the apparently-deterministic/stochastic nature of the universe is not really part of the subjective experience of free will ... it's a conceptual add-on that comes from trying to integrate our subjective experience with the modern scientific understanding of the world, in an overly simplistic and partially erroneous way.

An act of will may have causal implication, according to the psychological definition of the latter, without this action of will violating the basic deterministic/stochastic equations of the universe. The key point is that causality is itself a psychological notion (where within "psychological" I include cultural as well as individual psychology). Causality is not a physical notion; there is no branch of science that contains the notion of causation within its formal language.

In the internal language of mind, acts of will have causal impacts -- and this is consistent with the hypothesis that mental actions may potentially be ultimately determined via determistic/stochastic lower-level dynamics. Acts of will exist on a different level of description than these lower-level dynamics.

The lower-level dynamics are part of a theory that compactly explains the behavior of cells, molecules and particles; and some aspects of complex higher-level systems like brains, bodies and societies. Will is part of a theory that compactly explains the decisions of a mind to itself.

My own perspective is that neither the lower-level dynamics (e.g. physics equations) nor will should be considered as "absolutely real" -- there is no such thing as absolute reality. The equations of physics, glorious as they are, are abstractions we've created, and that we accept due to their utility for helping us carry out various goals and recognize various regularities in our own subjective experience.


Connecting Will and Consciousness

Connecting back to our first topic, consciousness, we may say that:


In the domain of reflective conscious experiences, acts of will are experienced as causal.

This of course looks