A New Approach to Computational Language Learning

I've been thinking about a new approach to computational language learning for a while, and finally found time to write it down -- see the 2 page document here.

Pursued on its own, this is a "narrow AI" approach, but it's also designed to be pursued in an AGI context, and integrated into an AGI system like OpenCog.

In very broad terms, these ideas are consistent with the integrative NLP approach I described in this 2008 conference paper. But the application of evolutionary learning is a new idea, which should allow a more learning-oriented integrative approach than the conference paper alluded to.

Refining and implementing these ideas would be a lot of work, probably the equivalent of a PhD thesis for a very good student.

Those with a pure "experiential learning" bent will not like the suggested approach much, because it involves making use of existing linguistic resources alongside experiential knowledge. However, there's no doubt that existing statistical and rule-based computational linguistics have made a lot of progress, in spite of not having achieved human-level linguistic performance. I think the outlined approach would be able to leverage this progress in a way that works for AGI and integrates well with experiential learning.

I also think it would be possible for an AGI system (e.g. OpenCog, or many other approaches) to learn language purely from perceptual experience. However, the possibility of such an approach, doesn't imply its optimality in practice, given the hardware, software and knowledge resources available to us right now.

A Mind-World Correspondence Principle

I had some more ideas, working toward a general theory of general intelligence, which I wrote in a paper posted online at Dynamical Psychology.

(Please note: it's fairly abstract theoretical/mathematical material, so if you're solely interested in current AGI engineering work, don't bother! The hope is that this theory will be able to help guide engineering work once it's further developed, but it's not at that stage yet. So for now my abstract mathematical AGI theory work and practical AGI engineering work are only loosely coupled.)

The crux of the paper is:

MIND-WORLD CORRESPONDENCE PRINCIPLE: For an organism with a reasonably high level of intelligence in a certain world, relative to a certain set of goals, the mind-world path transfer function is a goal-weighted approximate functor

To see what those terms mean and why it might be a useful notion, you'll have to read the paper.

A cruder expression of the same idea, with fewer special defined terms is:

MIND-WORLD CORRESPONDENCE-PRINCIPLE: For a mind to work intelligently toward certain goals in a certain world, there should be a nice mapping from goal-directed sequences of world-states into sequences of mind-states, where “nice” means that a world-state-sequence W composed of two parts W1 and W2, gets mapped into a mind-state-sequence M composed of two corresponding parts M1 and M2.

As noted toward the end of the paper, this principle gives us systematic way to approach questions like: Why do real-world minds seem to be full of hierarchical structures? The answer is probably that the real world is full of goal-relevant hierarchical structures. The Mind-World Correspondence Principle explains exactly why these hierarchical structures in the world have to be reflected by hierarchical structures in the mind of any system that's intelligent in the world.

As an aside, it also occurred to me that these ideas might give us a nice way to formalize the notion of a "good mind upload," in category-theoretic terms.

I.e., if we characterize minds via transition graphs in the way done in the paper, then we can argue that mind X is a valid upload of mind Y if there is a fairly accurate approximate functor from X's transition graph to Y's.

And, if Y is a nondestructive upload (so X still exists after the uploading), it would remain a good upload of X over time if, as X and Y both changed, there was a natural transformation governing the functors between them. Of course, your upload might not WANT to remain aligned with you in this manner, but that's a different issue...

Creating/Discovering New States of Mind

Just some quasi-random musings that went through my head yesterday…

Our society puts a fair bit of energy, these days, into creating new technologies and discovering new scientific facts.

But we don’t put hardly any effort at all into creating/discovering new states of mind.

I think maybe we should – and at the end of this odd, long, rambling blog post I’m going to suggest a specific type of new mind-state that I think is well worth trying to create/discover: one synthesizing spiritual mindfulness and intense scientific creativity.

On Old and New States of Consciousness

First, bear with me while I spend a few paragraphs framing the issue…

When I read Stcherbatsky’s book Buddhist Logic years ago, I was struck by the careful analysis of 128 states of consciousness. Allan Combs’ book The Radiance of Being provides a simpler, smaller conceptual analysis of states of consciousness, with similar foundations. These and other similar endeavors are very worthy – but how can we really know that the scope of all possible varieties of human consciousness-state has been thoroughly explored?

All sorts of amazing new states of consciousness will become possible once the human brain has been enhanced with technology – brain-computer interfacing, genetic engineering, mind uploading, etc. Advanced AGI systems may enjoy states of consciousness far beyond human comprehension. However, it seems quite possible that ordinary human brains may be capable of many states of consciousness not yet explored.

The individual human mind is not all that individual – so the states of mind accessible to an individual may depend to some extent on the culture in which they exist. The catalogue of states of mind available in medieval India when Buddhist logic was invented, may include some states that are extremely hard for modern people to get into, and may omit some states of which modern people are capable.

The Perceived Conflict Between Scientific and Spiritual Mind-States

I’ve often wondered whether there’s some intrinsic conflict between the states of mind labeled “spiritual enlightenment”, and the states of mind consistent with profound scientific discovery.

Great scientific creation often seems to involve a lot of struggle and persistence – along with long stretches of beautiful “flow” experience. Great scientific work seems to involve a lot of very hard thinking and analysis, whereas enlightenment is generally described as involving “stopping all thought.”

Personally, I find it a lot easier to be mindful (in the Zen sense) while walking through the park, washing the dishes, lying in bed, or building a chair -- than while analyzing genomic data, working out the details of a new AI algorithm, writing a novel, or debugging complex software code. Subjectively, this feels to me like it’s because being mindful requires a bit of mental effort at first – to actively pay attention to what my mind and body are doing. Once the effort is done, then mindfulness can flow along effortlessly for a while. But then I may drift away from it, and that little jump of effort is needed to become mindful again. This dynamic of mindfulness drifting and returning, or almost drifting but then not actually drifting after all, seems not to function properly when I’m doing something highly cognitively intensive. When I’m doing the highly intensive thing, I get deeply “into” the process, which puts me in a wonderful flow state for a while – but then when the flow state ends, I’m not necessarily in a quasi-enlightened mindful state. I may be elated, or I may be exhausted, or I may be frustrated. I can then try to be mindful of my elation, exhaustion or frustration – but this is then a moderately substantial effort; and definitely my degree of mindfulness is lower than if I hadn’t bothered to do the cognitively intensive thing.

Now, it might just be that I’m not a particularly enlightened guy. Indeed, I have never claimed to be! I do have my moments of spiritual purity and cosmic blissful wisdom and all that -- but then I also have some pretty boring routine moments, and also moments of being totally un-mindfully overcome with various kinds of positive or negative emotion. However, observing other humans around me, I note that the same dichotomy I feel in my mind occurs in the outside world. I know some enlightened minds, and I know some productive, brilliant artists and scientists – but I don’t know anyone in the intersection. Maybe someone of this nature does exist; but if they do, they’re an awfully rare bird.

You could argue that, since being a spiritual genius is rare and being a scientific genius is rare, it’s not surprising that few people lie in the intersection! But I’m not just talking about genius. I’m talking about passion. Who has true devoted passion for spiritual enlightenment, and also true devoted passion for doing revolutionary science? Most people I know, if they like either, pursue one as a central goal and the other as a sort of sideline.

I don’t particularly want to be this way myself – I’d like to pursue both simultaneously, without feeling any conflict between the two. But in practical life I do feel a conflict, and I tend to choose science and art most of the time. Yes, from the enlightened view, the dichotomy and the conflict are just constructs of my mind. And when I’m in certain states of mind, I feel that way – that dichotomy and all the rest feel bogus and mildly amusing. But when I’m in those states of mind, I’m not doing my best art or science! Similarly, thinking about playing the piano, it clear that my best music has been played in states of heightened emotion – not states of enlightened emptiness.

I think the difficulty of maintaining a mindful mind-state and scientifically intensely creative mind-state, is deeply tied with the conflict between modern scientific culture and some older cultures like those of ancient India or China, that were more spiritually focused. The enlightened master was one of the ideals of India and China; and the great scientist or artist is one of the ideals of the modern world. The differences in ideals reflect more thoroughgoing cultural differences.

You could say that both the great scientist and the enlightened master are exaggerations, and the right thing is to be more balanced – a little bit scientific, a little bit spiritual. Maybe, as someone said to me recently, an enlightened master is like an Arnold Schwarzenegger of the spirit – hyper-developed beyond what is natural or useful (except in contexts like the Mr. Universe contest where being at the extreme is useful in itself!). And maybe great super-scientists are unnecessarily and unhealthily obsessive, and science would progress OK without them, albeit a little more slowly. But something in me rebels against this kind of conclusion. Maybe it’s just that I’m an unbalanced individual – reeling back and forth endlessly between being excessively scientific and excessively spiritual, instead of remaining calmly in the middle where I belong -- but maybe there’s more to it than that.

A New Scientific/Spiritual Mind-State?

What if, instead of being frustrated at the apparent contradiction between the mind-states of spiritual enlightenment /mindfulness and intense scientific creativity, we took it as a multidimensional challenge: to create a new state of mind, synergizing both of these aspects?

The ancient Indians and Chinese didn’t include this sort of mind-state in their catalogue, but they didn’t have science or modern art … they had a very different culture.

Can we discover a new, intrinsically mindful way of doing science and art? Without sacrificing the intensity or the creativity?

What if we pursued the discovery/creation of new states of mind as avidly as we pursue the creation of new machines or chemical compounds? What if there were huge multinational organizations devoted to mind-state discovery, alongside our chemical and pharmaceutical and computer engineering firms?

Zum: A Thought-Experiment

To make the above idea a little more concrete, let’s imagine a specific social structure designed to produce a synergetically scientific-spiritual state of mind. Imagine an agile software development team – a group of software developers working closely together on a project – that was also, simultaneously, a “zendo” or “dojo” or whatever you want to call it … a group of people gathered together in the interest of their own enlightenment. That is, they were simultaneously trying to get stuff done together, and to help each other maintain a state of mindfulness and individual & collective spiritual awareness.

I can’t think of a good name for this kind of combination, so I’m going to call it a “Zum”, because that word currently has no English meaning, and it reminds me of Zen and scrum (the latter a term from agile software development), and I like the letter “Z.”

I have heard of a new type of Vipassana meditation, in which a group of people sit together and while they meditate, verbalize their feelings as they pass through – “cold”, “breathing”, “warm”, “stomach”, etc. One can imagine a Zum engaging in this kind of discussion at appropriate moments, in the midst of technical discussions or collaborative work. Would hearing others describe their state like this interrupt thought in an unacceptable way? Possibly. Or would people learn to flow with it, as I flow with the music I listen to as I work?

What would a Zum be like? Would it help to have a couple enlightened masters hanging around? – maybe sitting there and meditating, or playing ping pong? That would produce a rather different vibe than a usual software development lab!

The key ingredient of the Zum is the attitude and motivation of the individuals involved. They would need to be dedicated both to producing great software together, and to helping each other remain mindful and joyful as much as possible.

One thing that might come out of this is, simply, a kind of balance, where the team does reasonably good work and is also rather happy. This certainly wouldn’t be a disaster. Maybe they’d even be a bit more effective than an average team due to a diminished incidence of personality conflicts and fewer stress-induced errors.

Another possibility is that, if this sort of experiment were tried in a variety of different styles and places, eventually a new state of mind would evolve – one bypassing the dichotomy of spiritual mindfulness versus intensely creative science or art production.

Solo Zum?

But do we really need a Zum? Organizing groups of people in novel configurations involves considerable practical difficulty. Why not become a one-person Zum? Experiment with different ways of practicing intense scientific creation and mindfulness at the same time – maybe you’ll come up with something new. Try to describe your internal methodology so others can follow in your footsteps. This sort of experimentation is every bit as valid and important as scientific experimentation, or personal experimentation with smart drugs. The human brain is far more flexible than we normally realize, it’s hard to say what may be possible even without technological brain modification.

Heh... well I'm really not sure how much any of that means, but it was an amusing train of thought! Now, it's time to pick up my daughter from school, and then get back to work.... I will be trying to be as cosmically aware as possible while my work proceeds ;O ;-) ... and probably not succeeding all that well !! So it goes... bring on the brain chips please...

This blog post was written while repetitively listening to various versions of A Tear for Eddie by Ween. This one is perhaps my favorite, though the studio version is great too.