Saturday, March 12, 2005

Lojbanic AI and the Chaotic Committee of Sub-Bens

In 1998 when my dear departed AI software company Intelligenesis Corp. was just getting started, we had a summer "intern" by the name of Mark Shoulson, who was (if memory serves) a grad student at Rutgers University. Mark worked for us for a few summer months then went back to grad school. Although he was extremely bright with broad interests in computing and cognitive science, his work that summer focused on some technical issues in the computational linguistics portions of our software; he didn't really get into the deeper aspects of the AI theory Intelligenesis was pursuing. Informally, many of us referred to Mark as "The Klingon" because of his mastery of the Klingon language. (For the non-nerds in the audience: Yeah, when they created the Klingons in Star Trek, they hired a linguist to design an actual language for them. Cool, huh?) Mark was involved in the translation of Hamlet into Klingon and didn't mind showing off his Klingon fluency to curious colleagues. Mark's Klingon was smooth and fluent but often seemed a bit odd because of his kind and soft-spoken nature -- personality-wise, at least on the surface, Mark was pretty far from a Klingon-ish guy. He also told us about a friend of his who raised his daughter bilingual English-Klingon: speaking to his daughter only in Klingon from birth, while his wife spoke to her in English.

Along the way Mark also mentioned to me a language called Lojban, which he said was based on predicate logic. He observed to me in passing that it might be easier for us to make our AI system understand Lojban than English. I agreed that it might be, if Lojban was more logically structured, but I reckoned this wasn't very practical, since no one on the team except Mark spoke any Lojban. Also, we were interested in creating a real AI incrementally, along a path that involved spinning off commercial apps -- and the commercial applications of a Lojban-speaking AI system seemed rather few.

Well, six and a half years later, Mark's suggestion has started to seem like a pretty good one. In my new AI project Novamente, we have progressed moderately far along the path of computational language understanding. Our progress toward powerful general AI has been painfully slow due to the team's need to pay rent and the lack of any funding oriented toward the grand AI goal, but for 2004 and part of 2003 the Novamente team and I had some funding to build some English language processing software -- and while we didn't build anything profoundly real-AI-ish, we used the opportunity to explore the issues involved in AI language processing in some depth.

The language processing system that we built is called INLINK and is described here. It doesn't understand English that well by itself, but it interacts with a human user, presenting alternate interpretations of each sentence typed into it, until the human verifies it's found a correct interpretation. The interactive process is slow and sometimes irritating but it ultimately works,
allowing English sentences to be properly interpreted by the AI system. We have plans to create a version of the INLINK system called BioCurator, aimed at biological knowledge entry -- this should allow the construction of a novel biology database containing formal-logic expressions representing biological knowledge of a much subtler nature than exists in current online bio resources like the Gene Ontology.

I've had a lot of doubts about the value of computational linguistics research for "real AI" -- there's a moderately strong argument that it's better to focus on perception, action and embodiment, and let the AI learn language as it goes along interacting with humans using its (real or simulated) body. On the other hand, there's also an argument that a certain degree of "cheating" may be helpful -- that building in some linguistic knowledge and facility may be able to accelerate the experiential-language-learning process. I've outlined this argument in an article called Post-Embodied AI.

The work on INLINK has clarified for me exactly what's involved in having an AI system understand English (or any other natural language). Syntax processing is tricky but the problems with it can be circumvented using an interactive methodology as we've done in INLINK; and eventually the system can learn from its errors (based on repeated corrections by human users) and make fewer and fewer mistakes. The result of INLINK is that English sentences are translated into probabilistic logical expressions inside Novamente's memory, which may then be interpreted, reasoned on, data-mined, intercombined, and yadda yadda yadda. Very nice -- but nasty issues of computational efficiency arise.

Novamente's probabilistic-inference module currently exists only in prototype form, but the prototype has proven capable of carrying out commonsense reasoning pretty well on a number of simple test problems. But there's a catch: for the reasoning process to be computationally tractable, the knowledge has to be fed to the reasoning module in a reasonably simple format. For instance, the knowledge that Ben likes the Dead Kennedys, has to be represented by a relationship something like

#likes( #Ben_Goertzel, #Dead_Kennedys)

where the notation #X refers to a node inside Novamente that is linked by a high-strength link to the WordNode/PhraseNode representing the string X. Unfortunately, if one types the sentence

"Ben likes the Dead Kennedys"

into INLINK, the Novamente nodes and links that come out are more complicated and numerous and less elegant. So a process called "semantic transformation" has to be carried out. This particular case is simple enough that this process is unproblematic for the current Novamente version. But for more complex sentences, the process is, well, more complex, and the business of building semantic transformations becomes highly annoying. One runs into severe issues with the fuzziness and multiplicity of preposition and verb-argument relationships, for example. As occurs so many times in linguistics and AI, one winds up generating a whole bunch of rules which don't quite cover every situation -- and one realizes that in order to get true completeness, so many complexly interlocking small rules are needed that explicitly encoding them is bound to fail, and an experiential learning approach is the only answer.

And this is where -- as I just recently realized -- Lojban should come in! Mark Shoulson was right back in 1998, but I didn't want to see it (urrrgghh!! what useful things are smart people saying to me now that I'm not accepting simply because I'm wrapped up in my own approaches?? why it is to hard to truly keep an open mind?? why is my information processing capacity so small??!! wait a minute -- ok -- this is just the familiar complaint that the limitations of the human brain are what make it so damn hard to build a superior brain. And the familiar observation that cutting-edge research has a way of making the researcher feel REALLY REALLY STUPID. People tell me I'm super-smart but while working on AI every day I come to feel like quite a bloody moron. I only feel smart when I re-enter the everyday world and interact with other people ;-p)

What if instead of making INLINK for English, we made it for Lojban (LojLink!)? Of course this doesn't solve all the problems -- Lojban is a constructed language based on formal logic, but it's not equivalent to formal logic; it allows ambiguity where the speaker explicitly wants it, otherwise it would be un-usable in practice. Semantic transformation rules would still be necessary to make an AI system understand Lojban. But the human work required to encode such transformations -- and the AI learning required to learn such transformations -- would clearly be one or two orders of magnitude less for Lojban.

Lojban isn't perfect... in my study of Lojban over the last week I've run up against the expected large number of things I would have designed differently, if I were building the language. But I have decided to resist the urge to create my own Lojban-ish language for AI purposes, out of respect for the several decades of work that have gone into "tuning" Lojban to make it more usable than the original version was.

In some respects Lojban is based on similar design decisions to the knowledge representation inside my Novamente AI Engine. For instance, in both cases knowledge can be represented precisely and logically, or else it can be represented loosely and associatively, leaving precise interpretation reliant on contextual factors. In Lojban loose associations are represented by constructs called "tanru" whereas in Novamente they're represented by explicit constructs called AssociativeLinks, or by emergent associations between activity-patterns in the dynamic knowledge network.

Next, it's worth noting that Lojban was created and has been developed with a number of different goals in mind -- my own goal, easier interfacing between humans and early-stage AGI's, being just one of them.

Some Lojbanists are interested in having a "culturally neutral" language -- a goal which, while interesting, means fairly little to me.

In fact I don't really believe it's possible -- IMO Lojban is far from culturally neutral, it embodies its own culture, a nerdy and pedantic sort of culture which has plusses and minuses. There is a Lojban term "malglico" which translates roughly to "damn English" or "fucking English" -- it refers to the tendency to use Lojban in English-like ways. This is annoying to Lojban purists but really doesn't matter to me. What I care about is being able to communicate in a way that is fluid and simple and natural for me, and easy for an early-stage AI to comprehend. If the best way to achieve this is through a malglico dialect of Lojban, so be it. If malglico interferes with the comprehensibility of Lojban by AI software, however, then I'm opposed to it.

I've printed up a bunch of materials on Lojban and started studying it seriously -- if I keep up with it then in 6 months or so I'll be a decent Lojbanist. Generally I'm not much good at learning languages, but that's mostly because it bores me so much (I prefer learning things with more of a deep intrinsic structure -- languages always strike me as long lists of arbitrary decisions, and my mind wanders to more interesting things when "I" try to force it to study them...). But in this case I have a special motivation to help me overcome the boredom....

If you want to try to learn Lojban yourself, the most useful resources I've found are:


If it does happen that we teach Novamente to speak Lojban before English then in order to participate in its "AI preschool" you'll need to know Lojban! Of course once it gets beyond the preschool level it will be able to generalize from its initial language to any language. But the preschool level is my focus at the moment -- since as I'm intensely aware, we haven't gotten there yet!

I remain convinced that with 2-3 years of concentrated single-focused effort by myself and a handful of Novamente experts (which will probably only be possible if we get some pure-AI-focused funding, alas), we can create a Novamente system with the intelligence and creativity and self-understanding of a human preschooler. But I'm trying really hard to simplify every aspect of my plan in this regard, just to be sure that no unexpected time-sinks come along. One advantage of NOT having had pure-AI-focused funding for the last few years is that the AI design has been refined an awful lot during this frustrating period. The decision to take a "post-embodied" approach to linguistics -- incorporating both experiential learning and hard-wiring of linguistic knoweldge -- is not a new one; that was the plan with Webmind, back in the day. But the idea of doing initial linguistic instruction and hard-wiring for Novamente in Lojban rather than English is a new one and currently strikes me as quite a good one.

Ah -- there's a bit of a catch, but not a big one. In order to do any serious "hard-wiring" of Lojban understanding into Novamente or any other AI system, the existing computational linguistics resources for Lojban need to be beefed up a bit. I describe exactly what needs to be done here. It seems to me there's maybe 3/4 man-years of work in making pure Lojbanic resources, and another year of work in making resources to aid in automated Lojban-English translation.

And another interesting related point. While in 1998 when Mark Shoulson first pointed Lojban out to me, I thought there were no practical commercial applications for a Lojban-based AI system, I've now changed my mind. It seems to me that an AI system with a functional Lojban language comprehension module and modest level of inferential ability would actually be quite valuable in the area of knowledge management. If a group of individuals were trained in Lojban, they could enter precise knowledge into a computer system very rapidly, and this knowledge could then be reasoned on using Novamente or other tools. This knowledge base could then be queried and summarized in English -- because processing simple English queries using a system like INLINK isn't very hard, and doing crude Lojban-English translation for results reporting isn't that hard either. In any application where some institution has a LOT of knowledge to encode and several years to do it, it may actually make sense to take a Lojbanic approach rather than a more standard approach. Here you'll find an overview of this approach to knowledge management, which I call LojLink.

One example where this sort of approach to knowledge encoding could make sense is bioscience -- I've long thought that it would be good to have every PubMed abstract entered into a huge database of bio knowledge, where it could then be reasoned on and connected with online experimental biology data. But AI language comprehension tools aren't really up to this task -- all they can do now is fairly simplistic "information extraction." We plan to use a bio-customized version of INLINK to get around this problem, but entering knowledge using INLINK's interactive interface is always going to be a bit of a pain. There's enough biology out there, and the rate of increase of bio knowledge is fast enough, that it makes sense to train a crew of bio knowledge encoders in Lojban, so that the store of bio knowledge can be gotten into computer-comprehensible form at maximum rate and minimum cost. Yes, I realize this sounds really weird and would be a hard idea to sell to venture capitalists or pharma company executives -- but that doesn't mean it doesn't make sense....

As another aside, there is some Lojban poetry on the Net but I haven't found much Lojban music. I like to sing & play the keyboard sometimes (in terms of vocal style, think Bob Dylan meets Radiohead); I'm considering doing some of my future lyrics in Lojban! True, few listeners would understand what I was talking about -- but I reckon that, in many cases, the verbal contents of lyrics aren't all that important -- what's important is the genuineness of feeling attached to them, which is achievable if the words have deep meaning to the singer, whether or not the listener can understand them. Of course, I have some lyrics that violate this rule and succeed at least a bit in communicating poetically (even a bit of transhumanist lyricism here and there -- e.g. "I've got to tell you something / your lonely story made me cry / I wish we all could breathe forever / God damn the Universal Mind"). But even so I think Lojbanic lyrics could really rock....

But -- wow -- how to fit learning a new language into my schedule? Urgggh!! Way too much to do. Fortunately I have a wife who says she's willing to learn this weird language along with me, which will make things much easier; it'd be trickier to learn a language with no one to speak to. But still ... every time something new like this comes up I'm confronted with the multiplicity of Bens in my head: each with different goals and priority rankings on their shared goals ... some of them saying "Yeah! You've got to do this!", others cautioning that it will siphon away the sometimes irritatingly small amount of time currently allocated to enjoying the non-intellectual aspects of human life in the Ben-iverse....

But "I" digress. Or do I?

Perhaps internal multiplicity and the falsehood of the unified "I" is a topic best saved for another blog entry. But yet, it does tie back into Lojban -- which I notice contains a single word for "I" just like ordinary languages. This is an area where I'm tempted to introduce new Lojbanic vocabulary.

I don't know what "I" am. I like the Walt Whitman quote "I contradict myself? Very well then, I contradict myself. I am large, I contain multitudes." Indeed, I do. In From Complexity to Creativity I explored the notion of subselves extensively. This notion should be explicitly embodied in language. You should be able to say "One of my subselves wants X" rather than "I want X" -- easily, via a brief linguistic expression, rather than a complicated multi-phrasal description. The distinction between "Some of my subselves want this very intensely" and "All of my subselves want this moderately strongly" should be compactly and immediately sayable. If these things were compactly and simply expressible in language, maybe we'd get out of the habit of thinking of ourselves as unities when we're really not. At least, I'm definitely not. (Just like I feel idiotic most of the time, then feel more clever when interacting with others; similarly, when I'm on my own I often feel like a population of sub-Bens with loosely affiliated goals and desires, and then I feel more unified when interacting with others, both because others view me as a whole, and because compared to other peoples' subselves, mine all cluster together fairly tightly in spite of their differences... (and then I'm most unified of all when I let all the goals drift away and dissolve, and exist as a single non-self, basking in the 1=0, at which point humanity and transhumanity and language and all that seem no more important than un ... but now I really digress!)). And in an appropriately designed language -- say, a subself-savvy extension of Lojban -- this paragraph would be a lot shorter and simpler and sound much less silly.

And this brings up a potentially very interesting aspect of the idea of teaching AI systems in odd constructed languages. My main motivation for thinking about using Lojban instead of English to teach Novamente is to simplify the semantic mapping process. But, it's also the case that English -- like all other natural languages -- embodies a lot of really irritating illusions ... the illusion of the unified self being one of them. Lojban now also happens to embody the illusion of the unified self, but this is a lot easier to fix in Lojban than in English, because of the simpler and more flexible structure of the Lojban language. I don't buy the strongest versions of the Sapir-Whorf hypothesis (though I think everyone should read Whorf's essay-collection Language, Thought and Reality), but clearly it's true that language guides cognition to a significant extent, and this can be expected to be true of AI's at least as much as of humans.

I can envision a series of extensions to Lojban being made, with the specific objective of encouraging AI systems learn to think according to desired patterns. Avoidance of illusions regarding self is one issue among many. Two areas where Lojban definitely exceeds English are ethics and emotion. English tends to be very confused in these regards -- look at the unnecessary ambiguities of the words "happy" and "good", for example. The current Lojban vocabulary doesn't entirely overcome these problems, but it does so significantly, and could be improved in these regards with modest effort.

Well, as I type these words, my son Zeb is sitting next to me playing "Final Fantasy" (yes my work-desk sits in the livingroom next to the TV, which is mostly used by the kids for videogames, except for their obsessive viewing of South Park... the new season just started, there are new episodes now, and did you know Mr. Garrison is now Mrs. Garrison??!!). As I look over at the manly-chested (scrawny, artistic little 11-year-old Zeb's favorite utterance these days: "Admire my manly chest or go down trying!"), womanly-faced heroes run through their somewhat bleak simulated landscape, and feel really intensely sick of the repetitive background music, I can't help but observe that, obsessed as he is with that game, I'm even more obsessed with my own "final fantasy." Or am I? One of my "I"'s is. Another one feels a lot more like playing the piano for an hour or so before bed, even though clearly working on AI for that time-interval would be more productive in terms of the long-term good of Ben and the cosmos. Or is playing music a while justified by the mental peace it brings, enabling clearer thinking about AI research later? How to ensure against self-delusion in judgments like that? Ah, by 38 years old I have devised an excellent set of mental tools for guarding against delusion of one subself by itself, or of one subself by others -- and these tools are frustratingly hard to describe in the English language! No worries -- the community of sub-Bens remains reasonably harmonious, though in the manner of strange attractors rather than fixed psychological arrangements. The chaos goes on.... (ranji kalsa ... ) ... the human chaos goes on, moving inevitably toward its own self-annihilation or self-transcendence ... and my committee of sub-Bens unanimously agrees that it's worth spending a lot of thought and effort to bias the odds toward the latter ...

13 comments:

Mentifex said...

Sub-Bens 1, 2, & 4, youse guys are not really going to learn Lojban, are you? And leave poor Sub-Ben 3 to work on Novamente all by himself? Don't do it, Bens! It would be what Nietzsche calls "eine Abfuehrung von allen Pflichten" -- a seduction away from all duties. Take Mentifex here for instance, a Usenet crank/crackpot/you-name it, but fluently able to cogitare, scribere et conversari in ancient Latin. So, do I write Mind.Forth to be fluent in Latin? I could, but I don't dare -- it would have too limited an audience. And who cares about how exact and precise Lojban is purported to be? A perfectly logical language? There ain't no such animal. While you, Ben, and you and you, Ben and Ben, are learning Lojban, Mentifex here is coding away on standard Mind.forth at http://mind.sourceforge.net/mind4th.html and experimental Mind.Forth at http://www.visitware.com/AI4U/mind4th.html. Just two days ago I inserted code for users to see when the AGI Mind "went live," so that installations may vie to host the longest-running, immortal artificial intelligence. The Mind.Forth code needs a lot of improvement; please stand by. Lojban? Harumph! Maybe also scale Mount Everest, or perch at the top of a redwood tree -- anything to avoid hard work on hard AGI.

Ben said...

Hey Mentifex... A couple points in response to your post.

Firstly, Lojban doesn't attempt to be perfectly logical -- it just claims to be (roughly) only as non-logical as necessary. Some ambiguity is necessary for effective communication, and logic is not the right tool for expressing all kinds of ambiguity. But natural languages have a degree of ambiguity going well beyond what's necessary for effective communication; and a degree of non-logicality going well beyond what's necessary for expressing semantically useful ambiguity....

Secondly, my dear goofball friend, my goal in studying Lojban isn't to avoid hard work on AGI -- the goal is to make the amount of work more manageable, understanding that it's gonna be damn hard anyway! An AGI should really be learning language not having language hard-wired into it. If we want to "cheat" and hard-wire in some language knowledge that will later be replaced by learned knowledge -- then why not be clever and hardwire in knowledge about a language that's more amenbable to hardwiring, because it's closer to program code and AI knowledge representations?

Anyway, learning Lojban won't take all of sub-Bens 1, 2 & 4, it will only take roughly 1/e of sub-Ben pi. So I wouldn't worry about it too much ;-)

Ben said...

This comment is a response to a comment Alex Richter made to me in an e-mail to the AGI e-mail list.

Alex noted that he had been working with Lojban for a while, but had now shifted to working with simple English, as in

http://simple.wikipedia.org/wiki/Main_Page


I looked at the simple wikipedia, and my response is that I don't believe that Alex has solved the problem of automated comprehension of simple English as exemplified in those texts.

Those sentences are easily parseable but it's semantic understanding not parsing that's the hard part of computational English processing.

For instance the Simple Wikipedia biology page contains the sentence


"
Biology also studies how organisms react with each other and the environment.
"


But the LogicalArgumentRelationshipDictionary we developed for use with Novamente NLP projects isolates 5 meanings of "with", and determining which one is meant here automatically is kind of a bitch.

The best approach we've found to this problem is simple probabilistic induction: look at which sense of "with" has been meant in semantically/syntactically similar senses in the past. This requires training data, though -- cases where humans have told the system the correct sense, as is provided via the logs of users of our INLINK interactive NLP system.

Of course for humans the choice of senses of prepositions is obtained via experiential grounding...

It's not that Lojban eliminates problems like this, it just elegantly reduces them to a smaller set of problems, mostly avoiding unnecessary problems of this nature....

Like I said, a tactical rather than strategic point, but tactics are important too...

Dennis Gorelik said...

If you spend time/money on design/teaching AI to understand Lojban ---
it would be just a waste of your resources. It has neither strategical nor tactical use.

Argumentation is here

Anonymous said...

Ben,
This is very exciting news! I've been lurking around your site ever since I read the mention of you in Yudkowski's Staring Into The Singularity. But to discover that you also have an interest in Lojban is a thrill. Just a note to encourage you in this. By the way, Robin Lee Powell pointed me in the direction of Supermemo software (http://www.supermemo.com/) which I run on my Palm OS computer for Lojban memorization. It is also available for desktop systems. It can greatly cut down the amount of time you have to spend to learn this language. I recommend it.

Ben said...

This is a response to Dennis Gorelik's comment...

Dennis:
The idea is not to teach an AI Lojban *instead of* English, but rather that teaching it Lojban might be a valuable *intermediary step*.

I'm confident that Novamente *can* learn NL, either via the intermediary of learning Lojban first, or not.

The question is merely of the fastest route of teaching, which may be different for an AI than for a human.

You say

"
If AI can learn Lojban --- it proves nothing from strong AI standpoint. There are a lot of VB, Pascal, C#, C++ compilers already. So what?
"

But those languages are programming languages, not languages for communicating commonsense knowledge about the world.

This comment suggests that you don't really understand what Lojban is.

Teaching an AI Lojban has the practical sense that Lojban can be used to interact with the system and teach it all sorts of useful commonsense information about how to interact, think and so forth, as well as about the human and natural world.

This knowledge can then be very helpful to the system in learning English, among other purposes.

I am quite *sure* that teaching a Lojban-fluent AI English would be a lot easier than teaching a nonlinguistic AI English. This is obvious.

Now, whether getting to English via Lojban is the optimal path, is a whole other issue. I don't claim to be sure about that. All I can say at this moment is that it seems to be a path well worth exploring further. Your arguments definitely have not convinced me otherwise; particularly your analogy between Lojban and programming languages seems quite ill-conceived. In fact Lojban is far more similar to natural languages in both intent, semantics and syntax than to any of the programming languages you mention.

Anonymous said...

In the sprit of being receptive towards advice from overlooked sources, I am posting anonymously.

In regards to INLINK, I suggest that you read "The Symbolic Species." Also, I suggest investigating the use of recurrence to strengthen temporal pattern recognition ( http://www.mathworks.com/access/helpdesk/help/toolbox/nnet/recur94.html ) Terrance Deacon puts forth several excellent theories that human language is so constructed to be easy for children to learn, that the order and type of data is paramount for learning, etc etc. I highly recommend at least browsing through the sections on language.

Ben said...

This is a response to Anonymous, who suggests that I read Terrence Deacon's work.

Yeah, I read that work many years ago; it was the subject of significant discussion at Webmind Inc. in 1999-2000.

I'm also a big fan of the book "Lingua ex Machina" by William Calvin and Derek Bickerton.

And looking back far further, Vygostky and Piaget, who had similar incremental experiential-learning ideas, though they lacked the neuroscience foundation of these more recent thinkers....

Regarding the Novamente AI project: If we were building an AI with an android human body and vaguely humanlike cognitive wetware, then adhering closely to knowledge about human development would make a LOT of sense.

As it is, we're building a system with a radically nonhuman hardware infrastructure and a radically nonhuman embodiment (the intended initial embodiment being a simulated agent in the AGI-SIM simulation world). So exploring radically nonhuman approaches to language acquisition seems reasonably appropriate....

As for recurrent neural nets, I coded a bunch of them in the mid-90's and found them to be damn slow and only moderately impressive in their capabilities. Not to say that the learning techniques haven't advanced since then, though. I'm sure they have, though I haven't followed that subfield closely.

I have done a lot of work in temporal pattern recognition (specifically, recognition of patterns in multiple coupled numerical time series) in the context of some practical narrow-AI applications. Recurrent neural nets are commonly used here and are certainly appropriate, but we have found ways of using genetic programming and the Bayesian Optimization Algorithm for this purpose, as well as greedy algorithms such as relim (a more efficient and simpler algorithm in the spirit of Apriori).

So my view is that temporal pattern recognition can be conducted using Novamente's generic learning operations (BOA supplanted by greedy learning where BOA is too time-consuming). This seems to be the case so far based on our experiments with time series pattern recognition.

What we have NOT done yet is to hook up this nice code for temporal pattern recognition in an AGI-SIM context to see how it works for learning experiential perception/action/cognition patterns. This is intended, and with luck will happen in the next few months -- the initial experiments with hooking Novamente up to AGI-SIM should happen within March or April.

I certainly understand that this kind of temporal-learning-based experiential learning is the essence of language acquisition and "learning to think." However, given the radically nonhuman architecture of Novamente, it seems that cheating by hardwiring in some language may be a significant helper to the experiential learning process. Not a necessity but perhaps a big tactical advantage. It is in this context that the use of Lojban is being conceptually explored.

Dennis Gorelik said...

Lojban is too far from natural languages

Ben said...

A response to Gorelik's second comment ("Lojban is too far from natural langauge")

Gorelik, you compare Lojban to programming languages like C++ ...

A family of Lojban speakers could go about their daily life communicating about their job, their dog, their dinner and their lovelife only in Lojban, without any substantial loss compared to doing so in English or another natural language. There would be some gain and some loss, just as in any choice among languages. For instance, Lojban is better than English for discussing emotions, but it may be worse in some other aspects.

I could translate any English poem, novel or textbook into Lojban and a Lojban-speaker could read it and understand the ideas, roughly as quickly as if they wer reading an English text.

It is just not possible to use a programming language in this way. The notion doesn't even make sense.

Lojban is a language for commonsense human communication about everyday things, whereas programming languages have a different purpose and consequently very different purposes.

What you seem to be missing is that while the SYNTAX of Lojban is predefined, the SEMANTICS is not.

Much of Lojban semantics is precisely defined, but so is much of NLP semantics (e.g. tense in Lojban is somewhat more precisely defined than in English, but not dramatically more so. Words like "me" and "you" are defined in Lojban exactly as precisely as in English... etc.)

Crucial aspects of Lojban semantics are NOT precisely defined, which is exactly what makes it possible to use Lojban for commonsense communication.

For instance, the tanru (word grouping)


cinse ke pencu xajmi ke'e


means roughly


sex funny-touch


where the ke-ke'e pair indicates parenthetical grouping, causing the "funny" and "touch" to be more grouped together...

This has the argument structure of "touch" but it doesn't specify the particular way in which funny, touching and sex are combined. To fully specify what this tanru means in a precise way, would require a longer expression. So in real Lojban usage, the meaning of this tanru has to be figured out informally from context.

For instance, commonsensically, in most contexts this tanru probably doesn't mean "any event in which sex is involved, and some kind of funny touching is also involved." Because this would cover cases in which A was touching B while B was making a funny face, and A was having sex with C.

In context it will generally be clear that this tanru means something like "tickling someone in a sexual way." But the point is that one doesn't have to spell out "tickling someone in a sexual way", one can leave the expression more ambiguous (using the above tanru) and leave it to the user to figure out the meaning from context.

This kind of thing -- ambiguity resolvable only via common contextual understanding between the speaker and listener -- is what makes Lojban a usable practical language.

This also means that semantic interpretation of Lojban poses a nontrivial AI/cog-sci problem.....

UItimately, I suspect the only practical way to solve the semantics problem is via experiential interactive learning -- i.e. embody your AI system in an environment, interact with it in this environment & talk to it about what it's doing and seeing.... Not via induction on examples of *language*, but via induction on examples of *socially deployed, pragmatically relevant language*.

This is true whether one uses Lojban, English, Klingon or Azerbaijani or whatever.

The only significant difference between using Lojban and English for this purpose is that in Lojban the semantic learning problems involved are EASIER.

The fact that Lojban syntax is precisely defined isn't really the main point. We can work around the trickiness of English syntax using an interactive approach as we've done with INLINK.

The main point is that Lojban presents fewer syntactic means for expressing significant semantic ambiguity. This makes an AI's "learning appropriate semantic mappings" job significantly easier.

How much easier is hard to quantify. Not a thousand times easier. Maybe 3-30 times easier?

Of course, we want to make an AI that can learn any language. But there's a bootstrapping problem here. If an AI can learn Lojban then we can use Lojban to teach it a lot of stuff, and it will then be able to pick up English quite rapidly, leveraging all the knowledge we've given it.

Note that the Novamente system supports a variety of approaches to NLP, everything from

a) completely code in all syntactic and semantic NLP rules and then let the system adapt them via experience

to

b) give the system no hard-wire linguistic knowledge, not even the knowledge that language exists -- let it learn everything from experience.

I am attracted more to b than to a, but I am even more attracted to finding an optimal middle ground. What's appealing about Lojban is that one can more easily find a middle ground that

-- requires only a quite small amount of hard-wiring

-- gives the system a big boost toward more effective experiential learning

Quite specifically, the main "ambiguity resolution" algorithm required for Lojban interpretation is simply taking the N elements in a tanru (see example above) and figuring out the intended precise logical relationship between them.

This is a lot simpler than the host of ambiguity resolution algorithms needed for successful English processing.

And, elegantly, it's a function that already exists in Novamente, in the form of an algorithm for "given a bunch of Novamente nodes/links joined by an Association relationship, search for precise probabilistic logical relationships between them."

In sum, the difference btw Lojban and English in terms of language learning is that in Lojban syntax is a non-problem (convenient but not critical) and semantics is a smaller and more elegant problem (very important IMO).

David Hart said...

The specialist Lojban knowledge encoder reminds me of Egan's intermediary species, created for the purpose of communication between very divergent species. :-)

Anonymous said...

A

Alice Bevan-McGregor said...

There are two sections of chapter 5 of the Lojban CLL that provide the perfect example of the difficulty of parsing meaning from English and provide clear examples of how Lojban avoids the ambiguity.

https://lojban.github.io/cll/5/3/

"That's a pretty little girls' school." — A classic worst-case scenario. One resolved primarily via spoken inflection, which is a no-go for parsing written text.

You can "cheat" in Lojban in the case where you have three predicates: you can use a particle word "bo" to denote pairings.

Example 3.7: ta cmalu nixli bo ckule — That is-a-small type-of (girl type-of school).

Example 3.8: ta cmalu bo nixli ckule — That is-a-(small type-of girl) type-of school.

This removes the ambiguity in a succinct way. You can also use more explicit "ke" and "ke'e" particles, as previously mentioned.

https://lojban.github.io/cll/5/5/

One "pretty little girls' school" example from that page:

Example 5.6) ta melbi ke cmalu nixli ckule [ke'e] — That is-a-pretty type-of ( ( little type-of girl ) type-of school ). (A "beautiful school" for "little girls".)

Automatic machine parsing illustrates the grouping graphically: http://www.lojban.org/camxes/?text=ta%20melbi%20ke%20cmalu%20nixli%20ckule (note "tanruUnit2").