"Conceptual Spaces" and AGI

One of my AI collaborators from the late 1990s, Alexandru Czimbor, recently suggested I take a look at Peter Gardenfors book "Conceptual Spaces."

I read it and found it interesting, and closely related to some aspects of my own AGI approach ... this post contains some elements of my reaction to the book.

Gardenfors' basic thesis is that it makes sense to view a lot of mind-stuff in terms of topological or geometrical spaces: for example topological spaces with betweenness, or metric spaces, or finite-dimensional real spaces. He views this as a fundamentally different mind-model than the symbolic or connectionist perspectives we commonly hear about. Many of his examples are drawn from perception (e.g. color space) but he also discusses abstract concepts. He views both conceptual spaces and robust symbolic functionality as (very different) emergent properties of intelligent systems. Specific cognitive functions that he analyzes in terms of conceptual spaces include concept formation, classification, and inductive learning.

About the Book Itself

This blog post is mainly a review of the most AGI-relevant ideas in Gardenfors book, and their relationship to my own AI work ... not a review of his book. But I'll start with a few comments on the book as a book.

Basically, the book reads sorta like a series of academic philosophy journal papers, carefully woven together into a book. It's carefully written, and technical points are elucidated in ordinary language. There are a few equations here and there, but you could skip them without being too baffled. The pace is "measured." The critiques of alternate perspectives on AI strike me as rather facile in some places (more on that below), and -- this is a complaint lying on the border between exposition and content -- there is a persistent unclarity regarding which of his ideas require a dimensional model of mind-stuff, versus which merely require a metric-space or weaker topological model. More on the latter point below.

If you're interested in absorbing a variety of well-considered perspectives on the nature of the mind, this is certainly a worthwhile book to pay attention to. I'd stop short of calling it a must-read, though.

Mindspace as Metric Space

I'll start with the part of Gardenfors thesis that I most firmly agree with.

I agree that it makes sense to view mind-stuff as a metric space. Percepts, concepts, actions, relationships and so forth can be used as elements of a metric space, so that one can calculate distances and similarities between them.

As Gardenfors points out, this metric structures lets one do a lot of interesting things.

For instance, it gives us a notion of between-ness. As an example of why this is helpful, suppose one wants to find a way of drawing conclusions about Chinese politics from premises about Chinese individual personality. It's very helpful, in this case, to know which concepts lie in some sense "between" personality and politics in the conceptual metric space.

It also lets us specify the "exemplar" theory of concepts in an elegant way. Suppose that we have N prototypes, or more generally N "prototype-sets", each corresponding to a certain concept. We can then assign a new entity X to one of these concepts, based on which prototype or prototype-set it's closest to (where "close" is defined in terms of the metric structure).

Mindspace as Dimensional Space

Many of Gardenfors ideas only require metric space, but others go further and require dimensional space -- and one of my complaints with the book is that he's not really clear on which ideas fall into which category.

For instance, he cites some theorems that if one defines concepts via proximity to prototypes (as suggested above) in a dimensional space, then it follows that concepts are convex sets. The theorem he gives holds in dimensional spaces but it seems to me this should also hold in more general metric spaces, though I haven't checked the mathematics.

This leads up to his bold and interesting hypothesis that natural concepts are convex sets in mindspace.

I find this hypothesis fascinating, partly because it ties in with the heuristic assumption made in my own Probabilistic Logic Networks book, that natural concepts are spheres in mindspace. Of course I don't really believe natural concepts are spheres, but this was a convenient assumption to make to derive certain probabilistic inference formulas.

So my own suspicion is that cognitively natural concepts don't need to be convex, but there is a bias for them to be. And they also don't need to be roughly spherical, but again I suspect there is a bias for them to be.

So I suspect that Gardenfors hypothesis about the convexity of natural concepts is an exaggeration of the reality -- but still a quite interesting idea.

If one is designing a fitness function F for a concept-formation heuristic, so that F(C) estimates the likely utility of concept C, then it may be useful to incorporate both convexity and sphericality as part of the fitness function.

Conceptual Space and the Problem of Induction

Gardenfors presents the "convexity of natural concepts" approach as a novel solution to the problem of induction, via positing a hypothesis that when comparing multiple concepts encapsulating past observations, one should choose the convex concepts as the basis for extrapolation into the future. This is an interesting and potentially valuable idea, but IMO positing it as a solution to the philosophical induction problem is a bit peculiar.

What he's doing is making an a priori assumption that convex concepts -- in the dimensional space that the brain has chosen -- are more likely to persist from past to future. Put differently, he is assuming that "the tendency of convex concepts to continue from past into future",
a pattern he has observed during his past, is going to continue into his future. So, from the perspective of the philosophical problem of induction, his approach this still requires one to make a certain assumption about some properties of past experience continuing into the future.

He doesn't really solve the problem of induction -- what he does is suggest a different a priori assumption, a different "article of faith", which if accepted can guide be used to guide induction. Hume (when he first posed the problem of induction) suggested that "human nature" guides induction, and perhaps Gardenfors' suggestion is part of human nature.

Relating Probabilistic Logic and Conceptual Geometry

Gardenfors conceives the conceptual-spaces perspective as a radically different alternative to
the symbolic and subsymbolic perspectives. However, I don't think this is the right way to look at it. Rather, I think that

1. a probabilistic logic system can be considered as a metric space (and this is explained in detail in the PLN book)
2. either a probabilistic logic system or a neural network system can be projected into a dimensional space (using dimensional embedding algorithms such as developed by Haren and Koren among others, and discussed on the OpenCog wiki site)

Because of point 1, it seems that most of Gardenfors' points actually apply within a probabilistic logic system. One can even talk about convexity in a general metric space context.

However, there DO seem to be advantages to projecting logical knowledge bases into dimensional spaces, because certain kinds of computation are much more efficient in dimensional spaces than in straightforward logical representations. Gardenfors doesn't make this point in this exact way, but he hints at it when he says that dimensional spaces get around some of the computational problems plaguing symbolic systems. For instance, if you want to quickly get a list of everything reasonably similar to a given concept -- or everything along a short path between concept A and concept B -- these queries are much more efficiently done in a dimensional- space representation than in a traditional logic representation.

Gardenfors points out that, in a dimensional formulation, prototype-based concepts correspond to cells in Voronoi or generalized Voronoi tesselations. This is interesting, and in a system that generates dimensional spaces from probabilistic logical representations, it suggests a nice concept formation heuristic: tesselate the dimensional space based on a set of prototypes, and then create new concepts based on the cells in the tesselation.

This brings up the question of how to choose the prototypes. If one uses the Harel and Koren embedding algorithm, it's tempting to choose the prototypes as equivalent to the pivots, for which we already have a heuristic algorithm. But this deserves more thought.

Summary

Gardenfors' book gives many interesting ideas, and in an AGI design/engineering context, suggests some potentially valuable new heuristics. However its claim to have a fundamentally novel approach to modeling and understanding intelligence seems a bit exaggerated. Rather than a fundamentally disjoint sort of representation, "topological and geometric spaces" are just a different way of looking at the same knowledge represented by other methods such as probabilistic logic. Probabilistic logic networks are metric spaces, and can be projected into dimensional spaces; and the same things are likely true for many other representation schemes as well. But Gardenfors gives some insightful and maybe useful new twists on the use of dimensional spaces in intelligent systems.