Tuesday, October 14, 2014

The Neural Foundations of Complex Symbolic Thought


How can the brain, with its messy mix of neurons and glia spreading activation all over the place, give rise to the precise mathematical structures of symbolic reasoning?

This "symbolic / subsymbolic gap" has been a major puzzle at the center of cognitive science for decades, at least.

In a paper for the IJCNN conference in Beijing in July, I proposed a potential solution that -- while still speculative -- I believe has real potential for solving the issue.

The paper is linked here; and the abstract is as follows:



How Might the Brain Represent Complex Symbolic Knowledge?

Abstract—A novel category of theories is proposed, providing
a potential explanation for the representation of complex
knowledge in the human (and, more generally, mammalian)
brain. Firstly, a ”glocal” representation for concepts is suggested,
involving localized representations in a sparse network
of ”concept neurons” in the Medial Temporal Lobe, coupled
with a complex dynamical attractor representation in other
parts of cortex. Secondly, it is hypothesized that a combinatory
logic like representation is used to encode abstract
relationships without explicit use of variable bindings, perhaps
using systematic asynchronization among concept neurons to
indicate an analogue of the combinatory-logic operation of
function application. While unraveling the specifics of the
brain’s knowledge representation mechanisms will require data
beyond what is currently available, the approach presented
here provides a class of possibilities that is neurally plausible
and bridges the gap between neurophysiological realities and
mathematical and computer science concepts.






Note that this is a hypothesis about brains, and potentially a design principle for closely brain-like AGI systems -- but not a statement about, for example, the OpenCog AGI system, which implements symbolic thought more directly.   However, there are certainly analogies with things that happen inside OpenCog.  OpenCog has explicit symbolic representation (analogous to concept neurons, very roughly) and also subsymbolic representation from which symbolic-like representations may emerge; and the design intention of OpenCog is that these two kinds of representations can work together.   The specific mechanisms of this interaction are quite different in OpenCog from what I hypothesize to take place in the brain, but, on the level of the cognitive processes emerging from these systems at the highest levels, there may not be a large difference.

Wednesday, October 01, 2014

Is Physics Information Geometry on Causal Webs? (Speculations Toward Grand Unification)

I mentioned recently to my dad that I was fiddling around in my (egregiously nonexistent) "spare time" with some ideas regarding quantum gravity and he said "I would question the sanity of someone as busy as you trying to unify physics in his spare time."   Fair enough….

However, I think it's almost a social necessity these days to have one's own eccentric approach to grand unified physics, isn't it?   Admittedly, there are hundreds or maybe thousands of different approaches out there -- but still, if you don't have one of your own, you're not going to be invited into any of the really fancy parties….

And more importantly (I don't have time for those parties anyway) -- sometimes the mind's gotta do what the mind's gotta do.  For months now I've been plagued with some ideas about how to unify physics using information geometric structures defined over spacetime-like discrete structures, and I just HAD to write at least some of them down…. 

Today is a holiday here in Hong Kong (the anniversary of the founding of the People's Republic of China, which the Hongkongese are celebrating with their hugest political protests every), so along with taking a dramatic 5 hour hike with Ruiting (up a beautiful rocky stream rather than along a trail, with some great places to swim & climb en route; but the last 1.5 hours were in the dark which was a bit treacherous), I took some time to slightly polish some of my earlier scribblings on my half-baked but IMO tantalizing thoughts….

Physics as Information Geometry on Causal Webs


Rather than packing a huge load of speculative technical ideas into a blog post, I've put together a concept paper summarizing the direction of my thinking -- click here to read the paper in PDF form!.

For those  too lazy, busy or uncoordinated to point and click on the PDF, the title and abstract are pasted here:



Physics as Information Geometry on Causal Webs: 
Sketch of a Research Direction

A high-level outline is given, suggesting a research direction aimed at unifying the Standard Model with general relativity within a common information-based framework.   Spacetime is modeled spacetime using discrete ``causal webs'', defined as causal sets that have ternary rather than binary directed links, and a dynamic in which each ternary link propagates local values from its sources to its target using multiplication in an appropriate algebra.   One then looks at spaces of (real and complex) probability distributions over the space of ``causal web histories.''   One can then model dynamics as movement along geodesics in this space of probability distributions (under the Fisher-Rao metric).  

The emergence of gravitation in this framework (as a kind of ``entropic force'') is derived from Matsuoka's work founding General Relativity in information geometry; the emergence of quantum theory is largely implicit in work by Goyal and others founding the basic formalism of quantum states, measurements and observables in information geometry.   It is suggested that these various prior works can be unified by viewing quantum theory as consequent from information geometry on complex-valued probability distributions; and general relativity as consequent from the geometry of associated real probability distributions.  Further, quantum dynamics is known to be derivable from the correspondence principle and basic properties of classical mechanics; but the latter is an approximation to general relativity -- which as Matsuoka has shown is information-geometric, thus strongly suggesting that quantum dynamics also can be seen as emergent from information geometry.   It is hypothesized that the Standard Model, beyond basic quantum mechanics, could potentially be obtained from this approach via appropriate choice of the ``local field'' algebra propagated within the underlying causal webs. 

In addition to mathematical elegance, this approach to physics unification has the conceptual advantage of highlighting the parallels between physical dynamics and mental inference (given the close ties between Bayesian inference and information geometry).




Hypergraphs, Hypergraphs Everywhere ...


If you're familiar with the OpenCog AGI architecture (which I've played a leading role in designing) you may note a (maybe not so) peculiar similarity between the OpenCog design and the approach to physics proposed in the above.  In both cases one starts with a hypergraph -- OpenCog's Atomspace, which is a pretty unstructured hypergraph; versus a causal web, that is a hypergraph with a specific structure comprised of ternary directed links.  And in both cases one has nonlinear dynamics flowing across the hypergraph (though again, of fairly different sort).    And then, in both cases, it is posited that {\it emergent structures} from these hypergraph dynamics are key to giving rise to the important structures and dynamics.

Of course, this similarity may reflect nothing more than -- this is the way Ben likes to model things!   But I'd like to think there's something deeper going on, and that by modeling minds and the physical world using similar formalisms, one can more easily think about the relation between mind and matter.

Along these lines -- but even a smidge wackier -- thorough readers of this blog will note that some of these ideas were referenced in my recent blog post on morphic fields, psi and physics.   Of course the concepts in these two posts are somewhat independent: the physics ideas given here could be correct even if psi and morphic fields don't really exist; and my analysis of morphic fields in terms of pattern completion and surprisingness could be correct even if the right solution to unified physics involves utterly different ideas than the ones outlined here.  However, it's fair to say that these various concepts evolved together.  My thoughts about morphic fields and unified physics have definitely shaped each other, for whatever that's worth.

While I think the ideas outlined in the  document I linked here make sense, I'm under no illusion about how much work it would be to fill in the gaps in such a complex line of thinking.  Furthermore, some of the gap-filling might end up requiring the creation of substantial new ideas (though it's not obvious this is the case - it could just be a matter of crunching through relatively straightforward math).   I'm also under no illusion that I have time to pursue a lot of difficult physics calculations right now, nor that I will in the near future.   I'll have time to work on this in spare-time bits and pieces, for whatever that's worth.

However, I'd love to collaborate with someone on working out details of some or all of these ideas.  Any physics grad students out there with a high risk tolerance and a penchant for weird ideas, please consider!

Elsewise, I may get to working out the details (or trying to) in a few years when my task list clears up a bit.  AGI is going to remain the priority of the portion of my existence devoted to research until it's compellingly achieved and takes over its own development -- but to keep my creativity flowing (and not go nuts) I need to spend a certain percentage of time thinking about other areas of science.   That's just how my freaky little human brain works...