This is just some notes-to-myself type rambling about declarative and episodic memory and reasoning ... stuff I'm thinking through in the back of my mind related to some ongoing OpenCog detailed-design work....
It occurred to me last week that inference control (the control of which logical inference steps to take, in trying to resolve some question using reasoning based on knowledge) has a lot in common with the decisions a person makes about how to live their life --- both over the course of their whole lifetime, and in various specific contexts.
Further, I think this analogy may be important in terms of guiding the interaction between semantic (declarative) and episodic memory. That is -- I think that, in many cases, real-life or imagined episodes stored in episodic memory may serve as high-level structural templates for inference control...
At a very crude level, the analogy I see is: both an inference process aimed at resolving a question, and a series of life-decisions aimed at navigating a certain everyday situation, are concerned with achieving certain goals in a limited time-frame, using limited resources, and via a series of choices, where each choice made guides the set of choices that are next available.
At a more precise level, what I also see is that: both in inference control and in real-life everyday episodic human decision-making, the "fitness landscape" is such that it is a reasonably useful strategy to iteratively focus attention on local regions of decision-space, each of which can
So, in the inference context, one can break one's inference goal into subgoals in multiple ways, where crudely exploring each way of breaking the goal into subgoals may take 1/10 or 1/500 the amount of time one has available for the inference. Thoroughly exploring any one way of breaking the goal into subgoals may then take longer -- indeed it may take as much time as one has.
In the episodic context, for instance, if one is unsure what career to choose or who to marry, one can crudely explore a certain job or a certain potential mate in 1/5 or 1/500 the total amount of time one has to choose a career or mate. On the other hand, thoroughly exploring and optimizing the possibilities offered by a given job or a given mate, if the choice is not a terrible one, may take as much time as one has.
So in both cases one carrying out a sequence of actions over time, in a context where the available actions depend on the actions one has taken previously -- and in both cases one heuristically has time to crudely explore maybe dozens, hundreds or thousands of local regions of action-space, before one's time runs out ... but one has time to thoroughly explore only a small handful of local regions of action-space, before one's time runs out...
In this sort of context, it seems, a reasonable approach will often be to:
In part this is just straightforward "exploration versus exploitation" stuff.
For instance, in the everyday life context: When young, one should try out many different jobs and date many different people, and try to understand what one can about the landscape. But then once one gets middle-aged, the time has often come to pick a single mate or a single career area and focus on that. The logic behind this is: In the years one has on Earth, one probably only has time to thoroughly explore and become great at a small number of careers, and to develop deep love relationships (or build families with) a small number of partners. However, some careers and some mates will suit one better. So given the nature of the fitness landscape, the best strategy is to look around a bit till one finds something that seems about as good as one is going to find before one gets too close to the end, and then at a certain point pick something "good enough" and go with it. A review of this sort of process in the mate-selection context is given in this article.
In inference one has the same basic issue. Suppose one wants to figure out X so that X is a person and X lives in China and X knows both p-adic analysis and particle physics. But suppose one doesn't have much time to carry out the search. One option is to look at online publications in those areas, and check out which papers have authors based in China. Another option is to look at Chinese university websites and the listing of professors and their publications. Obviously it's a wrong idea to choose only one approach and pursue it solely, unless one is very, very short on time. Instead it makes more sense to attempt each approach a bit and see which one is more promising. This is just "exploration versus exploitation."
But the nature of inference, and the nature of life-decisions, is that one has a series of exploration-versus-exploitation type choices, where the choices that one is presented with depend on the choices one has made previously .. and where exploring each choice often take an amount of time that is meaningful but not humongous relative to one's total available time.
The same sort of structure applies to social decision-making in contexts briefer and less consequential than choosing who to marry: for instance, figuring out how to entertain a specific person on a date, or figuring out how to get ahead in a specific company. In each of these cases there is a limited amount of time, a series of sequential decisions to make, and a situation where one can explore a bunch of options roughly but very few options thoroughly.
An interesting question, then, is how much the analogy between inference-control decisions and everyday-life decisions helps a human-like mind in its thinking. Are we actually helped by being able to consider our inferences as being like stories?
A typical story has a beginning, middle and end -- where the beginning is about setting out context and making clear what possibilities exist, the middle is about exploring some particular possibility in depth (typically with great uncertainty about whether this possibility will yield a good result or not), and the end is about what happens from exploring this particular possibility (which ideally should have some surprising aspect, whether the exploration is successful or not). A typical inference process will have the same sort of beginning, middle and end ... though it may be a bit more like a big fat epic Russian novel, with multiple possibilities, involving different characters, explored in depth during the middle section.
What does seem likely is that the brain re-uses the same mechanisms it uses for managing stories in episodic memory, for managing memories and patterns of inference control. Evolutionarily, it is not clear to me whether sophisticated episodic memory came before sophisticated inference or not. Perhaps the two co-evolved.
Using similar mechanisms for controlling inference and guiding episodic memory and everyday-life decision-making, should ease "cognitive synergy" between declarative reasoning and episodic recollection. When declarative reasoning gets stuck, it can be helped out via recollection or simulation of episodes; and vice versa.
For instance, suppose a robot needs to figure out how to amuse a certain person. Episodic memory might give it examples of things that have amused that person, or associated people, in prior situations. Declarative reasoning might note that the person has a Pearl Jam T-shirt on, and might then tell the system to look for funny statements involving rock music. The same goal can be explored both via episodic mind's-eye simulations, and via logical reasoning based on general background knowledge. And the overlap can recurse. For instance, if logic tells the system to look for funny statements involving rock music, episodic memory search might come up with specific past situations in which funny statements involving rock music have been uttered. If episodic memory digs up certain people closely associated with the person in question, logic might come up with some clever conclusions regarding what would have amused these associates. Etc.
This sort of cross-memory-mode helping-out would seem to be eased by using the same sort of representation for both processes.
This is interesting to me in terms of our next phase of OpenCog work, because we need to figure out both the best way to represent real-life episodes from a robot's everyday life in the system's memory, and the best way to represent abstractions from probabilistic-logic inference chains. What this rambling, musing discussion makes clear is that we want to use essentially the same representation for both cases. This way the commonality of the decision-processes involved in the two cases can be manifested in the software, and the ease of achieving episodic-declarative cognitive synergy will be maximized.
It occurred to me last week that inference control (the control of which logical inference steps to take, in trying to resolve some question using reasoning based on knowledge) has a lot in common with the decisions a person makes about how to live their life --- both over the course of their whole lifetime, and in various specific contexts.
Further, I think this analogy may be important in terms of guiding the interaction between semantic (declarative) and episodic memory. That is -- I think that, in many cases, real-life or imagined episodes stored in episodic memory may serve as high-level structural templates for inference control...
At a very crude level, the analogy I see is: both an inference process aimed at resolving a question, and a series of life-decisions aimed at navigating a certain everyday situation, are concerned with achieving certain goals in a limited time-frame, using limited resources, and via a series of choices, where each choice made guides the set of choices that are next available.
At a more precise level, what I also see is that: both in inference control and in real-life everyday episodic human decision-making, the "fitness landscape" is such that it is a reasonably useful strategy to iteratively focus attention on local regions of decision-space, each of which can
- be explored within reasonable accuracy in, say, 1-3 orders of magnitude less time than is available for the overall process
- be explored more thoroughly, yielding generally better results, in the same order of magnitude of time as is available for the overall process
So, in the inference context, one can break one's inference goal into subgoals in multiple ways, where crudely exploring each way of breaking the goal into subgoals may take 1/10 or 1/500 the amount of time one has available for the inference. Thoroughly exploring any one way of breaking the goal into subgoals may then take longer -- indeed it may take as much time as one has.
In the episodic context, for instance, if one is unsure what career to choose or who to marry, one can crudely explore a certain job or a certain potential mate in 1/5 or 1/500 the total amount of time one has to choose a career or mate. On the other hand, thoroughly exploring and optimizing the possibilities offered by a given job or a given mate, if the choice is not a terrible one, may take as much time as one has.
So in both cases one carrying out a sequence of actions over time, in a context where the available actions depend on the actions one has taken previously -- and in both cases one heuristically has time to crudely explore maybe dozens, hundreds or thousands of local regions of action-space, before one's time runs out ... but one has time to thoroughly explore only a small handful of local regions of action-space, before one's time runs out...
In this sort of context, it seems, a reasonable approach will often be to:
- Start out by trying a bunch of different stuff, gaining information about "where in the space of possibilities a good answer may lie"
- Then, when one's time starts to run out, one should pick some of the best options one has found (maybe just one of them) and explore it more deeply.
In part this is just straightforward "exploration versus exploitation" stuff.
For instance, in the everyday life context: When young, one should try out many different jobs and date many different people, and try to understand what one can about the landscape. But then once one gets middle-aged, the time has often come to pick a single mate or a single career area and focus on that. The logic behind this is: In the years one has on Earth, one probably only has time to thoroughly explore and become great at a small number of careers, and to develop deep love relationships (or build families with) a small number of partners. However, some careers and some mates will suit one better. So given the nature of the fitness landscape, the best strategy is to look around a bit till one finds something that seems about as good as one is going to find before one gets too close to the end, and then at a certain point pick something "good enough" and go with it. A review of this sort of process in the mate-selection context is given in this article.
In inference one has the same basic issue. Suppose one wants to figure out X so that X is a person and X lives in China and X knows both p-adic analysis and particle physics. But suppose one doesn't have much time to carry out the search. One option is to look at online publications in those areas, and check out which papers have authors based in China. Another option is to look at Chinese university websites and the listing of professors and their publications. Obviously it's a wrong idea to choose only one approach and pursue it solely, unless one is very, very short on time. Instead it makes more sense to attempt each approach a bit and see which one is more promising. This is just "exploration versus exploitation."
But the nature of inference, and the nature of life-decisions, is that one has a series of exploration-versus-exploitation type choices, where the choices that one is presented with depend on the choices one has made previously .. and where exploring each choice often take an amount of time that is meaningful but not humongous relative to one's total available time.
The same sort of structure applies to social decision-making in contexts briefer and less consequential than choosing who to marry: for instance, figuring out how to entertain a specific person on a date, or figuring out how to get ahead in a specific company. In each of these cases there is a limited amount of time, a series of sequential decisions to make, and a situation where one can explore a bunch of options roughly but very few options thoroughly.
An interesting question, then, is how much the analogy between inference-control decisions and everyday-life decisions helps a human-like mind in its thinking. Are we actually helped by being able to consider our inferences as being like stories?
A typical story has a beginning, middle and end -- where the beginning is about setting out context and making clear what possibilities exist, the middle is about exploring some particular possibility in depth (typically with great uncertainty about whether this possibility will yield a good result or not), and the end is about what happens from exploring this particular possibility (which ideally should have some surprising aspect, whether the exploration is successful or not). A typical inference process will have the same sort of beginning, middle and end ... though it may be a bit more like a big fat epic Russian novel, with multiple possibilities, involving different characters, explored in depth during the middle section.
What does seem likely is that the brain re-uses the same mechanisms it uses for managing stories in episodic memory, for managing memories and patterns of inference control. Evolutionarily, it is not clear to me whether sophisticated episodic memory came before sophisticated inference or not. Perhaps the two co-evolved.
Using similar mechanisms for controlling inference and guiding episodic memory and everyday-life decision-making, should ease "cognitive synergy" between declarative reasoning and episodic recollection. When declarative reasoning gets stuck, it can be helped out via recollection or simulation of episodes; and vice versa.
For instance, suppose a robot needs to figure out how to amuse a certain person. Episodic memory might give it examples of things that have amused that person, or associated people, in prior situations. Declarative reasoning might note that the person has a Pearl Jam T-shirt on, and might then tell the system to look for funny statements involving rock music. The same goal can be explored both via episodic mind's-eye simulations, and via logical reasoning based on general background knowledge. And the overlap can recurse. For instance, if logic tells the system to look for funny statements involving rock music, episodic memory search might come up with specific past situations in which funny statements involving rock music have been uttered. If episodic memory digs up certain people closely associated with the person in question, logic might come up with some clever conclusions regarding what would have amused these associates. Etc.
This sort of cross-memory-mode helping-out would seem to be eased by using the same sort of representation for both processes.
This is interesting to me in terms of our next phase of OpenCog work, because we need to figure out both the best way to represent real-life episodes from a robot's everyday life in the system's memory, and the best way to represent abstractions from probabilistic-logic inference chains. What this rambling, musing discussion makes clear is that we want to use essentially the same representation for both cases. This way the commonality of the decision-processes involved in the two cases can be manifested in the software, and the ease of achieving episodic-declarative cognitive synergy will be maximized.
10 comments:
[Note: When I refer to “consolidation” below, I’m not using the term as it’s used in conventional memory theories, but rather in its simple/common sense.]
The representations/functions are in fact encoded within the same network. Imagine that we’ve got a combinatorial array of sorts where we can tally up the instances of any particular pattern of stimuli as they occur. I know that may sound naive in terms of memory capacity, but let’s take for granted for the moment that we have a practical way of implementing this array.
The basic idea is that the pattern instances which are sufficiently similar get consolidated into abstractions requiring relatively little storage space. These abstraction nodes, or whatever you want to call them, amount to assertions of the general form ‘feature (x) is present’. The activation/selection of such a node by present conditions propagates implications to related nodes: ‘if (x) is present then (y) is present with probability p and (z) is absent with probability q.’ This is of course a simplification, but getting on with it…
Think of the array as an environment where the primary selection pressure (on representations) is uniqueness. The most redundant patterns get consolidated, mainly because the implications that propagate from them are insufficiently unique; The ‘presence’ of one minor pattern variant doesn’t imply (or mean) anything consequentially different than any other minor variant in that ‘node’.
There are also peripheral/contextual connections among nodes which allow for more subtle conditioning of selection, and serving as ‘extra storage space’ in some senses. You can picture the redundant instances ‘over-writing’ onto the same space again and again, of course impacting its representational strength (historical consistency of coincidence). The most unique patterns (episodes), however, ‘survive’ as distributed encodings (probably mostly among those ‘contextual’ connections) specifically because they avoid this over-writing (and consequent consolidation).
While episodes are typically composed of familiar sub-patterns (entities and events), they represent a novel combination of them in the history of the individual. The novelty itself is often sufficient in its impact on predictability as an economic variable, to warrant a boost in priority/efficacy for encoding the trace. Sufficient cues, literally reminders, evoke re-activation of the distributed representation through that subtle selection conditioning I mentioned above. It is of course not invoked in the subset of the network representing what's literally 'present', but more in the subnetwork(s) of what is explicitly non-present. In the "mind's eye", as you said.
I could go on, but that’s the rough outline, anyway.
Zed, your terminology is a bit different from what I habitually use, but everything you say utterly makes sense to me and actually it describes what's supposed to happen inside OpenCog. We use weighted/labeled hypergraphs rather than "combinatorial arrays", but obviously one can also represent a hypergraph as a set of sparse arrays if one wishes...
Ben, pleases pardon my home-grown terminology. A complete lack of formal education in these matters may be to blame. I'm basically reverse-engineering from the neurosciences side.
In my effort to simplify that explanation, I inadvertently conveyed the “sparse arrays” concept as if it were my implementation approach, which is inaccurate. I actually had to look up hypergraphs, and while I don't entirely grok the formalism at first glance, it intuitively seems that I’ve reinvented something similar to that wheel. No time to analyze that properly, for now.
The method I’m working toward does something more akin to lumping all loci of the phase-space together, so long as their definitions don’t intrinsically contradict, then allowing stimuli to effectively ‘refute’ the (potential) correlations as experience progresses. The algorithm is a work in progress.
Thanks, by the way, for a bit of validation. An outsider such as myself has keep in mind that I could still turn out to be some Dunning–Kruger trainwreck. ;)
Have you thought about Ant colony optimization algorithms
Actually, my 'clarification' was pretty terrible. Anyway, I'll have that paper out soon and it should explain sufficiently.
This is my first time i visit here. I found so many interesting stuff in your blog especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the enjoyment here keep up the good work Feel free to visit my website; 야설
I admire this article for the well-researched content and excellent wording. I got so involved in this material that I couldn’t stop reading. I am impressed with your work and skill. Thank you so much. Feel free to visit my website;
한국야동
My brother suggested I would possibly like this web site. He used to be entirely right. This put up actually made my day. You can not consider just how much time I had spent for this information! Thanks! Feel free to visit my website;
국산야동
It's late finding this act. At least, it's a thing to be familiar with that there are such events exist. I agree with your Blog and I will be back to inspect it more in the future so please keep up your act. Feel free to visit my website;
일본야동
Great post! I am actually getting ready to across this information, is very helpful my friend. Also great blog here with all of the valuable information you have. Keep up the good work you are doing here. Feel free to visit my website; 일본야동
Post a Comment