Eldan Goldenberg's lab notebook

Does it still qualify as learning if the behavior changes, but does not necessarily improve? For instance, I've seen classifiers latch on to incidental information in a dataset. For your work, does it still qualify as learning, even if it doesn't learn what you want it to?

I would argue that if there isn't an improvement in behaviour then it's not learning. This definition does have a major drawback though, which is that "improvement" is in the eye of the beholder.

However, in some cases "improvement" can be defined more rigourously than that. For a biological organism I would argue that anything which increases the chances of passing genes on to later generations is an improvement—this may be difficult to measure but I think it's a meaningful concept anyway—and in both of our research the goals we set for the systems we're building are clearly defined by us as the designers.

Now I'm wondering what to call behaviour that is systematically changed by information received by an agent, without being an improvement. Clearly this is still of interest, if only so we can learn how to stop it from happening....

Hmm. How would I define learning?

In my lab, using the kind of stimuli we do, we think about it as the acquisition of expectations. A change in behavior is needed to measure it - a change in attentiveness, for the young ones, or a change in mean value of Likert scale ratings for the adults - but the learning itself we take to be internal. If I went another level down I would say I conceptualize the learning as changes in neural connectivity - so I guess that's a change of form, insofar as neurons change shape when this happens, and a change of function. The behavior, to me, is an approximator of what's going on at this lower level.

I think some of my problems with your definition stem from my developmental training here - it would seem that by your criteria, unless you mean something different by behavior than I expect, very young infants can't learn; and yet we know that they do because neonates, for instance, can discriminate their mother's voice from other voices. Put another way, behavioral changes may be discontinuous and appear stagelike even when the underlying system is changing in a gradual, perhaps even linear fashion - and so I fear that if we make behavior carry all the weight of learning, it'll lead right back to Piaget, and suddenly we're stuck with weird discontinuities and "why does behavior change so rapidly in this particular timespan as opposed to any other?" and alla that.

I don't know how helpful this is given that I don't work with artificial learning at all :). But I thought I'd throw it out there anyway.

Erin:

As a psychologist, your input is useful, not least because the common useage of terms may be different in your field. For the psychology and artificial learning communities to develop different 'jargon dialects' in which we don't mean the same thing by a particular term is in none of our interest, but to avoid it happening we must speak to each other and play this definition game sometimes.

I think that "behaviour" is the term we're using differently from each other. When people do studies with very young infants, the dependent variables are always things that I consider measures of behaviour; things like how long the subject looks at a stimulus for, or where the subject looks, etc. So those infants are showing changes in their behaviour.

Your point about the potential for continuous change in learning to produce discontinuous change in behaviour is interesting. Ironically, working with artificial agents gives a way around the practical problem, because I can actually record all activity in a controller's neural network, but it does require a different definition of learning. If I can point to a neuron and say "when this neuron's activation crosses a threshold X, the agent's behaviour will undergo a step change", it no longer seems coherent to say that the learning happens only at the threshold, but that is implied by my post.

Must give this more thought before basing any claims on it....

Yeah, you're right about infants' looking times, etc. being fine behaviors for the purpose at hand, and even neonates have behaviors like sucking. Hmm. I feel like there's something I meant here that I failed to make clear. Perhaps just this: I imagine that people must have done some partial learning before they're able to demonstrate learning in ways that we can observe...

I think we are on the same page here, and your last sentence expresses the issue that I think poses a problem for the definition I'm using. Strictly applying what I wrote in the post implies that there is no such thing as the partial learning you describe, but that's not really what I meant it to imply, so I need to re-work that.

If anything, the fact that I can directly measure the partial learning serves to make the problem more obvious. And then there's the detail that in my system, step changes in the controller can't happen, yet by classifying learning in purely behavioural terms (and because step changes in behaviour do happen) I'm defining learning in terms of a step change. This doesn't quite add up, so thanks for making me think about it.

Hi Eldan,

I read your “Thoughts on Types of Learning” and “Designs for a set of Experiments” in addition to your blog.

I’ve also read your paper “May We Have Your Attention: Analysis of a Selective Attention Task” co-authored with Dr. Beer.

I think you were one of the referees on a submission to Cognitive Science where I was a co-author: “Selective attention and action in an artificial evolved expert: Dynamic integration of perception and action.”

Regarding learning: do you think Beer’s minimally-cognitive, visual agent (VA) demonstrates implicit learning? Based on the agent’s interaction with the environment, the genetic algorithm (GA) adjusts the agent’s neural-controller weights. VA has no deliberate intention to explicitly learn except if we include the GA process as part of the VA cognitive system. In this case, GA is selecting for a specific learning goal, which might be considered explicit learning.

Thanks,
Ronnie

Ronnie,

I think you're right that this is an issue of frames of reference, and what's really important is whether we consider an individual agent to be a stand-alone learning system or whether the whole GA is thought of as a learning system. I prefer to look at this as a question of individual agents, in which case it does make sense to talk about implicit learning.

On some level it also makes sense to look at the whole GA as one learning system, but I prefer not to take that perspective because I think the analogy breaks down once one starts using more realistic models. What I mean by this is that the experiments I'm running right now (like the ones I've published before and those described in the two papers of yours I have read) can be described as systems with the single goal of learning a specific thing, but biological evolution can not, and nor can any sort of simulation in which the fitness function is less narrowly defined. To illustrate the point, here's a little thought experiment:

We could set up a complex simulation in which agents' behaviour determined how long each agent survived for, and the fitness of an agent was determined by how long it survived. In this simulation there could be a large variety of threats to the agent, and successful agents need to have a repertoire of strategies to evade them. In a sense, this simulation could still be described as a unitary system that learns how to control agents such that they do well in the environment, but I'm not sure this level of description tells us anything useful any more. After all, in the same sense a species could be described as 'learning' how to survive in the natural world, but that's pretty far from what I think of as a consensus description of what evolution does or what learning means.

So to get back to the original point, I do think that in the kind of very simplified systems we work with, it makes some sense to describe a GA as doing explicit learning, but I don't think that description generalises.