Entries in "brainstorming"
February 16, 2007
Learning how to use some new tools
Most of what I have accomplished in the past two weeks amounts to learning how to use the BN/PBN toolbox, and re-learning how to use Matlab. In a sense it's a bit frustrating—all this time not actually progressing in research—but it's also clearly necessary to do this sometimes.
The experiment replication I had almost finished with yesterday is now up and running, and replicates the published results very neatly. It turns out that my error wasn't in the code itself, but a simple typo in the input matrix.... I've also discovered that there is an existing function in the toolbox to do essentially the same thing as my code, which makes me feel a tad foolish, but really the point of the exercise was to learn the tools, so it wasn't entirely wasted effort. I was going to try and do the same thing in C++, but I feel like now that I'm over the hump of working out how to use it, the BN/PBN toolbox saves so much wheel-reinvention that it's just foolish not to use it, so unless I can find a C/C++ library to do the same sort of thing I'll stick to what I know.
This means I'm ready to start doing some 'real' work next week, and I'll sketch out one idea for this work behind the cut.
Continue reading "Learning how to use some new tools"
January 30, 2007
Knowing what to leave out
[this is really a continuation of last Thursday's post]
I remember going through a phase during my MSc of wanting to build neuron models that included every known detail. It ought in principle to be possible to model individual molecular interactions, in a non-deterministic way, and thereby effectively run a neuron in silico. My big idea was to test the theoretical understanding of how neurons work, by seeing if the model matched experimental observations, but I eventually came to realise some serious issues with this idea.
Continue reading "Knowing what to leave out"
January 25, 2007
How do we show that a model is appropriate?
In the background reading I'm doing to determine exactly what my new research project should be, certain questions keep popping up. Selfishly, this is a good thing, because anything that really seems to be unanswered by the literature is a potential starting point for interesting research, but for today I'm going to write about one that is worrying to me because I feel like it should have already been answered.
The issue is that for any given biological system, there are many possible ways to model that system. To a large extent, model selection has to depend on what the model is intended to achieve; missing out crucial details will invalidate any conclusions, but including more detail than necessary makes experimental data harder to interpret. We're also constrained by how much is known: for instance no-one can make gene regulation model that realistically accounts for the speed of gene transcription until someone finds a way to discover that speed, but this isn't really what I'm worried about. The question I'm interested in is how one can determine whether or not a given model captures all that is relevant about the process/system being modelled. This worries me most with respect to boolean network models of genetic regulatory networks, because there are many such models about, but they make some really big simplifications.
Continue reading "How do we show that a model is appropriate?"
August 02, 2006
Bloomington debrief
Last week I went to Bloomington to catch up with my lab. It was a shorter visit than would have been ideal, because of external constraints, but we got a lot done in two long, busy days. In terms of my own work, we mostly talked about ways I can make the task easier, because at the moment it takes too many hours of CPU time to get one good agent.
There's a bit of a tension here between the results I need in the end and the results I need to take the next steps with my PhD. Because I'm explicitly asking about the conditions under which learning does and does not evolve, all of the negative results are actually relevant in the long run, but for now I need to chase after positives. This is so that I can defend a research proposal—the sooner the better—which will be impossible without some positive results. If I can demonstrate some conditions under which good agents reliably evolve, then it's not too hard to argue that I'll be able to find many manipulations that cause them to fail, but the reverse is a lot trickier. So I'm left looking for ways of making the task easier.
Specific ideas I got from colleagues and want to take note of are behind the cut.
Continue reading "Bloomington debrief"
July 21, 2006
Ways of scaffolding the task
One of my main interests with the simulations I'm running is in showing how agents can be evolved for learning tasks by using 'shaping' protocols that progressively increase the complexity of the tasks faced by the agents. This is not a revolutionary idea, and in some cases it's very obvious how to go about it (for instance in the time-learning tasks it's just a matter of increasing the range of timings as agents' performance improves), but there can be a bit of an art to getting this right. With the stimulus-learning agents I'm having partial success, in that I now have a couple of high-scoring agents in the initial version of the task but the reliability of searches is low, so it's time to start looking at ways to scaffold this task.
Continue reading "Ways of scaffolding the task"
July 19, 2006
How to deal with evaluation noise
As I've mentioned before, I'm running into problems of evaluation noise with my runs, and the most obvious solution (give each agent more trials) is too costly to be practical. I do have a few ideas of what to do about this, which I'll be experimenting with over the next few weeks.
First, it's worth stating exactly what I mean by evaluation noise, and distinguishing between different kinds. Any evolutionary algorithm has to select between individuals in some way, in order to decide which will be reproduced. If the function for which the algorithm is being used is sufficiently trivial—matching a target string, for instance—then it's simple enough to score each individual precisely on its closeness to the goal. In practice, however, this doesn't work for any interesting problem because comprehensively evaluating each individual would simply take too long. Instead, it's necessary to find some way of approximating the real quality of each individual. In my experiments, I do this by presenting the agent with a sample of possible trials (generated randomly, but each agent in a given generation gets the same set). For many real world applications there's an additional problem—agents are tested on a simulation of the application they'll ultimately be used for—but I don't need to worry about that because my agents only exist in simulation.
Evaluation noise is the disconnect between the scores individuals are given during a GA run, and their real overall performance. I can think of at least three types:
- Systematic bias. This can be from bugs in the simulator (experience suggests that if there is an exploitable bug that incorrectly assigns high fitness, a GA will find the exploit rather than the desired behaviour) or biases in the way the samples of trials are generated. I've encountered both of these problems in the past, but right now there's nothing leading me to believe I'm in this situation with the current experiments.
- Random noise. Even if there is no systematic bias in the generation of trial sets, any given snapshot is unlikely to sample the whole space of possible trials evenly (and making it do so is a great way to create systematic bias). This opens up the possibility of selection being effectively random, because the individuals that happen to perform well on one particular set of trials are rubbish at all others, and the evaluation trials change fast.
- Overfitting. If the evaluation trials don't change fast enough to cause the random noise problem, agents can instead overspecialise, by evolving to exploit some regularity of the evaluation set that does not correspond to a regularity of the set of all possible trials.
Continue reading "How to deal with evaluation noise"
February 10, 2006
The interaction between evaluation noise and neutral networks
Neutral networks in an evolutionary search are the networks of genotypes that all evaluate to the same fitness value. They are important because they allow the population to drift and diversify, which can get a search out of a dead end. Until recently, I had only ever thought about neutral networks in terms of what I'll refer to as 'absolute' neutrality: the case where all genotypes on the network are of exactly equal fitness. [note: is this the ordinary useage of "neutral network", or was I assuming this incorrectly?]
The presence of evaluation noise in my searches has made me re-think this. If there is any randomness at all in the evaluations, then it doesn't make sense to talk about an absolute fitness value, which in turns means that there can't be absolute neutrality. Yet, when I compare the agent that scored the highest fitness in a search with the best agent from the final generation, I often find behaviour that looks like drift.
Continue reading "The interaction between evaluation noise and neutral networks"
January 30, 2006
How to normalise fitness scores
I've been thinking some more about how to normalise fitness scores, so that the number on its own is most meaningful. The issue really is one of making a comparison between agents tested under different conditions as much as possible into a comparison of like with like. This would be straightforward if there were clearly defined maxima and minima in the fitness space, but unfortunately it's not quite that simple.
Continue reading "How to normalise fitness scores"
October 03, 2005
What constitutes learning?
Last week I went back to Cleveland, as I will keep doing every month or two, to meet my advisor and colleagues. It was a useful meeting, giving me some specific feedback, a bunch of useful ideas, and some questions to chew on while I work on things back here. There is one in particular that keeps coming up, because it's very important to address in order to keep my simulations relevant, and the answer is not obvious:
What are the necessary and sufficient conditions for an artificial system to be described as learning?
Continue reading "What constitutes learning?"
September 15, 2005
What I'm doing
I need to get back into the habit of updating here regularly. I'll start with a very brief catch-up of the past month.
I spent last week at ECAL - the European Conference on Artificial Life, which was very productive and I'll write a review of a little later. It was also a bit of a prod to me to be more focussed, because I feel like this year I've done quite a lot of work with nothing really to show for it. I need to get on with turning at least one of the open projects I'm tinkering with into publishable work; preferably publishable work that makes a good basis for a thesis.
At this point I can see only too clearly why it's possible to spend a decade working on a PhD. At the same time, I know it isn't necessary, but it's entirely down to me to take care of this.
To that end, I'm working on a new set of simulations in the same general area as before. It's taken me a month to get around to posting a link here, for no good reason, but here are my most recent musings on the subject. I am working on the simulator for the one set of ideas that is actually fleshed out in any depth there. Meanwhile I've been reading a lot, but I need to cut down on that and produce some tangible results. It's not that there isn't a whole forest of papers worth me reading; but the whole activity becomes pointless if I never do anything with the information.
May 10, 2005
How to classify different types of learning experiment
I realise that I have as yet to write the introducing myself
or introducing my work
posts, which means that for some people this is going to be missing some context, but I'll get to those soon. Meanwhile I've been doing some brainstorming about what exactly to put in the research proposal that I'm hoping to write and defend in the next 4-6 weeks. The act of writing it all down has in itself helped me clarify a few ideas, but there are also plenty of blanks that still need filling in, and inadequately thought out parts that need to be refined. So here, with the caveat that it's really sub-draft quality, is the beginning of a description of what I'm working on:
evolution of learning brainstorm
If you're wondering what I actually do, this might explain things somewhat, and if you can give me any feedback on this (especially on section 4) I'd be very grateful indeed. It's not worth giving me feedback on the writing itself, as it's not at all edited and this document isn't something I anticipate turning into a paper, but any thoughts about the content would be helpful.
Update: Erin McMullen pointed out in an email that my characterisation of condition in this document was completely wrong. Both things that I described are instances of operant conditioning, whereas classical conditioning involves setting up stimulus -> response associations.