Entries in "minutiae"
January 26, 2006
Complex signatures from simple behaviour
I've spent about 3 hours today analysing one agent. This is longer than it usually takes, especially given that the agent's behaviour turned out to be quite simple. The reason it took me so long was that the agent's simple, 4-step control algorithm interacts with the environment in a way that creates a highly discontinuous [is this a word?] relationship between an environmental parameter (the length of oscillation between good food and bad) and the agent's effectiveness.
I'll put a little more detail behind the cut, but basically I realised that all of the apparent variety in this agent's performance was generated by artefacts of exactly how many epochs of good and bad food it had. I think this serves as an important warning to myself: sometimes the appearance of complexity is entirely deceptive.
Continue reading "Complex signatures from simple behaviour"
New category
I've found myself writing more about the day-to-day minutiae of what I'm finding than I had anticipated. This is useful to me, because I work things out in my head by writing them about them, and it's not always obvious which issues are going to turn out to be relevant a year from now. However, it must also be of very little interest to anyone else; or at least not of enough interest to justify wading through the many paragraphs I put online each day.
So I've started a new category—minutiae—to file all of these things away into. I'm going to try and write each minutiae post in such a way that the point is described by one or two paragraphs, and readers can safely ignore everything behind the cut. That way the detail is there for my reference, but hopefully everything that might actually be of general interest or ever make it into a paper ends up displayed on the main page of this blog without cluttering the site.
I've also moved some old posts into this category.
January 25, 2006
Establishing a fitness baseline
Having found that normalising agents' performance against the highest fitness achievable under the same set of conditions made me realise something else. For every individual trialset there is also a maximum possible fitness which depends on parameters, and which will feed back into the performance of an agent on a given trialset. So I decided to plot this. The pattern was more complex than I had anticipated, though in retrospect the causes of the pattern are straightforward.
Continue reading "Establishing a fitness baseline"
January 23, 2006
Disambiguating performance
Today's update is going to be about two things: how important that bug I found on Thursday actually was, and some consideration of the appropriate way to compare fitness values.
Continue reading "Disambiguating performance"
January 19, 2006
Slapping myself in the face
I have just found a bug I'd been looking for since Tuesday. Unfortunately it turns out to affect more of my experiments than I had realised, so it may invalidate my results from January so far. It's also possible that it didn't spoil anything—certainly the effects are subtle enough in most conditions that I didn't notice it until a run with just the right set of parameters—but I think I'm going to have to re-run quite a few experiments to find out.
Continue reading "Slapping myself in the face"
January 13, 2006
Death and test data
For the past couple of days, I've been running and analysing the output from the fixed experiments in which agents are correctly saved every 100 generations, and agents die as intended, as I described on Wednesday. I'm nowhere near finished with this batch of data analysis, but while I wait for some tests to finish, and before I finish work for the week, I wanted to mention a few notable things I seem to be seeing:
CAVEAT: this is all preliminary; it describes incomplete data on which I have not yet done any significance tests.
- Agents score higher on the harder task
- Agents do better if trial sets are longer, but the search is slower
- Random trials marginally understimate performance relative to systematic trials
- The stupid threshold strategy is very common
- The population continues to improve after the supposed best agent has been found