Eldan Goldenberg's lab notebook

Slow, slow experiments

Since before GECCO, I've been running experiments based on both strands of my roadmap. I now have rather a lot of computing power at my disposal, which makes it all the more damning that some of these experiments require more than a week to run. The problem is that evaluation noise has the potential to swamp any meaningful differences between agents, and if there's one certainty about genetic algorithms it's that they'll exploit whatever stupid trick their designer unintentionally leaves available, instead of doing what they were intended to.

In this case, it's that each generation of agents only gets exposed to a sample of the possible trials, and I seem to be having major problems with overspecialisation. One of my hypotheses (and a rather uncontroversial statement, I think) is that where the environment is predictable, learning is inferior to pre-set behaviour. The problem in my experiments is that given samples of the space of possible trials tend to have spurious regularities in them, so agents that happen to exploit these regularities will temporarily outperform agents that actually do what I want them to.

I've avoided sampling the space in a uniform way, because prior experience tells me that I'll get agents which only work with a certain granularity or harmonic series of stimuli. Instead, I'm sampling randomly, but this carries an unwinnable trade-off between too-small samples having other spurios regularities (such as a particular subset of the possible inputs being favoured) and too-large samples taking forever to run. It's not a complete wash—some of the experiments from last week that have yet to finish do seem to have produced some good all-round agents—but it will take me forever to do anything interesting if experiments take 10 days each to run and still only turn out good agents a small proportion of the time.

For now, I'm doing a little more tinkering with parameters to see if I can get the balance right, but next week I'll start testing (and writing about) some more sophisticated approaches to this problem.