The interaction between evaluation noise and neutral networks
Neutral networks in an evolutionary search are the networks of genotypes that all evaluate to the same fitness value. They are important because they allow the population to drift and diversify, which can get a search out of a dead end. Until recently, I had only ever thought about neutral networks in terms of what I'll refer to as 'absolute' neutrality: the case where all genotypes on the network are of exactly equal fitness. [note: is this the ordinary useage of "neutral network", or was I assuming this incorrectly?]
The presence of evaluation noise in my searches has made me re-think this. If there is any randomness at all in the evaluations, then it doesn't make sense to talk about an absolute fitness value, which in turns means that there can't be absolute neutrality. Yet, when I compare the agent that scored the highest fitness in a search with the best agent from the final generation, I often find behaviour that looks like drift.
A particularly clear example would be these two, which are the best agent and the last agent (separated by 1000 generations) from one of the shaping runs [click for enlargements]:
Both charts plot the performance of an agent in 5 sample runs at each period from 1 to 125. Points mark the individual runs, and the lines mark the mean (there aren't 5 points per period because both of these agents tended to do the same thing reliably).
On this particular batch of trials, the 'best' agent scored an overall fitness of 0.779557, and the 'final' agent scored 0.779027 . However, on other batches (generated with the same parameters but a different random seed), the 'final' agent can do better than the 'best' agent. So, allowing for noise, I think it's reasonable to argue that the fitness of the two agents is comparable, even though the charts (which were generated using the same trials for both) show that there is some difference in behaviour.
What I'm wondering is whether it makes sense to describe this drift as having occurred across a neutral network. In a sense, it did, because there's no meaningful fitness difference between the genotypes, but they are clearly not identical. But on the other hand, this definitely doesn't fit the absolutist definition of neutrality I gave above, because on a given set of trials the fitness of the two agents is different.
Comments