On Adaptive Situated Agents

WCCI (CEC) 2006 Invited Talk:
Coevolutionary Learning by Melanie Mitchell

This invited talk is about making better learning systems. The idea is evolving both “classifier” and “training set” together (co-evolving them in a competitive setting). The way that this works well is by spatial co-evolution. This “spatial” property is very important and other cases (non-spatial co-evolution or spatial evolution) did not work well. However, I should note that none of the experiments was about real classifier. One of them is designing a cellular automata to do some specific task (majority voting) and the other is fitting a tree-like structure (actually a GP tree) to some function.

She thoughts that spatial Co-evolution improves performance because:
-maintain diversity in population
-produces “arm races” with ICs targeting weaknesses in CA and CAs adapting to overcome weaknesses.

Also you may like to read a quotation. This is one for you: “Co-evolution is all about information propagation!”

Finally I asked her if it is reasonable to evolve training set for a classification task? By evolving them too, the pdf of the training set will changed, and this means that the evolved new samples may be completely different from the pdf of the test set, i.e. the classifier evolved in a way to optimize a loss function for a distribution other than “population” (or test or true) distribution.
She answered me, I did not satisfied completely by her answer.

Now, I am sitting in this big Pavillion Ballroom waiting for Marco Dorigo to start his presentation. He is going to talk about “Self-Assembling in Swarm Robotics: The Swarm-bot Experiment”.

His group has built swarm-bots (s-bots). S-bot are small robots than can connect to each other and do things that a single robot cannot do.

The technological motivation for this work is: Robustness, Flexibility, Scalability

s-robot are 12cm.
Also I found out that Khepera is pronounced as Kepera.

The simulator they developed has different levels of abstraction as the model of the robot. One important problem for them was selecting the right level of complexity for the simulation of their robot.

To design robots’ controllers, they either hand-coded behavior-based system or evolved a neural network to do so. They did all designing in the simulator and then downloaded the result to the real robot.

The evolved a NN to do collective movement of a swarm of s-bots.
They used Perceptron and evolved their parameters using a simple ES method.
I missed (or maybe it is not mentioned) the observation space of the NN.

It seems that the resulted controller can work in different situations. For instance, it is scalable, can do coordinated movement with different initial formation of robots and etc.

Then they tried to evolve a controller for more complex tasks such as hole avoidance. They used a ground-hole detector for each robot and also a microphone to detect sounds. The action space is motion AND some sound. This sound can be used to communicate with other robots.

Path Formation:

On going work:
-Functional self-assembling
-Functional shape formation
-Adaptive task allocation
-Evolution of recurrent neural nets for memory …

Finally, I asked about using ideas of learning (RL). He said that the main reason that they did not test it is the lack of resources (as each experiment takes a lot of time).
After the talk, I asked about the comparison of the reinforcement learning and evolutionary approach to design autonomous agents. He said that the main problem is that nobody is going to really compare different methods in the same robotic task because it is time-consuming (and people in the field are lazy (these are my words!)).

Well … I liked the talk. It was fun. More importantly, I really impressed by the results.

You may get more information the project’s website: Swarm-bots