IEEE removed most of its restrictions on people of embargoed countries. It includes peer review process, publishing, and even full electronic access. You can read more at there!
Conference on EMail and Anti-Spam
Who thought that spam would be this hazardous and annoying?! Let’s wish a world without spam.
If you are interested in some technical aspects of anti-spamming, you may like to visit this Conference on Email and Anti-Spam (CEAS 2005) and read a few papers about.
How to do Research in AI
I have read some parts of this How to do Research at the MIT AI Lab -which is written by a whole lot of people at MIT- a few times during my research. It is very interesting and motivating (and obviously is general enough that most parts of it can be used by any other AI student). For instance, I have felt some facts that is mentioned in Emotional Factors of this document and they are really true! I must read the whole bunch of suggestions again.
Debugging tip: Human Vision Capability
I faced a troublesome problem in my programs: I have made a lot of different versions of the same program with just a little difference between. This difference can result in a complete different performance and cannot be ignored. However, I cannot remember which one is which and what are their dissimilarities. In order to find out this, it is necessary to compare two different files and to do so, I open two codes in two different windows and switch between them very fast (a few many times a second). Human vision (like most other animals including lizards (remember Jurassic park!) is very sensitive to changes, thus I can detect a single character difference between two pages very easily. Anyway, this can be applied whenever there is no tremendous difference between those two files.
C codes for Neural Network
Implementing a neural network is not very easy task. For some, it may be even impossible. This site offers C implementation of various types of neural networks including ADALINE, Multi-layer Feedforward NN (they call it Back-Propagation Network which is not correct. BP is a learning method and not an architecture), Hopfield, Self Organization Network (SOM), Adaptive Resonance Theory 1 (ART1), and a few more. I have not seen their codes, so I don’t know whether they are a good one or not- but it exists anyway (!) and may be useful someday. Thanks to Saeed for the link (he does not have a site nowadays).
Quantum Mechanics Knowledge
There’s no way to understand the interpretation of quantum mechanics without also being able to solve quantum mechanics problems – to understand the theory, you need to be able to use it (and vice versa). If you don’t heed this advice, you’ll fall prey to all sorts of nonsense that’s floating around out there.” (John Baez – How to Learn Math and Physics)
Therefore, I must confess that I do not know Quantum Mechanics (have I ever claimed so?) emmm … QM fascinates me and I would like it much if I were a physicist.
Leonardo
AI Links
It is somehow disappointing that there are a lot of useful stuff in the Internet that you cannot even read its title – not mentioning their readings. Unfortunately, there is no simple way to read all of them. Emmm … let’s link to this site:
AI Links that is maintained by Mark Humphrys – whom has recently gained my attention due his works on action selection and specially that interesting W-Learning idea.
Let’s bring its title in order to be easier for you (and specially myself) to remember what you (I) can find in it.
Pre-HRL Presentation era!
I’m working on my Hierarchical Reinforcement Learning presentation that I will present in Distributed AI class a few hours later.It is 2:47AM and … emmm … yeap! The life is too compressed!
IROS 2004: Paper acceptance
I woke up today, checked my e-mail and suddenly I found this mail who announced me that my IROS 2004 paper has been accepted!! (: I have been waiting for this mail for a long time! (at least, it is a week that I’m too curious to know the result!!) The paper, which is entitled “Behavior hierarchy learning in a behavior-based system using reinforcement learningâ€Â, is based on my work on structure learning of Subsumption Architecture. Anyway, this news was a very good one! (:
These are its comments which I must answer:
Comment #1
——————————————–
Interesting preliminary results.
Further work is required including real experiments.
——————————————–
Comment #2
——————————————–
Summary:
The paper describes a reinforcement learning approach to selecting behaviors in a subsumption architecture. From a given set behaviors arranged in a hierarchy of layers, each layer learns to determine which behavior should be active. An appropriate (greedy, value-function based) reinforcement learning system is formulated for this problem, and evaluated in a simulated cooperative object lifting example with multiple robots.
General Comments:
Applying reinforcement learning (RL) to a subsumption architecture is not new, as cited correctly by the authors. What is finally developed in the paper looks like a standard value iteration RL method, i.e., a form of approximate dynamic programming. As the authors mention themselves, RL has seen a fair amount of work over the last year in learning with behaviors (the authors mention Options as future work). Thus, why did the authors not follow one of these established behavior-based RL approaches, or at least compare their results with related work? It will not be obvious for a reader where the originality and significance of the paper lies.
Detailed Comments:
– The use of English needs improvement in various places.
– Page 1: are the S i parts of the state space for each behavior overlapping or not?
——————————————–
Comment #3
——————————————–