I woke up today, checked my e-mail and suddenly I found this mail who announced me that my IROS 2004 paper has been accepted!! (: I have been waiting for this mail for a long time! (at least, it is a week that I’m too curious to know the result!!) The paper, which is entitled “Behavior hierarchy learning in a behavior-based system using reinforcement learningâ€Â, is based on my work on structure learning of Subsumption Architecture. Anyway, this news was a very good one! (:
These are its comments which I must answer:
Comment #1
——————————————–
Interesting preliminary results.
Further work is required including real experiments.
——————————————–
Comment #2
——————————————–
Summary:
The paper describes a reinforcement learning approach to selecting behaviors in a subsumption architecture. From a given set behaviors arranged in a hierarchy of layers, each layer learns to determine which behavior should be active. An appropriate (greedy, value-function based) reinforcement learning system is formulated for this problem, and evaluated in a simulated cooperative object lifting example with multiple robots.
General Comments:
Applying reinforcement learning (RL) to a subsumption architecture is not new, as cited correctly by the authors. What is finally developed in the paper looks like a standard value iteration RL method, i.e., a form of approximate dynamic programming. As the authors mention themselves, RL has seen a fair amount of work over the last year in learning with behaviors (the authors mention Options as future work). Thus, why did the authors not follow one of these established behavior-based RL approaches, or at least compare their results with related work? It will not be obvious for a reader where the originality and significance of the paper lies.
Detailed Comments:
– The use of English needs improvement in various places.
– Page 1: are the S i parts of the state space for each behavior overlapping or not?
——————————————–
Comment #3
——————————————–