Bayesian Interpretation of Probability Theory and Fuzzy Logic

Previously, I was a fan of Fuzzy logic and the way it deals with uncertainty. I considered it as a complement of the probability theory. At that time, I thought that probability theory is a theory that deals with randomness kind of uncertainty (so I was a frequentist).
However, after reading something about Bayesian interpretation of probability theory (probability as a measure of plausibility of logical propositions as stated by E. T. Jaynes (see Probability Theory: The Logic of Science) and others), my loyality to the Fuzzy logic weakened. Now, I am not sure if this interpretation of probability is really talking about some other thing and if the probability theory is not a “better” and more plausible thing to do.
I should read more. I know there are several debates around this topic. However, I am not sure if they consider probability theory a Jaynes, Cox, and etc. do.

Anyway, I am going to read this paper:
Didier Dubois, Henri Prade, “ Fuzzy sets and probability : Misunderstandings, bridges and gaps,” Proceedings of the Second IEEE Conference on Fuzzy Systems

A Quintessential Introduction to Dark Energy

There are a whole lot of things in the world that I do not know that much. It is true that I cannot learn all of them (or even a small portion of them) in my lifetime. Actually, it is not necessary for most cases, e.g. I do not need to know how that technology works if I do not want to use it. However, some kinds of knowledge is just essential. They shape our viewpoint to the world and “may” answer questions like “where do you live?”, “how do you live?”, and etc.
All intro. things said, I want to read this paper later. You may like to take a look at it some time:

Paul J. Steinhardt, A Quintessential Introduction to Dark Energy

Abstract:Most of the energy in the universe consists of some form of dark energy that is gravitationally self-repulsive and that is causing the expansion rate of the universe to accelerate. The possible can-didates are a vacuum energy density (or, equivalently, a cosmological constant) and quintessence, a time-evolving, spatially inhomogeneous component with negative pressure. In this review, we focus on quintessence and ideas on how it might solve the cosmic coincidence problem, how it might be distinguished observationally from a cosmological constant, and how it may affect the overall cosmic history of the universe.

@NIPS 2005

Right now, I am in the middle of the NIPS 2005 conference waiting for another oral session to begin.
In this afternoon session, Sanjoy Dasgupta talked about his new result about active learning and the possible merit of it comparing with the traditional supervised case.
He defined something named searchability index which shows the difficulty of the problem in active learning. This index shows that to what extent the ideal binary search-like division of hypothesis space is applicable. If that index is high (or constant over all hypothesis space), the sample complexity of the problem is VC_dim*log(1/eps) (binary division). The other extreme case is VC_dim*(1/eps) (supervised case).
Anyway, you may like to read his paper (I’ll cite it later. I can’t remember the title of the talk.).

I like the idea of active learning. Moreover, I wonder if RL can benefits from it (or vice versa). It seems that active learning strategy is somehow like selecting the exploration strategy.

Self-Organized Integration of Adaptive Visual Cues for Face Tracking

Jochen Triesch, Christoph von der Malsburg, Self-Organized Integration of Adaptive Visual Cues for Face Tracking

I have started reading a few papers on multi-cues and multi-modal hypothesis in visual object tracking. I guessed the field was not that interesting, but it seems that it is! The observer design problem in these situations is much more complex and realistic that what I have studied in my control courses. There, nothing was being said about multi-modality. Anyway, I summarize a few papers on this subject and express my ideas sometimes. Note that this weblog is semi-academic: so, my synopses may be not so accurate or correct. I would be happy if the author of papers help me understand their papers better. (:

It proposed a method called Democratic Integration. It is a weighted-based voting-like method that adapts the weights in a self-organizing manner, i.e. it does not use any external signal for those changes. Instead, it uses the difference between the overall result and the result of each cue. If they are alike, that cue’s weight (or reliability – as used in the paper) will be increased.

$$R(x,t)=\sum_i r_i(t)A_i(x,t)$$
$$\hat{x}(t) = argmax{R(x,t)}$$
And for adaptation, it defines a quality $q_i(t)$ as follows
$$\tilde{q_i}(t) = R(A_i(\hat{x}(t)) – E[A_i(x,t)])$$
in that $R(.)$ is the ramp function. In this formula, $A(.)$ is the saliency map of each tracker. It is stated in the paper and I see it in a few other that in the current literature, this saliency map is considered to be probability of the object to be in a specific place. The paper mentioned that this choice of quality function is ad hoc and some other ideas like using Kullback-Leibler distance would give better results.
utput of each tracker and represents After doing normalization over $q_i$, the change in reliability would be

$$\tau\dot{r_i}(t) = q_i(t) – r_i(t)$$

A similar formulation is given for adaptation of prototypes. The paper used a few simple cues to do face tracking. The result is not that exciting, but its performance comparing with the case without adaptation is much superior. I am not aware of the performance of other face tracking methods.

I wonder what would happen if I use context-based switching between different cues, i.e. storing the temporally steady-state reliability weights and then test them as initial guess whenever error happens. What is the measure of error?! I am not sure, but what about the number of conflicts between different trackers?! Or time-averaged gradient of reliability vector over time.

Martin Jagersand, Image Based Visual Simulation and Tele-Assisted Robot Control

[In this semester, I want to read many papers on different aspects of artificial intelligence concentrating on machine learning (in general), reinforcement learning (in particular), evolutionary mechanisms, intelligent robotics, and … . In this way, I try to write my notes and comments on the papers that I read. I will submit my comments to the papers’ authors and see if I can get a good feedback.
I will find the optimal way of doing so gradually. Different parameters affect the behavior ranging from my time to the benefit of this work for myself. Any suggestion or comments is highly appreciated.]

Martin Jagersand, “Image Based Visual Simulation and Tele-Assisted Robot Control,” IROS 97.
Continue reading “Martin Jagersand, Image Based Visual Simulation and Tele-Assisted Robot Control”

Behavior Hierarchy Learning in a Behavior-based System using Reinforcement Learning (abstract)

This is my IROS 2004 paper abstract which is entitled Behavior Hierarchy Learning in a Behavior-based System using Reinforcement Learning. It will be presented very soon at International Conference on Intelligence Robots Systems [It have been presented now! You can download the paper at my publication page or directly at there].
]. I am not sure if I present it or my advisor, Dr. Nili, do so as I have some problem getting passport and VISA. Anyway, it is very interesting to go to my first international conference and visit those guys that I read their papers and adore their work, e.g. Maja Mataric of USC, Cynthia Breazeal of MIT, Leslie Pack Kaelbling of CMU, Maneula Veloso of CMU, Lynne Parker of university of Tennessee, Sridhar Mahadevan of UMass, and a lot more who I haven’t found their names yet. I wish I could visit other ones like Rodney Brooks, Marvin Minsky, Andrew Barto, Richard Sutton, Marco Dorigo, Floreano, and … and … but it seems that they do not participate in this conference. It is somehow natural as this is a Robotic conference and not all-I-lovable-topics one!!

And now, you can see the abstract. I will put the paper as soon as I upgrade my host and run an updating section for my scientific work.

Behavior Hierarchy Learning in a Behavior-based System using Reinforcement Learning
Abstract: Hand-design of an intelligent agent’s behaviors and their hierarchy is a very hard task. One of the most important steps toward creating intelligent agents is providing them with capability to learn the required behaviors and their architecture. Architecture learning in a behavior-based agent with Subsumption architecture is considered in this paper. Overall value function is decomposed into easily calculate-able parts in order to learn the behavior hierarchy. Using probabilistic formulations, two different decomposition methods are discussed: storing the estimated value of each behavior in each layer, and storing the ordering of behaviors in the architecture. Using defined decompositions, two appropriate credit assignment methods are designed. Finally, the proposed methods are tested in a multi-robot object-lifting task that results in satisfactory performance.

Paper Download

I have downloaded a lot of paper today. I wonder if I read them before the Judgment day! (I think doing so is a kind of masochism.)

PACness of MDP

I found some useful papers about PAC of MDP and its VC (and …) dimensions. Read them later:

Rahul Jain and Pravin Varaiya, “PAC Learning for Markov Decision Process and Dynamics Games,” ?, 04.
R. Jain and P. Varaiya, “Extension of PAC Learning for Partially Observable Markov Decision Processes,” ?, 04.
Yishay Mansour, “Reinforcement Learning and Mistake Bounded Algorithsm,” ?, 1999.

Paper: Evolution of a Subsumption Architecture Neurocontroller

Julian Togelius, “Evolution of a Subsumption Architecture Neurocontroller”, ?

I’ve read this paper. It was interesting as it strenghten my idea of using (or possibility of using) incremental ideas in learning. I have done some experiments doing incremental learning, but I’m not yet in a place to make conclusions.
Before rewriting its abstract, let’s copy this informative table:

1-One layer – One fitness: Monolithic evolution
2-One layer – Many fitness: Incremental evolution
3-Many layers – One fitness: Modularized evolution
4-Many layers – Many fitness: Layered evolution

(One may call have another names for this one, i.e. I used to name every incrementally making “many layers” system, “incremental”.)
He found out that the forth method of evolution has indeed very good performance. It is the one I’m thinking about.

Here is the paper’s abstract:
Abstract. An approach to robotics called layered evolution and merging features from the subsumption architecture into evolutionary robotics is presented, and its advantages are discussed. This approach is used to construct a layered controller for a simulated robot that learns which light source to approach in an environment with obstacles. The evolvability and performance of layered evolution on this task is compared to (standard) monolithic evolution, incremental and modularised evolution. To corroborate the hypothesis that a layered controller performs at least as well as an integrated one, the evolved layers are merged back into a single network. On the grounds of the test results, it is argued that layered evolution provides a superior approach for many tasks, and it is suggested that this approach may be the key to scaling up evolutionary robotics.