The Problem of Representation is the Problem of Truth! (Saint Machinous Intelligous)
Reading some books on Statistical Learning Theory
I have started reading Vladimir Vapnik’s Statistical Learning Theory book. It is a fascinating book which I enjoy much whenever I read it. Parallel to it, I read Vidyasagar’s Learning and Generalization (2nd edition). It is also about statistical learning theory. However, its trend is somehow different from Vapnik’s. It is early to judge and compare these books, but from what I have read up to now, I can say that Vapnik’s book is much easier and insightful comparing the other. On the other hand, Vidyasagar’s book is more mathematically inclined and I cannot understand many parts of it easily (so I escape most proofs and …). A big problem (for me) in Vidyasagar’s books is that it does not try to explain the underlying phenomena intuitively.
Would you mind write your idea and suggestions about these two books? Moreover, I want to become aware of trends in the theoretical ML. Which book do you suggest? (Kearns?! I haven’t it and I don’t know from where I can find it.)
Happy Norooz!
Happy New Year!!!
Happy Norooz!!!
Today is the first day of Farvardin and the beginning of the new 1384 year. Persian (Iranian) calendar, which is called Khorshidi (=Solar), is the most exact calendar of the world that has only a day of error in every some million years. Its beginning is aligned with the start of the spring. There are many myths about it which I will not tell you! (:
Something … Nothing
Nothing very special … I am waiting for my admissions (and rejections!) to do a final decision. I cannot understand the reason someone might reject me. Nevertheless, those “You are very good, but there is no sufficient accommodations†is not a good explanation for rejecting me. So, I am angry at those guys at the current moment!!!
Well … I changed the title of Thesilog to “Hands off Iran … “. This change can be considered as political act – in the notion of Hannah Arendt (right spelling?). What does it mean? It means that I don’t want U.S.A troops to come to our country and help us getting rid of Mullahs (priest who rule Persia (Iran) ). I do not like Mullahs at all, but I don’t believe that a bloody war is the best way one can do. Beside that, I cannot believe that those troops are very kind people. Moreover, …
What else?! emmm … Apparently, Norooz (or Nowrouz) is coming fast. For those who do not know, Norooz (=New Day) is the first day of the Persians new year. It is the beginning of the spring (1st of Farvardin; usually 21 March) in which we celebrate and doing some fun and etc.
Well … nothing else!
Reinventing SLAM
Trying to reinvent the wheel for a while opens your insight as a tire maker! (Quote by myself!)
Hmmm … I am trying to reinvent robot’s map building method before really reading the detailed stuff. Right now, I am reinventing the Hans Moravec’s Evidence Grid. It is working and I know (don’t ask me from where because I must say that I took a small look to formulas of a few papers (; ) that it is not exactly the same.
I consider the belief of occupancy as P(occupy) (so, 0=Sebastian Thrun’s. I am interested to see the exact formulation of his usage of particle filters.
Blogging Paper Submission
-I am going to submit my paper to CDC-ECC 2005 conference. Hmmm … I want to blog it online to see what happens!
-I have just finished struggling with this idiot MS Word XP that changes the appearance of my papers every time I load the document. Let’s go to the site … I have not bookmarked the page, so I must use Google … yeap!! it is there! (12:02PM)
-Now, look for the page that I must submit my paper. Today’s morning, I got my PaperPlazza’s PIN code. So, I do not need to get it again. My Internet connection is a bit slow and I must wait a little so that my weblog become updated (12:04PM).
-I found it and I am going to PaperPlaza right now!
According to the site, the conference is this: 44th IEEE Conference on Decision and Control and European Control Conference ECC 2005 (CDC-ECC’05), December 12-15, 2005, Seville, Spain Well, I may not attend it by myself, but my co-author would probably do so. Click! (12:08PM – I am not that slow … blogging while reading instructions while … is not that easy! Try it!)
-My paper is regular paper, so go to that part …
-It says that I can test my PDF before going on! It is a good idea. Let’s test it! Hope it works (PLZ!!!) (12:13PM)
-oops … it says that it cannot detect my paper size (A4 or Letter). let’s see what is the problem … 🙁
-I used the CDC-ECC 02 template. The margin and … was the same as the standard A4 paper. emmm …
-I used PDFFactory to generate my files and it causes error. CutePDF does not have this problem, but the output is not that cute. (12:39PM)
-Well … I have uploaded it using CutePDF generated PDF. (1:09PM) So, it seems that is finished now!
This may be the first online blogging of a real paper submission! wow!
Red Queen Effect and Multi-agent Learning
Suppose that there are two populations A and B in the environment. Each population tries to evolve in order to increase its fitness which is coupled to the behavior of the other population, e.g. the performance of hunter is highly dependent on the performance of the prey. However, when A increases its fitness, B evolves in order to become fitter and then the fitness of A is not the same as before as the fitness landscape of it is highly dependent on the others’ policy. This (or something similar to this) is called Red Queen effect in co-evolutionary systems. There are many arguments about its definition and even the usefulness to consider it which you may read here.
I am not expert in co-evolution and I am not aware of the existence of the analyses regarding the dynamics of the co-evolutionary mechanisms (of course, there must be!); however, when I thought about Red Queen effect, I found this idea interesting:
We may analyze the co-evolutionary mechanism in the game theoretic framework. Red Queen effect is like not becoming stable to the Nash equilibrium in the game theory. As we do not know the model of the world (Payoff matrices and stochastic game transition probabilities) in a co-evolutionary setting, we may take a look at similar work in multi-agent reinforcement learning literature and get some inspiration. If we define rational and convergent properties as Bowling defined, we may say that the common implementation of co-evolutionary mechanism that leads to Red Queen effect is like using simple Q-learning (rational/non-convergent) in multi-agent learning. Not let we use something like Minimax-Q or Variable Learning rate Q-learning for co-evolutionary process in order to suppress the Red Queen effect.
Moreover, looking from another perspective (again game theoretic): An agent with Pure strategy may not have a Nash equilibrium but if it follows a Mixed strategy, it has. An individual in a population is certainly doing a pure strategy in most cases. What about evolving a set of strategies choosing randomly between them (evolving PDF too) to imitate evolving to a mixed strategy?!
Hmmm … don’t kill me if it is far from reality! I use my weblog exactly for this kind of conversations! Any idea about them or citing similar ideas? (I searched a little and found that R. Paul Wiegand have done some researches in this area. He used game theory to analyze the problem. I must see what he did.).
(:
Yeap!!! The first!
A Question on Stochastic vs Deterministic Policies
Is it possible that a stochastic strategy be better than a greedy one in the sense of obtained reward and after learning and convergence to a fixed policy? For instance, is there any situation that something like Boltzman action selection performs better than Greedy one? It is not the case in MDP, but what about POMDP?! I guess not! I am looking for a counterpart of game theory’s Mixed Strategy in other fields. For some multi-player games, there exist a mixed strategy Nash equilibrium but there is no such a point in pure strategy case. Have you seen something similar in other fields and more specifically in the cases that the performance is the comparison criterion. I wonder what the benefit of acting randomly can be.