SEAS

Several researchers have recently investigated the connection between reinforcement learning and classification. We are motivated by proposals of approximate policy iteration schemes without value functions, which focus on policy representation using classifiers and address policy learning as a supervised learning problem. This paper proposes variants of an improved policy iteration scheme which addresses the core sampling problem in evaluating a policy through simulation as a multi-armed bandit machine. The resulting algorithm offers comparable performance to the previous algorithm achieved, however, with significantly less computational effort. An order of magnitude improvement is demonstrated experimentally in two standard reinforcement learning domains: inverted pendulum and mountain-car.

We present a Bayesian variable order Markov model that shares many similarities with predictive state representations. The resulting models are compact and much easier to specify and learn than classical predictive state representations. Moreover, we show that they significantly outperform a more straightforward Bayesian hierarchical Markov chain model and approach the performance of an oracle hidden Markov model. The simplicity of the approach makes it attractive for applications where the actual hidden state of the system does not need to be explicitly tracked, such as sequential prediction and decision making, while its fully Bayesian nature allows us to take into account the model uncertainty in decision making.

The vast majority of RFID authentication protocols assume the proximity between readers and tags due to the limited range of the radio channel. However, in real scenarios an intruder can be located between the prover (tag) and the verifier (reader) and trick this last one into thinking that the prover is in close proximity. This attack is generally known as a relay attack in which scope distance fraud, mafia fraud and terrorist attacks are included. Distance bounding protocols represent a promising countermeasure to hinder relay attacks. Several protocols have been proposed during the last years but vulnerabilities of major or minor relevance have been identified in most of them. In 2008, Kim et al. proposed a new distance bounding protocol with the objective of being the best in terms of security, privacy, tag computational overhead and fault tolerance. In this paper, we analyze this protocol and we present a passive full disclosure attack, which allows an adversary to discover the long-term secret key of the tag. The presented attack is very relevant, since no security objectives are met in Kim et al.'s protocol. Then, design guidelines are introduced with the aim of facilitating protocol designers the stimulating task of designing secure and efficient schemes against relay attacks. Finally a new protocol, named Hitomi and inspired by , is designed conforming the guidelines proposed previously.

Several approximate policy iteration schemes without value functions, which focus on policy representation using classifiers and address policy learning as a supervised learning problem, have been proposed recently. Finding good policies with such methods requires not only an appropriate classifier, but also reliable examples of best actions, covering the state space sufficiently. Up to this time, little work has been done on appropriate covering schemes and on methods for reducing the sample complexity of such methods, especially in continuous state spaces. This paper focuses on the simplest possible covering scheme (a discretized grid over the state space) and performs a sample-complexity comparison between the simplest (and previously commonly used) rollout sampling allocation strategy, which allocates samples equally at each state under consideration, and an almost as simple method, which allocates samples only as needed and requires significantly fewer samples.