Multi-Robot Coordination

We study how fundamental statistical limits in reinforcement learning change when multiple
real-world challenges interact. Focusing on sample inefficiency, nonstationarity, partial observability, and
high-dimensional observations, we synthesise existing lower-bound arguments and show that their effects
are generally non-additive. We formalise three structure-conditioned mechanisms: multiplicative complexity
penalties in partially observable nonstationary environments, memory collapse under low-rank observation
structure, and explicit finite-horizon safety guarantees via probabilistic shielding. Rather than proposing
new algorithms, the paper clarifies how exploitable structure reshapes worst-case guarantees and motivates
a shift from pessimistic minimax analysis toward conditional complexity frontiers that tighten as structure
is detected online.

In this paper, we present a mechanism for coordinating multiple robots in the execution of cooperative tasks. The basic idea in the paper is to assign to each robot in the team, a role that determines its actions during the cooperation. The robots dynamically assume and exchange roles in a synchronized manner in order to perform the task successfully, adapting to unexpected events in the environment. We model this mechanism using a hybrid systems framework and apply it in different cooperative tasks: cooperative manipulation and cooperative search and transportation. Simulations and real experiments demonstrating the effectiveness of the proposed mechanism are presented.

One of the most important and challenging problems of any kind of autonomous mobile robot is the ability to accurately control its onboard actuators, enabling it to fulfill a specified task. In the case of a two-wheeled mobile robot, this can only be achieved through a pair of adequate steering control signals. The main goal of this paper is to design a nonlinear multivariable controller allowing a self-made mobile robot prototype to track a prescribed trajectory. The basic principle of this control approach uses the Lyapunov theory as a primary tool to derive two steering control laws, making a three-state error vector converge to zero. Tuning the proposed controller parameters is carried out using an equivalent dynamic simulated model. This controller is then applied to generate the resulting command signals to the actual robot. This is achieved through a real-time high-speed serial communication between a stationary personal computer (PC), on which a MATLAB/Simulink version of this controller is performing, and an onboard Microchip 16 bits dsPIC33FJ64MC802 microcontroller running a firmware that takes care of all the data exchange with the connected PC and a set of two proportional integral derivative (PID) controllers ensuring that the rotational speeds of the robot wheels are kept very close to those required by the main controller, running on this PC. The performance of the proposed controller is evaluated using two different shaped trajectories. These tests show that the robot is able to gradually follow the required path with minimal lateral error. The robustness of this controller is demonstrated through its capability to reject external disturbances triggered during these experimental tests.

This paper presents the results of an exploratory study that analyzes the main factors of the decision making process of recruitment and selection of strategic positions for companies located in Santiago of Chile, in the Metropolitan Region of Chile. The methodology developed uses AHP Multi-criteria concepts, addressing the major theoretical foundations of this methodology. The results of a survey of organizations in Santiago city, in order to obtain the similarities and the most relevant factors for the recruitment and selection of strategic positions, are presented, allowing the establishment of an exploratory common pattern for the process.

In the present document, the authors introduce the Cooperative Exploration problem as well as the most relevant approaches in order to show the most common drawbacks and opportunities to improve the state of art solutions. Subsequently, a preliminary version of a multi-robot exploration proposal is described. The first results obtained in simulated scenarios support the underlying ideas are feasible and promising. They show that is possible to cope with real communication constraints (always present in practice), being more fault tolerant and still having good performance regarding the total exploration time. Next steps to fully implement a more reliable and robust system are discussed.

During the past few years, point-based POMDP solvers have gradually scaled up to handle medium sized domains through better selection of the set of points and efficient backup methods. Point-based research has focused on flat, explicit representation of the state space, yet in many realistic domains a factored representation is more appropriate. The latter have exponentially large state-spaces, and current methods are unlikely to handle models of reasonable size. Thus, adapting point-based methods to factored representations by modeling propositional state spaces better, e.g. by using Algebraic Decision Diagrams (ADDs) is needed. While a straightforward ADD-based implementation can effectively tackle large factored POMDPs, we propose several techniques to further improve scalability. In particular, we show how ADDs can be used successfully in factored domains that exhibit reasonable locality. Our algorithms are several orders of magnitude faster than current point-based algorithms used with flat representations.

The growing use of autonomous tractor fleets with detachable implements presents complex logistical challenges in agriculture. Current systems often rely on simple heuristics and avoid implement swapping, limiting efficiency. A central challenge is to dynamically coordinate vehicle routing and implement exchanges to enable efficient, low-intervention task execution. Due to high costs, such fleets are owned mainly by large enterprises or cooperatives, where fair task allocation and profit sharing are critical. Addressing both coordination and fairness, in this paper, we introduce the Agricultural Fleet Vehicle Routing Problem with Implements (AFVRPI). We propose a distributed model derived from a centralized formulation also presented in this paper. This model is embedded within a Distributed Multi-Agent System Architecture (DIMASA), where autonomous vehicle agents manage routing and implement use under limited fuel autonomy, while implement agents ensure compatibility and sufficient capacity to meet task demands. Our solution applies systematic egalitarian social welfare optimization to iteratively maximize the profit of the worst-off vehicle, balancing fairness with system efficiency. To enhance scalability, we use column generation in the distributed model, achieving solution quality comparable to the centralized model while significantly reducing computing time. Simulation results on new benchmark instances demonstrate that our distributed multi-agent AFVRPI approach is scalable, efficient, and fair.

Recent scaling up of decentralized partially observable Markov decision process (DEC-POMDP) solvers towards realistic applications is mainly due to approximate methods. Of this family, MEMORY BOUNDED DYNAMIC PROGRAMMING (MBDP), which combines in a suitable manner top-down heuristics and bottom-up value function updates, can solve DEC-POMDPs with large horizons. The performances of MBDP, can be, however, drastically improved by avoiding the systematic generation and evaluation of all possible policies which result from the exhaustive backup. To achieve that, we suggest a heuristic search method, namely POINT BASED IN-CREMENTAL PRUNING (PBIP), which is able to distinguish policies with different heuristic estimates. Taking this insight into account, PBIP searches only among the most promising policies, finds those useful, and prunes dominated ones. Doing so permits us to reduce clearly the amount of computation required by the exhaustive backup. The computation experiment shows that PBIP solves DEC-POMDP benchmarks up to 800 times faster than the current best approximate algorithms, while providing solutions with higher values.

In an adversarial multi-robot task, such as playing robot soccer, decisions for team and single-robot behaviour must be made quickly to take advantage of short-term fortuitous events. When no such opportunities exist, the team must execute sequences of coordinated team action that increases the likelihood of future opportunities. A hierarchical architecture, called STP, has been developed to control an autonomous team of robots operating in an adversarial environment. STP consists of skills for executing the low-level actions that make up robot behaviour, tactics for determining what skills to execute, and plays for coordinating synchronized activity among team members. The STP architecture combines each of these components to achieve autonomous team control. Moreover, the STP hierarchy allows for fast team response in adversarial environments while carrying out actions with longer goals. This article presents the STP architecture for controlling an autonomous robot team in a dynami...

This paper reviews the state of the art in coordinated multi-robot exploration and proposes a new exploration objective based on a practical scenery, reducing the difference of waiting time among different regions of a workspace, which has not been still considered in the literature. A new global optimization strategy for coordinated multi-robot exploration based on a proper dispersion of robots in separate regions is presented. This strategy aims at achieving the lowest variance of regional waiting time and the lowest variance of regional exploration percentage. Both features reveal that the proposed strategy performs better than other state of the art approaches.

This paper reviews the state of the art in coordinated multi-robot exploration and proposes a new exploration objective based on a practical scenery, reducing the difference of waiting time among different regions of a workspace, which has not been still considered in the literature. A new global optimization strategy for coordinated multi-robot exploration based on a proper dispersion of robots in separate regions is presented. This strategy aims at achieving the lowest variance of regional waiting time and the lowest variance of regional exploration percentage. Both features reveal that the proposed strategy performs better than other state of the art approaches.

In this paper, we study a particular subclass of partially observable models, called quasi-deterministic partially observable Markov decision processes (QDET-POMDPs), characterized by deterministic transitions and stochastic observations. While this framework does not model the same general problems as POMDPs, it still captures a number of interesting and challenging problems and have, in some cases, interesting properties. By studying the observability available in this subclass, we suggest that QDET-POMDPs may fall many steps in the complexity hierarchy. An extension of this framework to the decentralized case also reveals a subclass of numerous problems that can be approximated in polynomial space.

We discuss the problem of policy representation in stochastic and partially observable systems, and address the case where the policy is a hidden parameter of the planning problem. We propose an adaptation of the Predictive State Representations (PSRs) to this problem by introducing tests (sequences of actions and observations) on policies. The new model, called the Predictive Policy Representations (PPRs), is potentially more compact than the other representations, such as decision trees or Finite-State Controllers (FSCs). In this paper, we show how PPRs can be used to improve the performances of a point-based algorithm for DEC-POMDP.

Social robots should be able to search and track people in order to help them. In this paper we present two different techniques for coordinated multi-robot teams for searching and tracking people. A probability map (belief) of a target person location is maintained, and to initialize and update it, two methods were implemented and tested: one based on a reinforcement learning algorithm and the other based on a particle filter. The person is tracked if visible, otherwise an exploration is done by making a balance, for each candidate location, between the belief, the distance, and whether close locations are explored by other robots of the team. The validation of the approach was accomplished throughout an extensive set of simulations using up to five agents and a large amount of dynamic obstacles; furthermore, over three hours of real-life experiments with two robots searching and tracking were recorded and analysed.

In order for multi-robot systems to efficiently assist in saving lives and infrastructures in the RoboCup Rescue Simulation, any strategy designed to allocate tasks and coordinate agents must adapt to the dynamic nature of the environment. In this work, we describe how to form teams of agents that take advantage of synergies among the different types of agents and we evaluate the effectiveness of the team configuration strategies on different maps.

Frontier-based exploration methods are efficient for multi-robot exploration systems. In this paper, enhanced frontier-based techniques are used with team of cooperating mobile robots to explore unknown environment. The aim of the exploration algorithm is to minimize the exploration time by coordinating the robots to maximize overall utility by minimizing the potential of overlapping in information gained amongst the robots. The proposed frontier-based exploration algorithm is based on a new bidding function to decrease the overlap between the robots in addition to the utility and cost parameters. A special parameter depends on the future positions of the robots is also considered. The proposed algorithm has been tested with different environments. We also compared it to A* algorithm. Comparisons demonstrated the efficiency of proposed algorithm.

In the exploration and implementation of formation control schemes, bandwidth limitations and communication breakdown form a barrier to large scale formation control applications. The limitations of current formation control strategies involving a leader-follower approach and a consensusbased formation control approach with fully available group trajectory information are explored. A unified formation control scheme that accommodates an arbitrary number of group leaders and arbitrary information flow between vehicles is proposed. The scheme requires only local neighbor-to-neighbor information exchange. In particular, an extended consensus algorithm is applied on the group level to estimate the timevarying group trajectory information in a distributed manner. Based on the estimated group trajectory information, a consensus-based distributed formation control strategy is then applied for vehicle level control. The proposed scheme is experimentally implemented on a multi-robot platform under local neighbor-to-neighbor information exchange. The effect of a single or multiple leaders within the formation is also experimentally explored and discussed.

Industry 4.0 which creates "smart factories" present a recent trend in development. The area represents a merge of cyberphysical systems and Internet of Things, which aims to improve manufacturing technologies. Industry 4.0 strives to boost the algorithms and technologies used in industrial processes during the production processes, process preparations, and products delivery. Our intention is to improve the robotics transport system in factory floor. There are a lot of different research approaches in this area for further improvement. Our approach is to deal with multi-agent systems control, because of the great potential it has in practical applications in industrial robotics. The strive for minimizing the work time and maximizing the efficiency can be satisfied through the usage of multiple coordinated agents to achieve the end goal. The use of Automated Guided Vehicles (AGVs), combined with concepts for task planning of multiple agents broadened during the late 20th century. In this paper, the multi-agent system consists of several mobile robots, in other words platforms, which need to transport materials in a workhouse. The goal of each mobile platform is to carry the specified object to a set position. These appointed goals are not predefined and can be changed according to the needs of the user. Working in a dynamic environment, numerous agents with different tasks to complete can be exposed to many obstacles which may be the cause of accidents. For this reason, a careful path planning is required in such environments. The suggested path planning algorithm for this system is A*. A* is a fast path finder, which can navigate quite well in a planar environment, but it is not favorable for dynamical settings. Therefore, a combination of the A* algorithm with a collision avoidance method is proposed for overcoming these difficulties. By doing this, the A* algorithm is expanded to work in dynamical situations and can assure the convergence of any agent towards their goal. This fusion of both, the path finding algorithm and the collision avoidance method, can aid the cooperation of the agents and improve the efficiency of the system as a whole.

Most existing formation control approaches are based on the assumption that the global/relative position and/or velocity measurements of mobile robots are directly available. To extend the application domain and to improve the formation control performance, it is extremely necessary to avoid the use of position and velocity measurements in the design of formation controllers. In this paper, we propose new leader-following formation tracking control schemes for nonholonomic mobile robots with onboard perspective cameras, without using both position and velocity measurements. To address the unavailability issue of position measurements, the leader-follower kinematics model in the image space is developed, which can facilitate the complete elimination of measurement/estimation of the position information. Furthermore, feedback information from the perspective camera of the follower robot is used to design adaptive observers to estimate the leader linear velocity for feedforward compensation, which can handle the absence of velocity measurements such that the proposed schemes can be applied to control formations of mobile robots without mutual communication abilities. By using the Lyapunov stability theory, a rigorous stability analysis based on the nonlinear formation dynamics is provided to show that the global stability of the combined observer-controller closed-loop system can be guaranteed. Both simulation and experimental results are also given to demonstrate the performance of the proposed formation tracking control schemes.

Consider a dynamic task allocation problem, where tasks are unknowingly distributed over an environment. is paper considers each task comprising two sequential subtasks: detection and completion, where each subtask can only be carried out by a certain type of agent. We address this problem using a novel nature-inspired approach called "hunter and gatherer." e proposed method employs two complementary teams of agents: one agile in detecting (hunters) and another skillful in completing (gatherers) the tasks. To minimize the collective cost of task accomplishments in a distributed manner, a game-theoretic solution is introduced to couple agents from complementary teams. We utilize market-based negotiation models to develop incentive-based decision-making algorithms relying on innovative notions of "certainty and uncertainty profit margins." e simulation results demonstrate that employing two complementary teams of hunters and gatherers can effectually improve the number of tasks completed by agents compared to conventional methods, while the collective cost of accomplishments is minimized. In addition, the stability and efficacy of the proposed solutions are studied using Nash equilibrium analysis and statistical analysis, respectively. It is also numerically shown that the proposed solutions function fairly; that is, for each type of agent, the overall workload is distributed equally.

Résumé/Abstract Un robot mobile autonome doit réaliser des tâches non répétitives dans un environnement imparfaitement connu et non-coopératif, voire hostile. Dans ce contexte les missions attribuées au robot ne peuvent être définies de façon précise, et le robot doit être ...

A distributed control mechanism for ground moving nonholonomic robots is proposed. It enables a group of mobile robots to autonomously manage formation shapes while navigating through environments with obstacles. The formation can be maintained without the need of any inter-robot communication. Obstacle avoidance is designed to be performed by the individual robots themselves. Formation scaling is implemented to ensure the formation shape is maintained for as long as possible. If the formation fails to hold its shape when navigating through environments with obstacles, formation morphing has been incorporated to preserve the interconnectivity of the robots, thus reducing the possibility of losing robots from the formation. The algorithm has been implemented on a nonholonomic multi-robot system for empirical analysis. Experimental results demonstrate formations completing an obstacle course within 12 seconds with zero collisions. Furthermore, the system is capable of withstanding up to 25% sensor noise.

The IEEE International Conference on Self-Adapting and Self-Organizing Systems (SASO) is the main forum for studying and discussing the foundations of a principled approach to engineering systems, networks, and services based on self-adaptation and self-organization. Over the past decade, it has consolidated as the primary scientific conference for sharing ideas on algorithms, technologies, tools, and applications across a wide range of scientific fields. In 2016, the conference was hosted by the University of Augsburg, in Augsburg, Germany; its scientific program comprised full papers, short papers, poster and demo presentations, workshops, doctoral symposium and tutorials. This special issue of ACM TAAS champions some of the most solid research results of SASO 2016, presenting selected, revised, and extended best articles.

Decentralized partially observable Markov decision processes (Dec-POMDPs) provide a general model for decision-making under uncertainty in cooperative decentralized settings, but are difficult to solve optimally (NEXP-Complete). As a new way of solving these problems, we introduce the idea of transforming a Dec-POMDP into a continuous-state deterministic MDP with a piecewise-linear and convex value function. This approach makes use of the fact that planning can be accomplished in a centralized offline manner, while execution can still be distributed. This new Dec-POMDP formulation, which we call an occupancy MDP, allows powerful POMDP and continuous-state MDP methods to be used for the first time. When the curse of dimensionality becomes too prohibitive, we refine this basic approach and present ways to combine heuristic search and compact representations that exploit the structure present in multiagent domains, without losing the ability to eventually converge to an optimal solution. In particular, we introduce feature-based heuristic search that relies on feature-based compact representations, point-based updates and efficient action selection. A theoretical analysis demonstrates that our feature-based heuristic search algorithms terminate in finite time with an optimal solution. We include an extensive empirical analysis using well known benchmarks, thereby demonstrating our approach provides significant scalability improvements compared to the state of the art.

One of the major challenges in the coordination of large, open, collaborative, and commercial vehicle fleets is dynamic task allocation. Self-concerned individually rational vehicle drivers have both local and global objectives, which require coordination using some fair and efficient task allocation method. In this paper, we review the literature on scalable and dynamic task allocation focusing on deterministic and dynamic two-dimensional linear assignment problems. We focus on multiagent system representation of open vehicle fleets where dynamically appearing vehicles are represented by software agents that should be allocated to a set of dynamically appearing tasks. We give a comparison and critical analysis of recent research results focusing on centralized, distributed, and decentralized solution approaches. Moreover, we propose mathematical models for dynamic versions of the following assignment problems well known in combinatorial optimization: the assignment problem, bottleneck assignment problem, fair matching problem, dynamic minimum deviation assignment problem, Σ k-assignment problem, the semiassignment problem, the assignment problem with side constraints, and the assignment problem while recognizing agent qualification; all while considering the main aspect of open vehicle fleets: random arrival of tasks and vehicles (agents) that may become available after assisting previous tasks or by participating in the fleet at times based on individual interest.

We propose an autonomous docking system for a mobile robot which helps the robot to dock at a specific location with a particular orientation. This system uses active IR beacons which transmit infrared signals all through out the workspace. The beacons are placed in such a way that the docking target forms the midpoint of the line joining the beacons. The robot is equipped with an Ultrasonic Range Meter (SRF05) and a set of infrared receivers. The robot, using all its sensors, detects the beacons and successfully reaches the docking-target with the required orientation. Several experiments are conducted and the results prove the efficacy of the docking system.

Obstacle avoidance is an essential problem for applications involving multiple wheeled mobile robots. This research proposes a simple obstacle avoidance rule utilizing only one type of sensor, i.e., infrared sensor. In this research, multiple infrared sensors are placed on a mobile robot, arranged 45° radially equidistance. By using a low-cost and easily available infrared sensor, the cost and time consumed to build and repair a wheeled mobile robot are considerably reduced. Avoiding rules, based on simple behavior, such as "turn", "stop", "follow", and "ignore" are developed. By applying these rules, each robot can refer to the motion of other robot or stationary object to avoid collision. Simulation results show that the proposed algorithm performs well, at 66.7% chance of avoiding a moving object and at 93% chance of avoiding a stationary object.

This paper addresses the problem of coordinating great numbers of vehicles in large geographical areas under network connective constraints. We leverage previous work on hierarchical potential fields to create advanced skills in multirobot systems. Skills group together various field objectives to accomplish the performance requirements in response to highlevel commands. Our framework calculates trajectories that comply with priority constraints while optimizing the desired task objectives in their null spaces. We use a model-based dynamics approach that provides a direct map from field objectives to vehicle accelerations, yielding smooth and accurate trajectory generation. We develop a real-time software system that implements the proposed methods and simulates the coordinated behaviors in a 3D graphical environment. To validate the methodology, we simulate a large exploration task and demonstrate that we can effectively enforce the required constraints while optimizing the exploration goals.

There is a growing interest in intelligent assistants for a variety of applications from sorting email to helping people with disabilities to do their daily chores. In this paper, we formulate the problem of intelligent assistance in a decision-theoretic framework, and present both theoretical and empirical results. We first introduce a class of POMDPs called hidden-goal MDPs (HGMDPs), which formalizes the problem of interactively assisting an agent whose goal is hidden and whose actions are observable. In spite of its restricted nature, we show that optimal action selection for HGMDPs is PSPACE-complete even for deterministic dynamics. We then introduce a more restricted model called helper action MDPs (HAMDPs), which are sufficient for modeling many real-world problems. We show classes of HAMDPs for which efficient algorithms are possible. More interestingly, for general HAMDPs we show that a simple myopic policy achieves a near optimal regret, compared to an oracle assistant that knows the agent's goal. We then introduce more sophisticated versions of this policy for the general case of HGMDPs that we combine with a novel approach for quickly learning about the agent being assisted. We evaluate our approach in two game-like computer environments where human subjects perform tasks, and in a real-world domain of providing assistance during folder navigation in a computer desktop environment. The results show that in all three domains the framework results in an assistant that substantially reduces user effort with only modest computation.

This paper describes the development of a pair of homogeneous mobile robot using AT89C51 microcontroller with two 6V DC motors to address the box-pushing problem. In this work, two homogeneous mobile robots have been fully developed exclusively to address the box-pushing application that is described as "Pushing a box like solid item to reach the goal position already defined by more than one mobile robot through the cooperation among them". Teaching the mobile robots using reinforcement learning is easier, because of specifying a reward function for every task completed by them.

This paper aims to present the Multi-Agent System to Control and Coordinate teAmworking Robots (MAS2CAR), a new architecture to control a group of coordinated autonomous robots in unstructured environments. MAS2CAR covers two main layers: (i) the Control Layer and we focus on (ii) the Coordination Layer. The control module is responsible for a part of the decision making process while taking into account robot's structural constraints. Despite this autonomy possibility, the Coordination Layer manages the robots in order to bring cooperative behavior and to allow teamwork. In this paper we present a scenario validating our approach based upon the multi-agent systems (MAS). Thanks to its reliability we have chosen the M OISE Inst organizational model and we will present how it can be used for this use-case. Moreover, regarding to the implementation part, we have retained U TOPIA, a framework which automatically build a MAS thanks to a M OISE Inst specification.

In this paper, we study a particular subclass of partially observable models, called quasi-deterministic partially observable Markov de- cision processes (QDET-POMDPs), characterized by deterministic transitions and stochastic observations. While this framework does not model the same general problems as POMDPs, they still cap- ture a number of interesting and challenging problems and have, in some cases, interesting properties. By studying the observability available in this subclass, we suggest that QDET-POMDPs may fall many steps in the complexity hierarchy. An extension of this frame- work to the decentralized case also reveals a subclass of numerous problems that can be approximated in polynomial space. Finally, a sketch of "-optimal algorithms for these classes of problems is given and empirically evaluated.

Researchers have introduced the Dynamic Distributed Constraint Optimization Problem (Dynamic DCOP) formulation to model dynamically changing multi-agent coordination problems, where a dynamic DCOP is a sequence of (static canonical) DCOPs, each partially different from the DCOP preceding it. Existing work typically assumes that the problem in each time step is decoupled from the problems in other time steps, which might not hold in some applications. Therefore, in this paper, we make the following contributions: (i) We introduce a new model, called Markovian Dynamic DCOPs (MD-DCOPs), where the DCOP in the next time step is a function of the value assignments in the current time step; (ii) We introduce two distributed reinforcement learning algorithms, the Distributed RVI Q-learning algorithm and the Distributed R-learning algorithm, that balance exploration and exploitation to solve MD-DCOPs in an online manner; and (iii) We empirically evaluate them against an existing multi-arm ba...

By forming a specific formation during motion, the robotic swarm is a good candidate for unknown region exploration applications. The members of this kind of system are generally low complexity, which limits the communication and perception capacities of the agents. How to merge to the desired formation under those constraints is essential for performing relevant tasks. In this paper, a limited visual field constrained formation control strategy inspired by flying geese coordinated motion is introduced. Usually, they flock together in a V-shape formations, which is a well-studied phenomenon in biology and bionics. This paper illustrates the proposed methods by taking the research results from the above subjects and mapping them from the swarm engineering point of view. The formation control is achieved by applying a behavior-based formation forming method with the finite state machine while considering anti-collision and obstacle avoidance. Furthermore, a cascade leader–follower str...

The paper presents a new approach for multi-agent replanning based on Distributed Constraint Satisfaction (DisCSP) and Graph planning techniques. In this approach, a new distributed refinement strategy is proposed to construct a graph plan for fixing errors occurred during the plan execution. The strategy employs an "max-branching" heuristic that can reduce the final graph plan size and allow faster completion time for the graph construction. The graph plan is then compiled into a DisCSP problem and solved using a multi-variable version of the Asynchronous Backtracking Algorithm. The approach is demonstrated with experiments which show that distributed planning graph and CSP can practically solve the replanning problems in a multi-agent environment.

Executive Summary This white paper on two currently hot areas on Cooperative Robotics research (Network Robot Systems and Formal Models and Methods) aims to:(1) survey the state of the art for the two areas,(2) list in a justified manner their expected advances in the upcoming ten years,(3) identify the application topics of interest for Europe so as to keep its research competitive at the international level, and (4) recommend lines of action for the support of EU research in Cooperative Robotics in the future. For each of the areas, this ...

Exploration of an unknown environment is one of the major applications of Multi-Robot Systems. Many works have proposed multi-robot coordination algorithms to accomplish exploration missions based on multi-agent techniques. Some of these works focus on multi-robot exploration under communication constraints. In this paper, we propose an original way to formalize and solve this issue. Our proposal relies on distributed constraint satisfaction problems (disCSP) which are an extension of classical constraint satisfaction problems (CSP). Compared to other works, our proposal is fully distributed and guaranties the exploration of an unknown environment with maintenance of connectivity between all the members of a robots' team.

Decentralized POMDPs provide an expressive framework for multi-agent sequential decision making. While finite-horizon DEC-POMDPs have enjoyed significant success, progress remains slow for the infinite-horizon case mainly due to the inherent complexity of optimizing stochastic controllers representing agent policies. We present a promising new class of algorithms for the infinite-horizon case, which recasts the optimization problem as inference in a mixture of DBNs. An attractive feature of this approach is the straightforward adoption of existing inference techniques in DBNs for solving DEC-POMDPs and supporting richer representations such as factored or continuous states and actions. We also derive the Expectation Maximization (EM) algorithm to optimize the joint policy represented as DBNs. Experiments on benchmark domains show that EM compares favorably against the state-of-the-art solvers.

Decentralized POMDPs provide an expressive framework for sequential multi-agent decision making. Despite their high complexity, there has been significant progress in scaling up existing algorithms, largely due to the use of pointbased methods. Performing point-based backup is a fundamental operation in state-of-the-art algorithms. We show that even a single backup step in the multi-agent setting is NP-Complete. Despite this negative worst-case result, we present an efficient and scalable optimal algorithm as well as a principled approximation scheme. The optimal algorithm exploits recent advances in the weighted CSP literature to overcome the complexity of the backup operation. The polytime approximation scheme provides a constant factor approximation guarantee based on the number of belief points. In experiments on standard domains, the optimal approach provides significant speedup (up to 2 orders of magnitude) over the previous best optimal algorithm and is able to increase the number of belief points by more than a factor of 3. The approximation scheme also works well in practice, providing near-optimal solutions to the backup problem.

Decentralized POMDPs provide an expressive framework for sequential multi-agent decision making. Despite their high complexity, there has been significant progress in scal- ing up existing algorithms, largely due to the use of point- based methods. Performing point-based backup is a funda- mental operation in state-of-the-art algorithms. We show that even a single backup step in the multi-agent setting is NP-Complete.

Coordination is an essential characteristic of any system, either natural or artificial, that is composed of multiple interacting agents. The mechanism by which the coordination is achieved determines such properties as how robust the system is to environmental perturbations, how efficient it is in performing a given task, and, more generally, what types of coordinated tasks can be achieved. A formal understanding of these issues will permit more focused analysis of coordinated natural systems and more effective approaches to the design of coordinated artificial systems. Toward this end, we are systematically investigating coordination in the context of multi-robot systems. We present a formal framework for multi-robot system (MRS) coordination and use it to discuss and reason about coordination. Using this principled foundation, we are developing a suite of general methods by which to automatically synthesize controllers for robots constituting a MRS such that a given task is performed in a coordinated fashion. This paper presents a method for the automatic synthesis of a specific type of controller, one that retains some internal state (e.g., memory of past occurrences) but is not capable of communication. Understanding the capabilities and limitations of a coordinated system composed of individual agents with such properties contributes to the understanding of when memory alone is sufficient to achieve the desired coordination and when other capabilities, such as communication, may be necessary. We validate our formal approach to the study of coordination in a multi-robot construction task domain through the use of both physically-realistic simulations and real-robot demonstrations.

Anytime algorithms allow a system to trade solution quality for computation time. In previous work, monitoring techniques have been developed to allow agents to stop the computation at the "right" time so as to optimize a given time-dependent utility function. However, these results apply only to the single-agent case. In this paper we analyze the problems that arise when several agents solve components of a larger problem, each using an anytime algorithm. Monitoring in this case is more challenging as each agent is uncertain about the progress made so far by the others. We develop a formal framework for decentralized monitoring, establish the complexity of several interesting variants of the problem, and propose solution techniques for each one. Finally, we show that the framework can be applied to decentralized flow and planning problems.

Metareasoning has been used as a means for achieving bounded rationality by optimizing the tradeoff between the cost and value of the decision making process. Effective monitoring techniques have been developed to allow agents to stop their computation at the "right" time so as to optimize the overall time-dependent utility of the decision. However, these methods were designed for a single decision maker. In this chapter, we analyze the problems that arise when several agents solve components of a larger problem, each using an anytime algorithm. Metareasoning is more challenging in this case because each agent is uncertain about the progress made so far by the others. We develop a formal framework for decentralized monitoring of decision making, establish the complexity of several interesting variants of the problem, and propose solution techniques for each case.

Recently, auction methods have been investigated as effective, decentralized methods for multi-robot coordination. Experimental research has shown great potential, but has not been complemented yet by theoretical analysis. In this paper we contribute a theoretical analysis of the performance of auction methods for multi-robot routing. We suggest a generic framework for auction-based multi-robot routing and analyze a variety of bidding rules for different team objectives. This is the first time that auction methods are shown to offer theoretical guarantees for such a variety of bidding rules and team objectives.

Frontier-based exploration methods are efficient for multi-robot exploration systems. In this paper, enhanced frontier-based techniques are used with team of cooperating mobile robots to explore unknown environment. The aim of the exploration algorithm is to minimize the exploration time by coordinating the robots to maximize overall utility by minimizing the potential of overlapping in information gained amongst the robots. The proposed frontier-based exploration algorithm is based on a new bidding function to decrease the overlap between the robots in addition to the utility and cost parameters. A special parameter depends on the future positions of the robots is also considered. The proposed algorithm has been tested with different environments. We also compared it to A* algorithm. Comparisons demonstrated the efficiency of proposed algorithm.

Aiming at the formation and path planning of multirobot systems in an unknown environment, a path planning method for multirobot formation based on improved Q -learning is proposed. Based on the leader-following approach, the leader robot uses an improved Q -learning algorithm to plan the path and the follower robot achieves a tracking strategy of gravitational potential field (GPF) by designing a cost function to select actions. Specifically, to improve the Q-learning, Q -value is initialized by environmental guidance of the target’s GPF. Then, the virtual obstacle-filling avoidance strategy is presented to fill non-obstacles which is judged to tend to concave obstacles with virtual obstacles. Besides, the simulated annealing (SA) algorithm whose controlling temperature is adjusted in real time according to the learning situation of the Q -learning is applied to improve the action selection strategy. The experimental results show that the improved Q -learning algorithm reduces the ...