PROJECT TITLE :

Asynchronous Stochastic Approximation Based Learning Algorithms for As-You-Go Deployment of Wireless Relay Networks Along a Line - 2018

ABSTRACT:

We are motivated by the need, in emergency situations, for impromptu (or “as-you-go”) deployment of multihop wireless networks, by human agents or robots (e.g., unmanned aerial vehicles (UAVs)); the agent moves along a line, makes wireless link quality measurements at regular intervals, and makes on-line placement decisions using these measurements. As a initial step, we have a tendency to have formulated such deployment along a line as a sequential decision drawback. In our earlier work, reported in [one], we have a tendency to proposed two doable deployment approaches: (i) the pure as-you-go approach where the deployment agent can only move forward, and (ii) the explore-forward approach where the deployment agent explores some successive steps and then selects the most effective relay placement location among them. The latter was shown to produce higher performance (in terms of network cost, network performance, and power expenditure), but at the expense of additional measurements and deployment time, which makes explore-forward impractical for quick deployment by an energy constrained agent such as a UAV. Any, since in emergency things the terrain would be unknown, the deployment algorithm ought to not need a-priori knowledge of the parameters of the wireless propagation model. In [one], we have a tendency to, so, developed learning algorithms for the explore-forward approach. The present paper fills in an important gap by providing deploy-and-learn algorithms for the pure as-you-go approach. We tend to formulate the sequential relay deployment drawback as a mean price Markov call process (MDP), which trades off among power consumption, link outage chances, and the number of relay nodes within the deployed network. Whereas the pure as-you-go deployment drawback was previously formulated as a reduced value MDP (see [one]), the discounted cost MDP formulation was not amenable for learning algorithms that are proposed during this Project. During this Project, initial we tend to show structural results for the optimal policy equivalent to the typical value MDP, and offer new insights into the optimal policy. Next, by exploiting the special structure of the average cost optimality equation and by using the theory of asynchronous stochastic approximation (in single and 2 timescale), we have a tendency to develop two learning algorithms that asymptotically converge to the set of optimal policies as deployment progresses. Numerical results show reasonably quick speed of convergence, and hence the model-free algorithms will be useful for practical, quick deployment of emergency wireless networks.