PROJECT TITLE :

JouleMR: Towards Cost-Effective and Green-Aware Data Processing Frameworks - 2018

ABSTRACT:

Interests are growing in energy management of the cluster effectively so as to reduce the energy consumption also because the electricity cost. Renewable energy and dynamic pricing schemes in sensible grids are two major emerging trends in energy markets. But, current information processing frameworks aren't awake to the potency of every joule consumed by the information center workloads in the context of those 2 major trends. In fact, not all joules are equal in the way that the quantity of labor which will be done by a joule can vary considerably in data centers. Ignoring this reality ends up in significant energy waste (by twenty five percent of the total energy consumption in Hadoop YARN on a Facebook production trace per our study). In this Project, we tend to propose JouleMR, a value-effective and inexperienced-aware knowledge processing framework. Specifically, we investigate how to take advantage of such joule potency to maximise the advantages of renewable energy furthermore dynamic pricing schemes for MapReduce framework. We have a tendency to develop job/task scheduling algorithms with a particular focus on the factors on joule potency in the info center, together with the energy efficiency of MapReduce workloads, renewable energy offer, dynamic pricing and the battery usage. We tend to further develop a easy however effective performanceenergy consumption model to guide our scheduling decisions. We have a tendency to have implemented JouleMR on prime of Hadoop YARN. The experiments demonstrate the accuracy of our models, and therefore the effectiveness of our value-effective and green-aware optimizations outperform the state-of-the-art implementations over Hadoop YARN.