PROJECT TITLE :

Trust-based Scheduling Framework for Big Data Processing with MapReduce

ABSTRACT:

Security and privacy have emerged as major concerns in relation to cloud computing platforms because users run the risk of their personal information becoming publicly available. The data could leak while it is being processed, while it is being stored, or while it is being moved either within a cloud or between different cloud infrastructures, such as when going from a private cloud to a public cloud. The protection of data "in processing" is the primary focus of this paper. The MapReduce framework has been shown to be an effective solution for big data applications, and it has seen widespread use in a variety of settings, including healthcare and business data analysis, amongst others. The purpose of this article is to present a trust-based framework for using MapReduce in the processing of large amounts of data. To be more specific, we first quantify the sensitive values for the data slots and the trust values for the map and reduce slots, and then we propose to assign those values. After that, we determine the level of trustworthiness associated with each resource utilized in the big data processing tasks. A certain level of trust is necessary for completing a task, and this level of trust is proportional to the level of sensitivity of the data involved in the task (i.e., more sensitive data requires servers/slots with a higher trust level). The MapReduce scheduling problem is then formulated as the maximum weighted matching problem of a bipartite graph. The goal of this problem is to maximize the total trust value over all possible assignments while taking into account the varying trust requirements of the various tasks. It is well known that this problem is NP-hard. In order to circumvent this issue, we have discovered that all of the slots that make up a computing node (VM) share the same trust value that is awarded from the phase of secured transformation. The number of slot nodes in a weight bipartite graph can be decreased as a result of this. Taking advantage of this fact, we have developed a powerful heuristic algorithm that, when applied, achieves 94.7 percent of the optimal solution that can only be found through exhaustive searching. Extensive simulations have shown that the trust-based scheduling scheme offers significantly better protection for the confidentiality of data while also guaranteeing satisfactory performance for applications that deal with large amounts of data.