An agent-based hierarchical approach for executing bag-of-tasks in clouds

Abstract

Unlike message-passing applications, »bag-of-tasks« applications (BoTs), whose tasks are unrelated and independent (no inter-task communication), can be highly parallelized and executed in any acceptable order. A common practice when executing bag-of-tasks applications (BoT) is to exploit the master-slave topology. Cloud environments offer some features that facilitate the execution of BoT applications. One of the approaches to control-cloud resources is to use agents that are flexible actors in a dynamic environment. Given these assumptions, we have designed a combination of approaches which can be classified as distributed, hierarchical solutions to the issue of scalable execution of bag-of-tasks. The concept of our system relates to a project that is focused on processing huge quantities of incoming data from a network of sensors through the Internet. Our aim is to create a mechanism for processing such data as a system that executes jobs while exploiting load balancing for cloud resources which use applications such as Eucalyptus. The idea is to create a hybrid architecture which takes advantage of some centralized parts of the system and full distributedness in other parts. On the other hand, we balance dependencies between system components using a hierarchical master-slave structure.