A hyper-heuristic cost optimisation approach for Scientific Workflow Scheduling in cloud computing

Effective management of Scientific Workflow Scheduling (SWFS) processes in a cloud environment remains a challenging task when dealing with large and complex Scientific Workflow Applications (SWFAs). Cost optimisation of SWFS benefits cloud service consumers and providers by reducing temporal and monetary costs in processing SWFAs. However, cost optimisation performance of SWFS approaches is affected by the inherent nature of the SWFA as well as various types of scenarios that depend on the number of available virtual machines and varied sizes of SWFA datasets. Cost optimisation performance of existing SWFS approaches is still not satisfactory for all considered scenarios. Thus, there is a need to propose a dynamic hyper-heuristic approach that can effectively optimise the cost of SWFS for all different scenarios. This can be done by employing different meta-heuristic algorithms in order to utilise their strengths for each scenario. Thus, the main objective of this paper is to propose a Completion Time Driven Hyper-Heuristic (CTDHH) approach for cost optimisation of SWFS in a cloud environment. The CTDHH approach employs four well-known population-based meta-heuristic algorithms, which act as Low Level Heuristic (LLH) algorithms. In addition, the CTDHH approach enhances the native random selection way of existing hyper-heuristic approaches by incorporating the best computed workflow completion time to act as a high-level selector to dynamically pick a suitable algorithm from the pool of LLH algorithms after each run. A real-world cloud based experimentation environment has been considered to evaluate the performance of the proposed CTDHH approach by comparing it with five baseline approaches, i.e. four population-based approaches and an existing hyper-heuristic approach named Hyper-Heuristic Scheduling Algorithm (HHSA). Several different scenarios have also been considered to evaluate data-intensiveness and computation-intensive performance. Based on the results of the experimental comparison, the proposed approach has proven to yield the most effective performance results for all considered experimental scenarios.