Providing Cloud Resources for Running Composite Applications in the Workflow-as-a-Service Paradigm
Keywords:
cloud computing, workflow management system, scheduling, provision of resources, virtual machine, container
Abstract
The article analyzes the existing models, methods, and algorithmic support for performing science-intensive applications on cloud platforms within the framework of the new concept called Workflow-as-a-Service (WaaS). Being so-called multitenant environments, the WaaS platforms offer the possibility to implement efficient mechanisms for managing continuous and heterogeneous workflows in cloud computing. A number of most important aspects are considered: the availability of various IaaS providers, which offer different types of resources; geographic distribution of data processing centers; heterogeneity of workflows; the need to implement the “pay-per-use” model for a specific user; and solution of the problem of deploying virtual machines on physical servers and setting up a multitude of containers in these machines, with each container ready for being used by tasks from different flows. The article also points out promising lines in the development of methods for providing the cloud platform resources based on technologies for deploying virtual machines and containers to run composite applications within the framework of the actively developing WaaS paradigm.
References
1. Workflows Community Summit [Электрон. ресурс] https://workflowsri.org/summits/community (дата обращения 24.02.2023).
2. Existing Workflow Systems [Электрон. ресурс] https://s.apache.org/existing-workflow-systems (дата обращения 24.02.2023).
3. Rodriguez M.A., Buyya R. Scheduling Dynamic Workloads in Multi-tenant Scientific Workflow as a Service Platforms // Future Generation Computer Systems. 2018. V. 79. Pp. 739—750.
4. Hilman M.H., Rodriguez M.A., Buyya R. Workflow-as-a-Service Cloud Platform and Deployment of Bioinformatics Workflow Applications [Электрон. ресурс] https://www.researchgate.net/scientific-contributions/Maria-A-Rodriguez-2114894132 (дата обращения 24.02.2023).
5. Abrishami S., Naghibzadeh M., Epema D.H. Deadline-constrained Workflow Scheduling Algorithms for Infrastructure as a Service Clouds // Future Generation Computer Systems. 2013. V. 29(1). Pp. 158—169.
6. Calheiros R.N., Buyya R. Meeting Deadlines of Scientific Workflows in Public Clouds with Tasks Replication // IEEE Trans. Parallel Distrib. Syst. 2014. V. 25(7). Pp. 1787—1796.
7. Liu S., Ren K., Deng K., Song J. A Task Backfill-based Scientific Workflow Scheduling Strategy on Cloud Platform // Proc. Intern. Conf. Information Sci. and Technol. 2016. Pp. 105—110.
8. Deldari A., Naghibzadeh M., Abrishami S. CCA: a Deadline-constrained Workflow Scheduling Algorithm for Multicore Resources on the Cloud // J. Supercomput. 2017. V. 73(2). Pp. 756—781.
9. Maechling P. e. a. SCEC CyberShake Workflows — Аutomating Probabilistic Seismic Hazard Analysis Сalculations // Workflows for E-Science. N.-Y.: Springer, 2007. Pp. 143—163.
10. Deelman E. e. a. The Cost of Doing Science on the Cloud: the Montage Example // Proc. ACM/IEEE Conf. Supercomputing. 2008. P. 50.
11. Vockler J.-S. e. a. Experiences Using Cloud Computing for a Scientific Workflow Application // Proc. Intern. Workshop Sci. Cloud Computing. 2011. Pp. 15—24.
12. Malawski M., Juve G., Deelman E., Nabrzyski J. Algorithms for Cost- and Deadline-constrained Provisioning for Scientific Workflow Ensembles in IaaS Clouds // Future Generation Computer Syst. 2015. V. 48. Pp. 1—18.
13. Pietri I. e. a. Energy-constrained Provisioning for Scientific Workflow Ensembles // Proc. International Conf. Cloud and Green Computing. 2013. Pp. 34—41.
14. Jiang Q., Lee Y.C., Zomaya A.Y. Executing Large Scale Scientific Workflow Ensembles in Public Clouds // Proc. Intern. Conf. Parallel Proc. 2015. Pp. 520—529.
15. Bryk P., Malawski M., Juve G., Deelman E. Storage-aware Algorithms for Scheduling of Workflow Ensembles in Clouds // J. Grid Comput. 2016. V. 14(2). Pp. 359—378.
16. Yu Z., Shi W. A Planner-guided Scheduling Strategy for Multiple Workflow Applications // Proc. Intern. Conf. Parallel Processing. 2008. Pp. 1—8.
17. Jiang H.-J. e. a. Scheduling Concurrent Workflows in HPC Cloud through Exploiting Schedule Gaps // Proc. Intern. Conf. Algorithms and Architectures for Parallel Processing. 2011. Pp. 282—293.
18. Stavrinides G.L., Karatza H.D. A Cost-effective and QoS-aware Approach to Scheduling Real-time Workflow Applications in PaaS and SaaS Clouds // Proc. Intern. Conf. Future Internet of Things and Cloud. 2015. Pp. 231—239.
19. Xu M., Cui L., Wang H., Bi Y. A Multiple QoS-constrained Scheduling Strategy of Multiple Workflows for Cloud Computing // Proc. IEEE Intern. Symp. Parallel and Distributed Processing with Appl. 2009. Pp. 629—634.
20. Chen W., Lee Y.C., Fekete A., Zomaya A.Y. Adaptive Multiple-workflow Scheduling with Task Rearrangement // J. Supercomput. 2015. V. 71(4). Pp. 1297—1317.
21. Zhou A.C., He B., Liu C. Monetary Cost Optimizations for Hosting Workflow-as-a-Service in IaaS Clouds // IEEE Trans. Cloud Comput. 2016. V. 4(1). Pp. 34—48.
22. Mao M., Humphrey M. Auto-scaling to Minimize Cost and Meet Application Deadlines in Cloud Workflows // Proc. Intern. Conf. High Performance Computing, Networking, Storage and Analysis. 2011. P. 49.
23. Shi J., Luo J., Dong F., Zhang J. A Budget and Deadline-aware Scientific Workflow Resource Provisioning and Scheduling Mechanism for Cloud // Proc. IEEE Intern. Conf. Computer Supported Cooperative Work in Design. 2014. Pp. 672—677.
24. Wang J. e. a. Workflow as a Service in the Cloud: Architecture and Scheduling Algorithms // Proc. Comput. Sci. 2014. V. 29. Pp. 546—556.
25. Gerlach W. e. a. Skyport: Container-based Execution Environment Management for Multi-cloud Scientific Workflows // Proc. Intern. Workshop on Data-Intensive Computing in the Clouds. 2014. Pp. 25—32.
26. Filgueira R. e. a. Asterism: Pegasus and Dispel4py Hybrid Workflows for Data-intensive Science // Proc. Intern. Workshop Data-Intensive Computing in the Cloud. 2016. Pp. 1—8.
27. Esteves S., Veiga L. Waas: Workflow-as-a-service for the Cloud with Scheduling of Continuous and Data-intensive Workflows // Comput. J. 2016. V. 59(3). Pp. 371—383.
28. Hashem I.A.T. e. a. The Rise of “Big Data” on Cloud Computing: Review and Open Research Issues // Inf. Syst. 2015. V. 47. Pp. 98—115.
29. Senyo P.K., Addae E., Boateng R. Cloud Computing Research: a Review of Research Themes, Frameworks, Methods and Future Research Directions // Int. J. Inf. Manag. 2018. V. 38(1). Pp. 128—139.
30. Stergiou C., Psannis K.E., Kim B.G., Gupta B. Secure Integration of IoT and Cloud Computing // Future Gener. Comput. Syst. 2018. V. 78. Pp. 964—975.
31. Kumar M.R.V., Raghunathan S. Heterogeneity and Thermal-aware Adaptive Heuristics for Energy Efficient Consolidation of Virtual Infrastructure Clouds // J. Comput. Syst. Sci. 2016. V. 82(2). Pp. 191—212.
32. Lopez-Pires F., Baran B. Virtual Machine Placement Literature Review [Электрон. ресурс] https://arxiv.org/abs/1506.01509 (дата обращения 24.02.2023).
33. Usmani Z., Singh S. A Survey of Virtual Machine Placement Techniques in a Cloud Data Center // Proc. Comput. Sci. 2016. V. 78. Pp. 491—498.
34. Liu X.F. e. a. An Energy Efficient ant Colony System for Virtual Machine Placement in Cloud Computing // IEEE Trans. Evol. Comput. 2016. V. 22(1). Pp. 113—128.
35. Abdel-Basset M., Abdle-Fatah L., Sangaiah A.K. An Improved Lévy Based Whale Optimization Algorithm for Bandwidth-efficient Virtual Machine Placement in Cloud Computing Environment // Cluster Computing. 2018. V. 22(1). Pp. 1—16.
36. Shabeera T.P., Kumar S.M., Salam S.M., Krishnan K.M. Optimizing VM Allocation and Data Placement for Data-intensive Applications in Cloud using ACO Metaheuristic Algorithm // Eng. Sci. Technol. Int. J. 2017. V. 20(2). Pp. 616—628.
37. Abdelaziz A. e. a. Intelligent Algorithms for Optimal Selection of Virtual Machine in Cloud Environment, Towards Enhance Healthcare Services // Proc. Intern. Conf. Advanced Intelligent Systems and Informatics. 2017. Pp. 289—298.
38. Ghobaei-Arani M., Shamsi M., Rahmanian A.A. An Efficient Approach for Improving Virtual Machine Placement in Cloud Computing Environment // J. Exp. Theor. Artif. Intell. 2017. V. 29(6). Pp. 1149—171.
39. Saber T., Thorburn J., Murphy L., Ventresque A. VM Reassignment in Hybrid Clouds for Large Decentralized Companies: a Multi-objective Challenge // Future Gener. Comput. Syst. 2018. V. 79. Pp. 751—764.
40. Gao Y. e. a. A Multi-objective ant Colony System Algorithm for Virtual Machine Placement in Cloud Computing // J. Comput. Syst. Sci. 2013. V. 79(8). Pp. 1230—1242.
41. Ganesan T., Vasant P., Litvinchev I. Chaotic Simulator for Bilevel Optimization of Virtual Machine Placements in Cloud Computing // J. Operations Research Soc. China. 2022. V. 10(4). Pp. 703—723.
42. Deelman E. e. a. The Future of Scientific Workflows // Int. J. High Perform. Comput. Appl. 2018. V. 32(1). Pp. 159—175.
43. Chen W., Ferreira da Silva R., Livny M., Wenger K. Pegasus: a Workflow Management System for Science Automation // Futur. Gener. Comput. Syst. 2015. V. 46. Pp. 17—35.
44. Apache Airflow Documentation [Электрон. ресурс] https://airflow.apache.org/ (дата обращения 24.02.2023).
45. Hull D. e. a. Taverna: a Tool for Building and Running Workflows of Services // Nucleic Acids Research. 2006. V. 34. Pp. 729—732.
46. Altintas I. e. a. Kepler: an Extensible System for Design and Execution of Scientific Workflows // Proc. XVI Intern. Conf. Scientific and Statistical Database Management. 2004. Pp. 423—424.
47. Kramer M., Wurz H.M., Altenhofen C. Executing Cyclic Scientific Workflows in the Cloud // J. Cloud Computing: Advances, Systems and Appl. 2021. V. 10(25). Pp. 1—26.
48. Toporkov V., Yemelyanov D., Toporkova A. Coordinated Global and Private Job-flow Scheduling in Grid Virtual Organizations // Simulation Modelling Practice and Theory. 2021. V. 107(14). P. 102228.
49. Toporkov V., Yemelyanov D. Micro-scheduling for Dependable Resources Allocation // Performance Evaluation Models for Distributed Service Networks. Studies in Systems, Decision and Control. 2021. V. 343. Pp. 81—105.
50. Toporkov V., Yemelyanov D., Grigorenko M. Optimization of Resources Allocation in High Performance Computing under Utilization Uncertainty // Proc. Intern. Conf. Computational Sci. 2021. V. 14747. Pp. 540—553.
51. Toporkov V., Yemelyanov D., Bulkhak A. Machine Learning-Based Scheduling and Resources Allocation in Distributed Computing // Proc. Internat. Conf. Computational Sci. 2022. V. 13353. Pp. 3—16.
---
Для цитирования: Топорков В.В., Емельянов Д.М., Булхак А.Н. Предоставление ресурсов облачных платформ для выполнения композитных приложений в парадигме «поток работ как сервис» // Вестник МЭИ. 2023. № 5. С. 156—168. DOI: 10.24160/1993-6982-2023-5-156-168
---
Работа выполнена при поддержке: Российского научного фонда (грант № 22-21-00372), https://rscf.ru/project/22-21-00372/
#
1. Workflows Community Summit [Electron. Resurs] https://workflowsri.org/summits/community (Data Obrashcheniya 24.02.2023).
2. Existing Workflow Systems [Electron. Resurs] https://s.apache.org/existing-workflow-systems (Data Obrashcheniya 24.02.2023).
3. Rodriguez M.A., Buyya R. Scheduling Dynamic Workloads in Multi-tenant Scientific Workflow as a Service Platforms. Future Generation Computer Systems. 2018;79:739—750.
4. Hilman M.H., Rodriguez M.A., Buyya R. Workflow-as-a-Service Cloud Platform and Deployment of Bioinformatics Workflow Applications [Electron. Resurs] https://www.researchgate.net/scientific-contributions/Maria-A-Rodriguez-2114894132 (Data Obrashcheniya 24.02.2023).
5. Abrishami S., Naghibzadeh M., Epema D.H. Deadline-constrained Workflow Scheduling Algorithms for Infrastructure as a Service Clouds. Future Generation Computer Systems. 2013;29(1):158—169.
6. Calheiros R.N., Buyya R. Meeting Deadlines of Scientific Workflows in Public Clouds with Tasks Replication. IEEE Trans. Parallel Distrib. Syst. 2014;25(7):1787—1796.
7. Liu S., Ren K., Deng K., Song J. A Task Backfill-based Scientific Workflow Scheduling Strategy on Cloud Platform. Proc. Intern. Conf. Information Sci. and Technol. 2016:105—110.
8. Deldari A., Naghibzadeh M., Abrishami S. CCA: a Deadline-constrained Workflow Scheduling Algorithm for Multicore Resources on the Cloud. J. Supercomput. 2017;73(2):756—781.
9. Maechling P. e. a. SCEC CyberShake Workflows — Аutomating Probabilistic Seismic Hazard Analysis Сalculations. Workflows for E-Science. N.-Y.: Springer, 2007:143—163.
10. Deelman E. e. a. The Cost of Doing Science on the Cloud: the Montage Example. Proc. ACM/IEEE Conf. Supercomputing. 2008:50.
11. Vockler J.-S. e. a. Experiences Using Cloud Computing for a Scientific Workflow Application. Proc. Intern. Workshop Sci. Cloud Computing. 2011:15—24.
12. Malawski M., Juve G., Deelman E., Nabrzyski J. Algorithms for Cost- and Deadline-constrained Provisioning for Scientific Workflow Ensembles in IaaS Clouds. Future Generation Computer Syst. 2015;48:1—18.
13. Pietri I. e. a. Energy-constrained Provisioning for Scientific Workflow Ensembles. Proc. International Conf. Cloud and Green Computing. 2013:34—41.
14. Jiang Q., Lee Y.C., Zomaya A.Y. Executing Large Scale Scientific Workflow Ensembles in Public Clouds. Proc. Intern. Conf. Parallel Proc. 2015:520—529.
15. Bryk P., Malawski M., Juve G., Deelman E. Storage-aware Algorithms for Scheduling of Workflow Ensembles in Clouds. J. Grid Comput. 2016;14(2):359—378.
16. Yu Z., Shi W. A Planner-guided Scheduling Strategy for Multiple Workflow Applications. Proc. Intern. Conf. Parallel Processing. 2008:1—8.
17. Jiang H.-J. e. a. Scheduling Concurrent Workflows in HPC Cloud through Exploiting Schedule Gaps. Proc. Intern. Conf. Algorithms and Architectures for Parallel Processing. 2011:282—293.
18. Stavrinides G.L., Karatza H.D. A Cost-effective and QoS-aware Approach to Scheduling Real-time Workflow Applications in PaaS and SaaS Clouds. Proc. Intern. Conf. Future Internet of Things and Cloud. 2015:231—239.
19. Xu M., Cui L., Wang H., Bi Y. A Multiple QoS-constrained Scheduling Strategy of Multiple Workflows for Cloud Computing. Proc. IEEE Intern. Symp. Parallel and Distributed Processing with Appl. 2009:629—634.
20. Chen W., Lee Y.C., Fekete A., Zomaya A.Y. Adaptive Multiple-workflow Scheduling with Task Rearrangement. J. Supercomput. 2015;71(4):1297—1317.
21. Zhou A.C., He B., Liu C. Monetary Cost Optimizations for Hosting Workflow-as-a-Service in IaaS Clouds. IEEE Trans. Cloud Comput. 2016;4(1):34—48.
22. Mao M., Humphrey M. Auto-scaling to Minimize Cost and Meet Application Deadlines in Cloud Workflows. Proc. Intern. Conf. High Performance Computing, Networking, Storage and Analysis. 2011:49.
23. Shi J., Luo J., Dong F., Zhang J. A Budget and Deadline-aware Scientific Workflow Resource Provisioning and Scheduling Mechanism for Cloud. Proc. IEEE Intern. Conf. Computer Supported Cooperative Work in Design. 2014:672—677.
24. Wang J. e. a. Workflow as a Service in the Cloud: Architecture and Scheduling Algorithms. Proc. Comput. Sci. 2014;29:546—556.
25. Gerlach W. e. a. Skyport: Container-based Execution Environment Management for Multi-cloud Scientific Workflows. Proc. Intern. Workshop on Data-Intensive Computing in the Clouds. 2014:25—32.
26. Filgueira R. e. a. Asterism: Pegasus and Dispel4py Hybrid Workflows for Data-intensive Science. Proc. Intern. Workshop Data-Intensive Computing in the Cloud. 2016:1—8.
27. Esteves S., Veiga L. Waas: Workflow-as-a-service for the Cloud with Scheduling of Continuous and Data-intensive Workflows. Comput. J. 2016;59(3):371—383.
28. Hashem I.A.T. e. a. The Rise of “Big Data” on Cloud Computing: Review and Open Research Issues. Inf. Syst. 2015;47:98—115.
29. Senyo P.K., Addae E., Boateng R. Cloud Computing Research: a Review of Research Themes, Frameworks, Methods and Future Research Directions. Int. J. Inf. Manag. 2018;38(1):128—139.
30. Stergiou C., Psannis K.E., Kim B.G., Gupta B. Secure Integration of IoT and Cloud Computing. Future Gener. Comput. Syst. 2018;78:964—975.
31. Kumar M.R.V., Raghunathan S. Heterogeneity and Thermal-aware Adaptive Heuristics for Energy Efficient Consolidation of Virtual Infrastructure Clouds. J. Comput. Syst. Sci. 2016;82(2):191—212.
32. Lopez-Pires F., Baran B. Virtual Machine Placement Literature Review [Electron. Resurs] https://arxiv.org/abs/1506.01509 (Data Obrashcheniya 24.02.2023).
33. Usmani Z., Singh S. A Survey of Virtual Machine Placement Techniques in a Cloud Data Center. Proc. Comput. Sci. 2016;78:491—498.
34. Liu X.F. e. a. An Energy Efficient ant Colony System for Virtual Machine Placement in Cloud Computing. IEEE Trans. Evol. Comput. 2016;22(1):113—128.
35. Abdel-Basset M., Abdle-Fatah L., Sangaiah A.K. An Improved Lévy Based Whale Optimization Algorithm for Bandwidth-efficient Virtual Machine Placement in Cloud Computing Environment. Cluster Computing. 2018;22(1):1—16.
36. Shabeera T.P., Kumar S.M., Salam S.M., Krishnan K.M. Optimizing VM Allocation and Data Placement for Data-intensive Applications in Cloud using ACO Metaheuristic Algorithm. Eng. Sci. Technol. Int. J. 2017;20(2):616—628.
37. Abdelaziz A. e. a. Intelligent Algorithms for Optimal Selection of Virtual Machine in Cloud Environment, Towards Enhance Healthcare Services. Proc. Intern. Conf. Advanced Intelligent Systems and Informatics. 2017:289—298.
38. Ghobaei-Arani M., Shamsi M., Rahmanian A.A. An Efficient Approach for Improving Virtual Machine Placement in Cloud Computing Environment. J. Exp. Theor. Artif. Intell. 2017;29(6):1149—171.
39. Saber T., Thorburn J., Murphy L., Ventresque A. VM Reassignment in Hybrid Clouds for Large Decentralized Companies: a Multi-objective Challenge. Future Gener. Comput. Syst. 2018;79:751—764.
40. Gao Y. e. a. A Multi-objective ant Colony System Algorithm for Virtual Machine Placement in Cloud Computing. J. Comput. Syst. Sci. 2013;79(8):1230—1242.
41. Ganesan T., Vasant P., Litvinchev I. Chaotic Simulator for Bilevel Optimization of Virtual Machine Placements in Cloud Computing. J. Operations Research Soc. China. 2022;10(4):703—723.
42. Deelman E. e. a. The Future of Scientific Workflows. Int. J. High Perform. Comput. Appl. 2018;32(1):159—175.
43. Chen W., Ferreira da Silva R., Livny M., Wenger K. Pegasus: a Workflow Management System for Science Automation. Futur. Gener. Comput. Syst. 2015;46:17—35.
44. Apache Airflow Documentation [Electron. Resurs] https://airflow.apache.org/ (Data Obrashcheniya 24.02.2023).
45. Hull D. e. a. Taverna: a Tool for Building and Running Workflows of Services. Nucleic Acids Research. 2006;34:729—732.
46. Altintas I. e. a. Kepler: an Extensible System for Design and Execution of Scientific Workflows. Proc. XVI Intern. Conf. Scientific and Statistical Database Management. 2004:423—424.
47. Kramer M., Wurz H.M., Altenhofen C. Executing Cyclic Scientific Workflows in the Cloud. J. Cloud Computing: Advances, Systems and Appl. 2021;10(25):1—26.
48. Toporkov V., Yemelyanov D., Toporkova A. Coordinated Global and Private Job-flow Scheduling in Grid Virtual Organizations. Simulation Modelling Practice and Theory. 2021;107(14):102228.
49. Toporkov V., Yemelyanov D. Micro-scheduling for Dependable Resources Allocation. Performance Evaluation Models for Distributed Service Networks. Studies in Systems, Decision and Control. 2021;343:81—105.
50. Toporkov V., Yemelyanov D., Grigorenko M. Optimization of Resources Allocation in High Performance Computing under Utilization Uncertainty. Proc. Intern. Conf. Computational Sci. 2021;14747:540—553.
51. Toporkov V., Yemelyanov D., Bulkhak A. Machine Learning-Based Scheduling and Resources Allocation in Distributed Computing. Proc. Internat. Conf. Computational Sci. 2022;13353:3—16
---
For citation: Toporkov V.V., Yemelyanov D.M., Bulkhak A.N. Providing Cloud Resources for Running Composite Applications in the Workflow-as-a-Service Paradigm. Bulletin of MPEI. 2023;5:156—168. (in Russian). DOI: 10.24160/1993-6982-2023-5-156-168
---
The work is executed at support: Russian Science Foundation (Grant No. 22-21-00372), https://rscf.ru/project/22-21-00372/
2. Existing Workflow Systems [Электрон. ресурс] https://s.apache.org/existing-workflow-systems (дата обращения 24.02.2023).
3. Rodriguez M.A., Buyya R. Scheduling Dynamic Workloads in Multi-tenant Scientific Workflow as a Service Platforms // Future Generation Computer Systems. 2018. V. 79. Pp. 739—750.
4. Hilman M.H., Rodriguez M.A., Buyya R. Workflow-as-a-Service Cloud Platform and Deployment of Bioinformatics Workflow Applications [Электрон. ресурс] https://www.researchgate.net/scientific-contributions/Maria-A-Rodriguez-2114894132 (дата обращения 24.02.2023).
5. Abrishami S., Naghibzadeh M., Epema D.H. Deadline-constrained Workflow Scheduling Algorithms for Infrastructure as a Service Clouds // Future Generation Computer Systems. 2013. V. 29(1). Pp. 158—169.
6. Calheiros R.N., Buyya R. Meeting Deadlines of Scientific Workflows in Public Clouds with Tasks Replication // IEEE Trans. Parallel Distrib. Syst. 2014. V. 25(7). Pp. 1787—1796.
7. Liu S., Ren K., Deng K., Song J. A Task Backfill-based Scientific Workflow Scheduling Strategy on Cloud Platform // Proc. Intern. Conf. Information Sci. and Technol. 2016. Pp. 105—110.
8. Deldari A., Naghibzadeh M., Abrishami S. CCA: a Deadline-constrained Workflow Scheduling Algorithm for Multicore Resources on the Cloud // J. Supercomput. 2017. V. 73(2). Pp. 756—781.
9. Maechling P. e. a. SCEC CyberShake Workflows — Аutomating Probabilistic Seismic Hazard Analysis Сalculations // Workflows for E-Science. N.-Y.: Springer, 2007. Pp. 143—163.
10. Deelman E. e. a. The Cost of Doing Science on the Cloud: the Montage Example // Proc. ACM/IEEE Conf. Supercomputing. 2008. P. 50.
11. Vockler J.-S. e. a. Experiences Using Cloud Computing for a Scientific Workflow Application // Proc. Intern. Workshop Sci. Cloud Computing. 2011. Pp. 15—24.
12. Malawski M., Juve G., Deelman E., Nabrzyski J. Algorithms for Cost- and Deadline-constrained Provisioning for Scientific Workflow Ensembles in IaaS Clouds // Future Generation Computer Syst. 2015. V. 48. Pp. 1—18.
13. Pietri I. e. a. Energy-constrained Provisioning for Scientific Workflow Ensembles // Proc. International Conf. Cloud and Green Computing. 2013. Pp. 34—41.
14. Jiang Q., Lee Y.C., Zomaya A.Y. Executing Large Scale Scientific Workflow Ensembles in Public Clouds // Proc. Intern. Conf. Parallel Proc. 2015. Pp. 520—529.
15. Bryk P., Malawski M., Juve G., Deelman E. Storage-aware Algorithms for Scheduling of Workflow Ensembles in Clouds // J. Grid Comput. 2016. V. 14(2). Pp. 359—378.
16. Yu Z., Shi W. A Planner-guided Scheduling Strategy for Multiple Workflow Applications // Proc. Intern. Conf. Parallel Processing. 2008. Pp. 1—8.
17. Jiang H.-J. e. a. Scheduling Concurrent Workflows in HPC Cloud through Exploiting Schedule Gaps // Proc. Intern. Conf. Algorithms and Architectures for Parallel Processing. 2011. Pp. 282—293.
18. Stavrinides G.L., Karatza H.D. A Cost-effective and QoS-aware Approach to Scheduling Real-time Workflow Applications in PaaS and SaaS Clouds // Proc. Intern. Conf. Future Internet of Things and Cloud. 2015. Pp. 231—239.
19. Xu M., Cui L., Wang H., Bi Y. A Multiple QoS-constrained Scheduling Strategy of Multiple Workflows for Cloud Computing // Proc. IEEE Intern. Symp. Parallel and Distributed Processing with Appl. 2009. Pp. 629—634.
20. Chen W., Lee Y.C., Fekete A., Zomaya A.Y. Adaptive Multiple-workflow Scheduling with Task Rearrangement // J. Supercomput. 2015. V. 71(4). Pp. 1297—1317.
21. Zhou A.C., He B., Liu C. Monetary Cost Optimizations for Hosting Workflow-as-a-Service in IaaS Clouds // IEEE Trans. Cloud Comput. 2016. V. 4(1). Pp. 34—48.
22. Mao M., Humphrey M. Auto-scaling to Minimize Cost and Meet Application Deadlines in Cloud Workflows // Proc. Intern. Conf. High Performance Computing, Networking, Storage and Analysis. 2011. P. 49.
23. Shi J., Luo J., Dong F., Zhang J. A Budget and Deadline-aware Scientific Workflow Resource Provisioning and Scheduling Mechanism for Cloud // Proc. IEEE Intern. Conf. Computer Supported Cooperative Work in Design. 2014. Pp. 672—677.
24. Wang J. e. a. Workflow as a Service in the Cloud: Architecture and Scheduling Algorithms // Proc. Comput. Sci. 2014. V. 29. Pp. 546—556.
25. Gerlach W. e. a. Skyport: Container-based Execution Environment Management for Multi-cloud Scientific Workflows // Proc. Intern. Workshop on Data-Intensive Computing in the Clouds. 2014. Pp. 25—32.
26. Filgueira R. e. a. Asterism: Pegasus and Dispel4py Hybrid Workflows for Data-intensive Science // Proc. Intern. Workshop Data-Intensive Computing in the Cloud. 2016. Pp. 1—8.
27. Esteves S., Veiga L. Waas: Workflow-as-a-service for the Cloud with Scheduling of Continuous and Data-intensive Workflows // Comput. J. 2016. V. 59(3). Pp. 371—383.
28. Hashem I.A.T. e. a. The Rise of “Big Data” on Cloud Computing: Review and Open Research Issues // Inf. Syst. 2015. V. 47. Pp. 98—115.
29. Senyo P.K., Addae E., Boateng R. Cloud Computing Research: a Review of Research Themes, Frameworks, Methods and Future Research Directions // Int. J. Inf. Manag. 2018. V. 38(1). Pp. 128—139.
30. Stergiou C., Psannis K.E., Kim B.G., Gupta B. Secure Integration of IoT and Cloud Computing // Future Gener. Comput. Syst. 2018. V. 78. Pp. 964—975.
31. Kumar M.R.V., Raghunathan S. Heterogeneity and Thermal-aware Adaptive Heuristics for Energy Efficient Consolidation of Virtual Infrastructure Clouds // J. Comput. Syst. Sci. 2016. V. 82(2). Pp. 191—212.
32. Lopez-Pires F., Baran B. Virtual Machine Placement Literature Review [Электрон. ресурс] https://arxiv.org/abs/1506.01509 (дата обращения 24.02.2023).
33. Usmani Z., Singh S. A Survey of Virtual Machine Placement Techniques in a Cloud Data Center // Proc. Comput. Sci. 2016. V. 78. Pp. 491—498.
34. Liu X.F. e. a. An Energy Efficient ant Colony System for Virtual Machine Placement in Cloud Computing // IEEE Trans. Evol. Comput. 2016. V. 22(1). Pp. 113—128.
35. Abdel-Basset M., Abdle-Fatah L., Sangaiah A.K. An Improved Lévy Based Whale Optimization Algorithm for Bandwidth-efficient Virtual Machine Placement in Cloud Computing Environment // Cluster Computing. 2018. V. 22(1). Pp. 1—16.
36. Shabeera T.P., Kumar S.M., Salam S.M., Krishnan K.M. Optimizing VM Allocation and Data Placement for Data-intensive Applications in Cloud using ACO Metaheuristic Algorithm // Eng. Sci. Technol. Int. J. 2017. V. 20(2). Pp. 616—628.
37. Abdelaziz A. e. a. Intelligent Algorithms for Optimal Selection of Virtual Machine in Cloud Environment, Towards Enhance Healthcare Services // Proc. Intern. Conf. Advanced Intelligent Systems and Informatics. 2017. Pp. 289—298.
38. Ghobaei-Arani M., Shamsi M., Rahmanian A.A. An Efficient Approach for Improving Virtual Machine Placement in Cloud Computing Environment // J. Exp. Theor. Artif. Intell. 2017. V. 29(6). Pp. 1149—171.
39. Saber T., Thorburn J., Murphy L., Ventresque A. VM Reassignment in Hybrid Clouds for Large Decentralized Companies: a Multi-objective Challenge // Future Gener. Comput. Syst. 2018. V. 79. Pp. 751—764.
40. Gao Y. e. a. A Multi-objective ant Colony System Algorithm for Virtual Machine Placement in Cloud Computing // J. Comput. Syst. Sci. 2013. V. 79(8). Pp. 1230—1242.
41. Ganesan T., Vasant P., Litvinchev I. Chaotic Simulator for Bilevel Optimization of Virtual Machine Placements in Cloud Computing // J. Operations Research Soc. China. 2022. V. 10(4). Pp. 703—723.
42. Deelman E. e. a. The Future of Scientific Workflows // Int. J. High Perform. Comput. Appl. 2018. V. 32(1). Pp. 159—175.
43. Chen W., Ferreira da Silva R., Livny M., Wenger K. Pegasus: a Workflow Management System for Science Automation // Futur. Gener. Comput. Syst. 2015. V. 46. Pp. 17—35.
44. Apache Airflow Documentation [Электрон. ресурс] https://airflow.apache.org/ (дата обращения 24.02.2023).
45. Hull D. e. a. Taverna: a Tool for Building and Running Workflows of Services // Nucleic Acids Research. 2006. V. 34. Pp. 729—732.
46. Altintas I. e. a. Kepler: an Extensible System for Design and Execution of Scientific Workflows // Proc. XVI Intern. Conf. Scientific and Statistical Database Management. 2004. Pp. 423—424.
47. Kramer M., Wurz H.M., Altenhofen C. Executing Cyclic Scientific Workflows in the Cloud // J. Cloud Computing: Advances, Systems and Appl. 2021. V. 10(25). Pp. 1—26.
48. Toporkov V., Yemelyanov D., Toporkova A. Coordinated Global and Private Job-flow Scheduling in Grid Virtual Organizations // Simulation Modelling Practice and Theory. 2021. V. 107(14). P. 102228.
49. Toporkov V., Yemelyanov D. Micro-scheduling for Dependable Resources Allocation // Performance Evaluation Models for Distributed Service Networks. Studies in Systems, Decision and Control. 2021. V. 343. Pp. 81—105.
50. Toporkov V., Yemelyanov D., Grigorenko M. Optimization of Resources Allocation in High Performance Computing under Utilization Uncertainty // Proc. Intern. Conf. Computational Sci. 2021. V. 14747. Pp. 540—553.
51. Toporkov V., Yemelyanov D., Bulkhak A. Machine Learning-Based Scheduling and Resources Allocation in Distributed Computing // Proc. Internat. Conf. Computational Sci. 2022. V. 13353. Pp. 3—16.
---
Для цитирования: Топорков В.В., Емельянов Д.М., Булхак А.Н. Предоставление ресурсов облачных платформ для выполнения композитных приложений в парадигме «поток работ как сервис» // Вестник МЭИ. 2023. № 5. С. 156—168. DOI: 10.24160/1993-6982-2023-5-156-168
---
Работа выполнена при поддержке: Российского научного фонда (грант № 22-21-00372), https://rscf.ru/project/22-21-00372/
#
1. Workflows Community Summit [Electron. Resurs] https://workflowsri.org/summits/community (Data Obrashcheniya 24.02.2023).
2. Existing Workflow Systems [Electron. Resurs] https://s.apache.org/existing-workflow-systems (Data Obrashcheniya 24.02.2023).
3. Rodriguez M.A., Buyya R. Scheduling Dynamic Workloads in Multi-tenant Scientific Workflow as a Service Platforms. Future Generation Computer Systems. 2018;79:739—750.
4. Hilman M.H., Rodriguez M.A., Buyya R. Workflow-as-a-Service Cloud Platform and Deployment of Bioinformatics Workflow Applications [Electron. Resurs] https://www.researchgate.net/scientific-contributions/Maria-A-Rodriguez-2114894132 (Data Obrashcheniya 24.02.2023).
5. Abrishami S., Naghibzadeh M., Epema D.H. Deadline-constrained Workflow Scheduling Algorithms for Infrastructure as a Service Clouds. Future Generation Computer Systems. 2013;29(1):158—169.
6. Calheiros R.N., Buyya R. Meeting Deadlines of Scientific Workflows in Public Clouds with Tasks Replication. IEEE Trans. Parallel Distrib. Syst. 2014;25(7):1787—1796.
7. Liu S., Ren K., Deng K., Song J. A Task Backfill-based Scientific Workflow Scheduling Strategy on Cloud Platform. Proc. Intern. Conf. Information Sci. and Technol. 2016:105—110.
8. Deldari A., Naghibzadeh M., Abrishami S. CCA: a Deadline-constrained Workflow Scheduling Algorithm for Multicore Resources on the Cloud. J. Supercomput. 2017;73(2):756—781.
9. Maechling P. e. a. SCEC CyberShake Workflows — Аutomating Probabilistic Seismic Hazard Analysis Сalculations. Workflows for E-Science. N.-Y.: Springer, 2007:143—163.
10. Deelman E. e. a. The Cost of Doing Science on the Cloud: the Montage Example. Proc. ACM/IEEE Conf. Supercomputing. 2008:50.
11. Vockler J.-S. e. a. Experiences Using Cloud Computing for a Scientific Workflow Application. Proc. Intern. Workshop Sci. Cloud Computing. 2011:15—24.
12. Malawski M., Juve G., Deelman E., Nabrzyski J. Algorithms for Cost- and Deadline-constrained Provisioning for Scientific Workflow Ensembles in IaaS Clouds. Future Generation Computer Syst. 2015;48:1—18.
13. Pietri I. e. a. Energy-constrained Provisioning for Scientific Workflow Ensembles. Proc. International Conf. Cloud and Green Computing. 2013:34—41.
14. Jiang Q., Lee Y.C., Zomaya A.Y. Executing Large Scale Scientific Workflow Ensembles in Public Clouds. Proc. Intern. Conf. Parallel Proc. 2015:520—529.
15. Bryk P., Malawski M., Juve G., Deelman E. Storage-aware Algorithms for Scheduling of Workflow Ensembles in Clouds. J. Grid Comput. 2016;14(2):359—378.
16. Yu Z., Shi W. A Planner-guided Scheduling Strategy for Multiple Workflow Applications. Proc. Intern. Conf. Parallel Processing. 2008:1—8.
17. Jiang H.-J. e. a. Scheduling Concurrent Workflows in HPC Cloud through Exploiting Schedule Gaps. Proc. Intern. Conf. Algorithms and Architectures for Parallel Processing. 2011:282—293.
18. Stavrinides G.L., Karatza H.D. A Cost-effective and QoS-aware Approach to Scheduling Real-time Workflow Applications in PaaS and SaaS Clouds. Proc. Intern. Conf. Future Internet of Things and Cloud. 2015:231—239.
19. Xu M., Cui L., Wang H., Bi Y. A Multiple QoS-constrained Scheduling Strategy of Multiple Workflows for Cloud Computing. Proc. IEEE Intern. Symp. Parallel and Distributed Processing with Appl. 2009:629—634.
20. Chen W., Lee Y.C., Fekete A., Zomaya A.Y. Adaptive Multiple-workflow Scheduling with Task Rearrangement. J. Supercomput. 2015;71(4):1297—1317.
21. Zhou A.C., He B., Liu C. Monetary Cost Optimizations for Hosting Workflow-as-a-Service in IaaS Clouds. IEEE Trans. Cloud Comput. 2016;4(1):34—48.
22. Mao M., Humphrey M. Auto-scaling to Minimize Cost and Meet Application Deadlines in Cloud Workflows. Proc. Intern. Conf. High Performance Computing, Networking, Storage and Analysis. 2011:49.
23. Shi J., Luo J., Dong F., Zhang J. A Budget and Deadline-aware Scientific Workflow Resource Provisioning and Scheduling Mechanism for Cloud. Proc. IEEE Intern. Conf. Computer Supported Cooperative Work in Design. 2014:672—677.
24. Wang J. e. a. Workflow as a Service in the Cloud: Architecture and Scheduling Algorithms. Proc. Comput. Sci. 2014;29:546—556.
25. Gerlach W. e. a. Skyport: Container-based Execution Environment Management for Multi-cloud Scientific Workflows. Proc. Intern. Workshop on Data-Intensive Computing in the Clouds. 2014:25—32.
26. Filgueira R. e. a. Asterism: Pegasus and Dispel4py Hybrid Workflows for Data-intensive Science. Proc. Intern. Workshop Data-Intensive Computing in the Cloud. 2016:1—8.
27. Esteves S., Veiga L. Waas: Workflow-as-a-service for the Cloud with Scheduling of Continuous and Data-intensive Workflows. Comput. J. 2016;59(3):371—383.
28. Hashem I.A.T. e. a. The Rise of “Big Data” on Cloud Computing: Review and Open Research Issues. Inf. Syst. 2015;47:98—115.
29. Senyo P.K., Addae E., Boateng R. Cloud Computing Research: a Review of Research Themes, Frameworks, Methods and Future Research Directions. Int. J. Inf. Manag. 2018;38(1):128—139.
30. Stergiou C., Psannis K.E., Kim B.G., Gupta B. Secure Integration of IoT and Cloud Computing. Future Gener. Comput. Syst. 2018;78:964—975.
31. Kumar M.R.V., Raghunathan S. Heterogeneity and Thermal-aware Adaptive Heuristics for Energy Efficient Consolidation of Virtual Infrastructure Clouds. J. Comput. Syst. Sci. 2016;82(2):191—212.
32. Lopez-Pires F., Baran B. Virtual Machine Placement Literature Review [Electron. Resurs] https://arxiv.org/abs/1506.01509 (Data Obrashcheniya 24.02.2023).
33. Usmani Z., Singh S. A Survey of Virtual Machine Placement Techniques in a Cloud Data Center. Proc. Comput. Sci. 2016;78:491—498.
34. Liu X.F. e. a. An Energy Efficient ant Colony System for Virtual Machine Placement in Cloud Computing. IEEE Trans. Evol. Comput. 2016;22(1):113—128.
35. Abdel-Basset M., Abdle-Fatah L., Sangaiah A.K. An Improved Lévy Based Whale Optimization Algorithm for Bandwidth-efficient Virtual Machine Placement in Cloud Computing Environment. Cluster Computing. 2018;22(1):1—16.
36. Shabeera T.P., Kumar S.M., Salam S.M., Krishnan K.M. Optimizing VM Allocation and Data Placement for Data-intensive Applications in Cloud using ACO Metaheuristic Algorithm. Eng. Sci. Technol. Int. J. 2017;20(2):616—628.
37. Abdelaziz A. e. a. Intelligent Algorithms for Optimal Selection of Virtual Machine in Cloud Environment, Towards Enhance Healthcare Services. Proc. Intern. Conf. Advanced Intelligent Systems and Informatics. 2017:289—298.
38. Ghobaei-Arani M., Shamsi M., Rahmanian A.A. An Efficient Approach for Improving Virtual Machine Placement in Cloud Computing Environment. J. Exp. Theor. Artif. Intell. 2017;29(6):1149—171.
39. Saber T., Thorburn J., Murphy L., Ventresque A. VM Reassignment in Hybrid Clouds for Large Decentralized Companies: a Multi-objective Challenge. Future Gener. Comput. Syst. 2018;79:751—764.
40. Gao Y. e. a. A Multi-objective ant Colony System Algorithm for Virtual Machine Placement in Cloud Computing. J. Comput. Syst. Sci. 2013;79(8):1230—1242.
41. Ganesan T., Vasant P., Litvinchev I. Chaotic Simulator for Bilevel Optimization of Virtual Machine Placements in Cloud Computing. J. Operations Research Soc. China. 2022;10(4):703—723.
42. Deelman E. e. a. The Future of Scientific Workflows. Int. J. High Perform. Comput. Appl. 2018;32(1):159—175.
43. Chen W., Ferreira da Silva R., Livny M., Wenger K. Pegasus: a Workflow Management System for Science Automation. Futur. Gener. Comput. Syst. 2015;46:17—35.
44. Apache Airflow Documentation [Electron. Resurs] https://airflow.apache.org/ (Data Obrashcheniya 24.02.2023).
45. Hull D. e. a. Taverna: a Tool for Building and Running Workflows of Services. Nucleic Acids Research. 2006;34:729—732.
46. Altintas I. e. a. Kepler: an Extensible System for Design and Execution of Scientific Workflows. Proc. XVI Intern. Conf. Scientific and Statistical Database Management. 2004:423—424.
47. Kramer M., Wurz H.M., Altenhofen C. Executing Cyclic Scientific Workflows in the Cloud. J. Cloud Computing: Advances, Systems and Appl. 2021;10(25):1—26.
48. Toporkov V., Yemelyanov D., Toporkova A. Coordinated Global and Private Job-flow Scheduling in Grid Virtual Organizations. Simulation Modelling Practice and Theory. 2021;107(14):102228.
49. Toporkov V., Yemelyanov D. Micro-scheduling for Dependable Resources Allocation. Performance Evaluation Models for Distributed Service Networks. Studies in Systems, Decision and Control. 2021;343:81—105.
50. Toporkov V., Yemelyanov D., Grigorenko M. Optimization of Resources Allocation in High Performance Computing under Utilization Uncertainty. Proc. Intern. Conf. Computational Sci. 2021;14747:540—553.
51. Toporkov V., Yemelyanov D., Bulkhak A. Machine Learning-Based Scheduling and Resources Allocation in Distributed Computing. Proc. Internat. Conf. Computational Sci. 2022;13353:3—16
---
For citation: Toporkov V.V., Yemelyanov D.M., Bulkhak A.N. Providing Cloud Resources for Running Composite Applications in the Workflow-as-a-Service Paradigm. Bulletin of MPEI. 2023;5:156—168. (in Russian). DOI: 10.24160/1993-6982-2023-5-156-168
---
The work is executed at support: Russian Science Foundation (Grant No. 22-21-00372), https://rscf.ru/project/22-21-00372/
Published
2023-06-06
Section
Computing systems and their elements (technical sciences) (2.3.2.)
