NexTech 2021 Congress
October 03, 2021 to October 07, 2021 - Barcelona, Spain

  • UBICOMM 2021, The Fifteenth International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies
  • ADVCOMP 2021, The Fifteenth International Conference on Advanced Engineering Computing and Applications in Sciences
  • SEMAPRO 2021, The Fifteenth International Conference on Advances in Semantic Processing
  • AMBIENT 2021, The Eleventh International Conference on Ambient Computing, Applications, Services and Technologies
  • EMERGING 2021, The Thirteenth International Conference on Emerging Networks and Systems Intelligence
  • DATA ANALYTICS 2021, The Tenth International Conference on Data Analytics
  • GLOBAL HEALTH 2021, The Tenth International Conference on Global Health Challenges
  • CYBER 2021, The Sixth International Conference on Cyber-Technologies and Cyber-Systems

SoftNet 2021 Congress
October 03, 2021 to October 07, 2021 - Barcelona, Spain

  • ICSEA 2021, The Sixteenth International Conference on Software Engineering Advances
  • ICSNC 2021, The Sixteenth International Conference on Systems and Networks Communications
  • CENTRIC 2021, The Fourteenth International Conference on Advances in Human-oriented and Personalized Mechanisms, Technologies, and Services
  • VALID 2021, The Thirteenth International Conference on Advances in System Testing and Validation Lifecycle
  • SIMUL 2021, The Thirteenth International Conference on Advances in System Simulation
  • SOTICS 2021, The Eleventh International Conference on Social Media Technologies, Communication, and Informatics
  • INNOV 2021, The Tenth International Conference on Communications, Computation, Networks and Technologies
  • HEALTHINFO 2021, The Sixth International Conference on Informatics and Assistive Technologies for Health-Care, Medical Support and Wellbeing

NetWare 2021 Congress
November 14, 2021 to November 18, 2021 - Athens, Greece

  • SENSORCOMM 2021, The Fifteenth International Conference on Sensor Technologies and Applications
  • SENSORDEVICES 2021, The Twelfth International Conference on Sensor Device Technologies and Applications
  • SECURWARE 2021, The Fifteenth International Conference on Emerging Security Information, Systems and Technologies
  • AFIN 2021, The Thirteenth International Conference on Advances in Future Internet
  • CENICS 2021, The Fourteenth International Conference on Advances in Circuits, Electronics and Micro-electronics
  • ICQNM 2021, The Fifteenth International Conference on Quantum, Nano/Bio, and Micro Technologies
  • FASSI 2021, The Seventh International Conference on Fundamentals and Advances in Software Systems Integration
  • GREEN 2021, The Sixth International Conference on Green Communications, Computing and Technologies

TrendNews 2021 Congress
November 14, 2021 to November 18, 2021 - Athens, Greece

  • CORETA 2021, Advances on Core Technologies and Applications
  • DIGITAL 2021, Advances on Societal Digital Transformation

 


ThinkMind // CLOUD COMPUTING 2021, The Twelfth International Conference on Cloud Computing, GRIDs, and Virtualization // View article cloud_computing_2021_3_30_20014


Take Me to the Clouds Above: Bridging On Site HPC with Clouds for Capacity Workloads

Authors:
Jay Lofstead
Dmitry Duplyakin

Keywords: Cloud; HPC; Cloud Bursting; schedulers;

Abstract:
Sites with limited compute cluster capacity aimed at supporting large-scale applications of scientific and parallel computing must deal with the additional demand for small-scale jobs---in many cases, single-node and coming in large volumes---that help further develop the capabilities of large-scale applications as well as run simple data analysis tasks. Many of these analysis tasks and other small-scale jobs are run on High Performance Computing (HPC) systems because they offer familiar environments, making future scaling convenient, or because the proximity to the input or output data sets is important. When these small-scale jobs explode in count and create significant competition for resources, ensuring that the most effective use of the cluster resources is achieved requires either non-trivial scheduler tuning for better management of job mixes or adoption of entirely different models for management of computing environments. We claim that a hybrid ``on-site HPC + cloud'' model can offer the best of both worlds. Thus, small jobs should target cloud resources in cases where they can be offloaded to clouds without significant penalties on performance, and, at the same time, large-scale application runs can better utilize HPC clusters, achieving lower wait times and increasing job throughput. This is not without challenges.

Pages: 54 to 59

Copyright: Copyright (c) The Government of USA, 2021. Used by permission to IARIA.

Publication date: April 18, 2021

Published in: conference

ISSN: 2308-4294

ISBN: 978-1-61208-845-7

Location: Porto, Portugal

Dates: from April 18, 2021 to April 22, 2021

SERVICES CONTACT
2010 - 2017 © ThinkMind. All rights reserved.
Read Terms of Service and Privacy Policy.