DataSys 2017
June 25 - 29, 2017- Venice, Italy

  • AICT 2017, The Thirteenth Advanced International Conference on Telecommunications
  • ICIW 2017, The Twelfth International Conference on Internet and Web Applications and Services
  • ICIMP 2017, The Twelfth International Conference on Internet Monitoring and Protection
  • SMART 2017, The Sixth International Conference on Smart Cities, Systems, Devices and Technologies
    • URBAN COMPUTING 2017, The International Symposium on Emerging Frontiers of Urban Computing and Smart Cities
  • IMMM 2017, The Seventh International Conference on Advances in Information Mining and Management
    • DATASETS 2017, The International Symposium on Designing, Validating, and Using Datasets
  • INFOCOMP 2017, The Seventh International Conference on Advanced Communications and Computation
    • MODOPT 2017, The International Symposium on Modeling and Optimization
  • MOBILITY 2017, The Seventh International Conference on Mobile Services, Resources, and Users
  • SPWID 2017, The Third International Conference on Smart Portable, Wearable, Implantable and Disability-oriented Devices and Systems
  • ACCSE 2017, The Second International Conference on Advances in Computation, Communications and Services

InfoWare 2017
July 23 - 27, 2017 - Nice, France

  • ICCGI 2017, The Twelfth International Multi-Conference on Computing in the Global Information Technology
  • ICWMC 2017, The Thirteenth International Conference on Wireless and Mobile Communications
  • VEHICULAR 2017, The Sixth International Conference on Advances in Vehicular Systems, Technologies and Applications
  • INTERNET 2017, The Ninth International Conference on Evolving Internet
  • COLLA 2017, The Seventh International Conference on Advanced Collaborative Networks, Systems and Applications
  • INTELLI 2017, The Sixth International Conference on Intelligent Systems and Applications
    • InManEnt 2017, The International Symposium on Intelligent Manufacturing Environments
  • VISUAL 2017, The Second International Conference on Applications and Systems of Visual Paradigms
  • HUSO 2017, The Third International Conference on Human and Social Analytics
  • BRAININFO 2017, The Second International Conference on Neuroscience and Cognitive Brain Information

NetWare 2017
September 10 - 14, 2017 - Rome, Italy

  • SENSORCOMM 2017, The Eleventh International Conference on Sensor Technologies and Applications
  • SENSORDEVICES 2017, The Eighth International Conference on Sensor Device Technologies and Applications
  • SECURWARE 2017, The Eleventh International Conference on Emerging Security Information, Systems and Technologies
  • AFIN 2017, The Ninth International Conference on Advances in Future Internet
  • DEPEND 2017, The Tenth International Conference on Dependability
  • CENICS 2017, The Tenth International Conference on Advances in Circuits, Electronics and Micro-electronics
  • ICQNM 2017, The Eleventh International Conference on Quantum, Nano/Bio, and Micro Technologies
  • FASSI 2017, The Third International Conference on Fundamentals and Advances in Software Systems Integration
  • GREEN 2017, The Second International Conference on Green Communications, Computing and Technologies

SoftNet 2017
October 8 - 12, 2017- Athens, Greece

  • ICSEA 2017, The Twelfth International Conference on Software Engineering Advances
  • ICSNC 2017, The Twelfth International Conference on Systems and Networks Communications
  • CENTRIC 2017, The Tenth International Conference on Advances in Human-oriented and Personalized Mechanisms, Technologies, and Services
  • VALID 2017, The Ninth International Conference on Advances in System Testing and Validation Lifecycle
  • SIMUL 2017, The Ninth International Conference on Advances in System Simulation
  • SOTICS 2017,The Seventh International Conference on Social Media Technologies, Communication, and Informatics
  • INNOV 2017, The Sixth International Conference on Communications, Computation, Networks and Technologies
  • HEALTHINFO 2017, The Second International Conference on Informatics and Assistive Technologies for Health-Care, Medical Support and Wellbeing

NexTech 2017
November 12 - 16, 2017 - Barcelona, Spain

  • UBICOMM 2017, The Eleventh International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies
  • ADVCOMP 2017, The Eleventh International Conference on Advanced Engineering Computing and Applications in Sciences
  • SEMAPRO 2017, The Eleventh International Conference on Advances in Semantic Processing
  • AMBIENT 2017, The Seventh International Conference on Ambient Computing, Applications, Services and Technologies
  • EMERGING 2017, The Ninth International Conference on Emerging Networks and Systems Intelligence
  • DATA ANALYTICS 2017, The Sixth International Conference on Data Analytics
  • GLOBAL HEALTH 2017, The Sixth International Conference on Global Health Challenges
  • CYBER 2017, The Second International Conference on Cyber-Technologies and Cyber-Systems

 


ThinkMind // CLOUD COMPUTING 2010, The First International Conference on Cloud Computing, GRIDs, and Virtualization // View article cloud_computing_2010_3_20_50031


The Limitation of MapReduce: A Probing Case and a Lightweight Solution

Authors:
Zhiqiang Ma
Lin Gu

Keywords: Distributed computing; Parallel architectures

Abstract:
MapReduce is arguably the most successful parallelization framework especially for processing large data sets in datacenters comprising commodity computers. However, difficulties are observed in porting sophisticated applications to MapReduce, albeit the existence of numerous parallelization opportunities. Intrinsically, the MapReduce design allows a program to scale up to handle extremely large data sets, but constrains a program's ability to process smaller data items and exploit variable-degrees of parallelization opportunities which are likely to be the common case in general application. In this paper, we analyze the limitations of MapReduce and present the design and implementation of a new lightweight parallelization framework, MRlite. MRlite can efficiently process moderatesize data with dependences among numerous computational steps. In the mean time, the parallelization on each step emulates the MapReduce model. Hence, the MRlite framework can also scale up for large data sets if massive parallelism with minimal dependence exists. MRlite can significantly improve the flexibility and parallel execution performance for a number of typical programs. Our evaluation shows that MRlite is one order of magnitude faster than Hadoop on problems that MapReduce has difficulty in handling.

Pages: 68 to 73

Copyright: Copyright (c) IARIA, 2010

Publication date: November 21, 2010

Published in: conference

ISSN: 2308-4294

ISBN: 978-1-61208-106-9

Location: Lisbon, Portugal

Dates: from November 21, 2010 to November 26, 2010

SERVICES CONTACT
2010 - 2015 © ThinkMind. All rights reserved.
Read Terms of Service and Privacy Policy.