ComputationWorld 2017
February 19 - 23, 2017 - Athens, Greece

  • SERVICE COMPUTATION 2017, The Ninth International Conferences on Advanced Service Computing
  • CLOUD COMPUTING 2017, The Eighth International Conference on Cloud Computing, GRIDs, and Virtualization
  • FUTURE COMPUTING 2017, The Ninth International Conference on Future Computational Technologies and Applications
  • COGNITIVE 2017, The Ninth International Conference on Advanced Cognitive Technologies and Applications
  • ADAPTIVE 2017, The Ninth International Conference on Adaptive and Self-Adaptive Systems and Applications
  • CONTENT 2017, The Ninth International Conference on Creative Content Technologies
  • PATTERNS 2017, The Ninth International Conferences on Pervasive Patterns and Applications
  • COMPUTATION TOOLS 2017, The Eighth International Conference on Computational Logics, Algebras, Programming, Tools, and Benchmarking
  • BUSTECH 2017, The Seventh International Conference on Business Intelligence and Technology

DigitalWorld 2017
March 19 - 23, 2017 - Nice, France

  • ICDS 2017, The Eleventh International Conference on Digital Society and eGovernments
  • ACHI 2017, The Tenth International Conference on Advances in Computer-Human Interactions
  • GEOProcessing 2017, The Ninth International Conference on Advanced Geographic Information Systems, Applications, and Services
  • eTELEMED 2017, The Ninth International Conference on eHealth, Telemedicine, and Social Medicine
  • DIGITAL HEALTHY LIVING 2017, A Multidisciplinary View on Digital Support for Healthy Living and Self-management for Health
  • MATH 2017, The International Symposium on Mobile and Assistive Technology for Healthcare
  • eLmL 2017, The Ninth International Conference on Mobile, Hybrid, and On-line Learning
  • eKNOW 2017, The Ninth International Conference on Information, Process, and Knowledge Management
  • ALLSENSORS 2017, The Second International Conference on Advances in Sensors, Actuators, Metering and Sensing

NexComm 2017
April 23 - 27, 2017 - Venice, Italy

  • ICDT 2017, The Twelfth International Conference on Digital Telecommunications
  • SPACOMM 2017, The Ninth International Conference on Advances in Satellite and Space Communications
  • ICN 2017, The Sixteenth International Conference on Networks
  • SOFTNETWORKING 2017, The International Symposium on Advances in Software Defined Networking and Network Functions Virtualization
  • ICONS 2017, The Twelfth International Conference on Systems
  • MMEDIA 2017, The Ninth International Conferences on Advances in Multimedia
  • PESARO 2017, The Seventh International Conference on Performance, Safety and Robustness in Complex Systems and Applications
  • CTRQ 2017, The Tenth International Conference on Communication Theory, Reliability, and Quality of Service
  • COCORA 2017, The Seventh International Conference on Advances in Cognitive Radio
  • ALLDATA 2017, The Third International Conference on Big Data, Small Data, Linked Data and Open Data
  • KESA 2017, The International Workshop on Knowledge Extraction and Semantic Annotation
  • SOFTENG 2017, The Third International Conference on Advances and Trends in Software Engineering

 


ThinkMind // CLOUD COMPUTING 2010, The First International Conference on Cloud Computing, GRIDs, and Virtualization // View article cloud_computing_2010_3_20_50031


The Limitation of MapReduce: A Probing Case and a Lightweight Solution

Authors:
Zhiqiang Ma
Lin Gu

Keywords: Distributed computing; Parallel architectures

Abstract:
MapReduce is arguably the most successful parallelization framework especially for processing large data sets in datacenters comprising commodity computers. However, difficulties are observed in porting sophisticated applications to MapReduce, albeit the existence of numerous parallelization opportunities. Intrinsically, the MapReduce design allows a program to scale up to handle extremely large data sets, but constrains a program's ability to process smaller data items and exploit variable-degrees of parallelization opportunities which are likely to be the common case in general application. In this paper, we analyze the limitations of MapReduce and present the design and implementation of a new lightweight parallelization framework, MRlite. MRlite can efficiently process moderatesize data with dependences among numerous computational steps. In the mean time, the parallelization on each step emulates the MapReduce model. Hence, the MRlite framework can also scale up for large data sets if massive parallelism with minimal dependence exists. MRlite can significantly improve the flexibility and parallel execution performance for a number of typical programs. Our evaluation shows that MRlite is one order of magnitude faster than Hadoop on problems that MapReduce has difficulty in handling.

Pages: 68 to 73

Copyright: Copyright (c) IARIA, 2010

Publication date: November 21, 2010

Published in: conference

ISSN: 2308-4294

ISBN: 978-1-61208-106-9

Location: Lisbon, Portugal

Dates: from November 21, 2010 to November 26, 2010

SERVICES CONTACT
2010 - 2015 © ThinkMind. All rights reserved.
Read Terms of Service and Privacy Policy.