DigitalWorld 2015
February 22 - 27, 2015 - Lisbon, Portugal

ComputationWorld 2015
March 22 - 27, 2015 - Nice, France

NexComm 2015
April 19 - 24, 2015 - Barcelona, Spain

InfoSys 2015
May 24 - 29, 2015 - Rome, Italy

BioSciencesWorld 2015
May 24 - 29, 2015 - Rome, Italy

DataSys 2015
June 21 - 26, 2015 - Brussels, Belgium

NexTech 2015
July 19 - 24, 2015 - Nice, France

NetWare 2015
August 23 - 28, 2015 - Venice, Italy

InfoWare 2015
October 11 - 16, 2015 - St. Julians, Malta

SoftNet 2015
November 15 - 20, 2015 - Barcelona, Spain

 

ThinkMind // CLOUD COMPUTING 2010, The First International Conference on Cloud Computing, GRIDs, and Virtualization // View article cloud_computing_2010_3_20_50031


The Limitation of MapReduce: A Probing Case and a Lightweight Solution

Authors:
Zhiqiang Ma
Lin Gu

Keywords: Distributed computing; Parallel architectures

Abstract:
MapReduce is arguably the most successful parallelization framework especially for processing large data sets in datacenters comprising commodity computers. However, difficulties are observed in porting sophisticated applications to MapReduce, albeit the existence of numerous parallelization opportunities. Intrinsically, the MapReduce design allows a program to scale up to handle extremely large data sets, but constrains a program's ability to process smaller data items and exploit variable-degrees of parallelization opportunities which are likely to be the common case in general application. In this paper, we analyze the limitations of MapReduce and present the design and implementation of a new lightweight parallelization framework, MRlite. MRlite can efficiently process moderatesize data with dependences among numerous computational steps. In the mean time, the parallelization on each step emulates the MapReduce model. Hence, the MRlite framework can also scale up for large data sets if massive parallelism with minimal dependence exists. MRlite can significantly improve the flexibility and parallel execution performance for a number of typical programs. Our evaluation shows that MRlite is one order of magnitude faster than Hadoop on problems that MapReduce has difficulty in handling.

Pages: 68 to 73

Copyright: Copyright (c) IARIA, 2010

Publication date: November 21, 2010

Published in: conference

ISSN: 2308-4294

ISBN: 978-1-61208-106-9

Location: Lisbon, Portugal

Dates: from November 21, 2010 to November 26, 2010

SERVICES CONTACT
2010 - 2014 © ThinkMind. All rights reserved.
Read Terms of Service and Privacy Policy.