Hadoop

ms133

Hadoop 官方网站

 

Hadoop - Cloudera

 

Hadoop - Yahoo!

 

Hadoop - Wiki

Doug Cutting - Wiki

Doug Cutting - blog

 

Hadoop 包括下面这些子项目：

• HDFS:A distributed file system that provides high throughput access toapplication data. HDFS: 一个能够提供高吞吐量访问应用数据的分布式文件系统。其思想来自于 Google 的 The Google File System (GFS)
  
• MapReduce:A software framework for distributed processing of large data sets oncompute clusters. MapReduce: 在。其思想来自于 Google 的 MapReduce: Simplified Data Processing on Large Clusters
  
  

 

《Hadoop权威指南(中文版)》

      本人已买且读过部分章节。翻译的语句明显不通，但是该刚接触 Hadoop 挚友的还是很有帮助的。从中文版的内容来看，英文原版的质量非常不错。所以，建议将她和英文版（下载电子版即可，下载地址详见下面，附件也有文件下载），以及 Hadoop 官方文档信息一起结合起来学习和实践。这应该是一种不错的折衷方案吧，毕竟有关 Hadoop 的经典中文书籍少之又少。

《Hadoop: The Definitive Guide》

    从中文版的内容介绍来看，她对 Hadoop 的 HDFS 和 MapReduce 的具体实现细节都介绍地很详细。个人认为她与《Java 编程思想》有的一拼。英文原版下载地址：Oreilly.Hadoop.The.Definitive.Guide.Jun.2009.rar

《云计算的关键技术与应用实例》

     有选择的看了这本书的部分章节，发现她对云计算（包括概念、相关技术）的解释还是颇有深度，且是用通俗易懂的语言阐明非常深奥的知识实属难得。同时也看出作者对云计算的理解还是很有深度的。

 

　　The Google File System
Sanjay Ghemawat,Howard Gobioff, andShun-Tak Leung
　　Abstract
　　Wehave designed and implemented the Google File System, a scalabledistributed file system for large distributed data-intensiveapplications. It provides fault tolerance while running on inexpensivecommodity hardware, and it delivers high aggregate performance to alarge number of clients.
　　

　　Whilesharing many of the same goals as previous distributed file systems,our design has been driven by observations of our application workloadsand technological environment, both current and anticipated, thatreflect a marked departure from some earlier file system assumptions.This has led us to reexamine traditional choices and explore radicallydifferent design points.
　　

　　Thefile system has successfully met our storage needs. It is widelydeployed within Google as the storage platform for the generation andprocessing of data used by our service as well as research anddevelopment efforts that require large data sets. The largest clusterto date provides hundreds of terabytes of storage across thousands ofdisks on over a thousand machines, and it is concurrently accessed byhundreds of clients.
　　

　　In thispaper, we present file system interface extensions designed to supportdistributed applications, discuss many aspects of our design, andreport measurements from both micro-benchmarks and real world use.
　　

　　Appeared in:
19th ACM Symposium on Operating Systems Principles,
Lake George, NY, October, 2003.
　　

　　Download: PDF Version
　　

　　MapReduce: Simplified Data Processing on Large Clusters
Jeffrey Deanand Sanjay Ghemawat
　　Abstract
　　MapReduceis a programming model and an associated implementation for processingand generating large data sets. Users specify a map function thatprocesses a key/value pair to generate a set of intermediate key/valuepairs, and a reduce function that merges all intermediate valuesassociated with the same intermediate key. Many real world tasks areexpressible in this model, as shown in the paper.
　　

　　Programswritten in this functional style are automatically parallelized andexecuted on a large cluster of commodity machines. The run-time systemtakes care of the details of partitioning the input data, schedulingthe program's execution across a set of machines, handling machinefailures, and managing the required inter-machine communication. Thisallows programmers without any experience with parallel and distributedsystems to easily utilize the resources of a large distributed system.
　　

　　Ourimplementation of MapReduce runs on a large cluster of commoditymachines and is highly scalable: a typical MapReduce computationprocesses many terabytes of data on thousands of machines. Programmersfind the system easy to use: hundreds of MapReduce programs have beenimplemented and upwards of one thousand MapReduce jobs are executed onGoogle's clusters every day.
　　

　　Appeared in:
OSDI'04: Sixth Symposium on Operating System Design and Implementation,
San Francisco, CA, December, 2004.
　　

　　Download: PDF Version
　　Slides: HTML Slides
 
想要学习 Google 技术的挚友，不妨时常访问她：Google Research 技术论文中心