开源云计算技术系列（四）（Cloudera安装配置hadoop 0.20最新版配置）

wolong

接上文，我们继续体验Cloudera 0.20最新版。
wgethadoop-0.20-conf-pseudo_0.20.0-1cloudera0.5.0~lenny_all.deb
wgethadoop-0.20_0.20.0-1cloudera0.5.0~lenny_all.deb
debian:~# dpkg –ihadoop-0.20-conf-pseudo_0.20.0-1cloudera0.5.0~lenny_all.deb
dpkg –ihadoop-0.20_0.20.0-1cloudera0.5.0~lenny_all.deb
就这么简单。ok
如果不清楚安装到哪里了，可以用
debian:~# dpkg -L hadoop-0.20
可以看到清晰的安装目录结构。

启动：
debian:~# cd /etc/init.d/hadoop-0.20-
hadoop-0.20-datanode hadoop-0.20-namenode hadoop-0.20-tasktracker
hadoop-0.20-jobtracker hadoop-0.20-secondarynamenode
debian:~# cd /etc/init.d/hadoop-0.20-
debian:~# /etc/init.d/hadoop-0.20-namenode start
debian:~# /etc/init.d/hadoop-0.20-namenode status
hadoop-0.20-namenode is running
debian:~# /etc/init.d/hadoop-0.20-datanode start
debian:~# /etc/init.d/hadoop-0.20-datanode status
hadoop-0.20-datanode is running

debian:~# /etc/init.d/hadoop-0.20-jobtracker start
debian:~# /etc/init.d/hadoop-0.20-jobtracker status
hadoop-0.20-jobtracker is running

debian:~# /etc/init.d/hadoop-0.20-tasktracker start
debian:~# /etc/init.d/hadoop-0.20-tasktracker status
hadoop-0.20-tasktracker is running
启动完成。
接着可以进行常规的example的测试了。
值得测试的是
debian:~# sqoop –help
Usage: hadoop sqoop.jar org.apache.hadoop.sqoop.Sqoop (options)
Database connection options:
–connect (jdbc-uri) Specify JDBC connect string
–driver (class-name) Manually specify JDBC driver class to use
–username (username) Set authentication username
–password (password) Set authentication password
–local Use local imp<wbr>ort fast path (mysql on<wbr>ly)</wbr></wbr>
Imp<wbr>ort control options:<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">–table (tablename) Table to read<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">–columns (col,col,col…) Columns to export from table<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">–order-by (column-name) Column of the table used to order results<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">–hadoop-home (dir) Override $HADOOP_HOME<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">–warehouse-dir (dir) HDFS path for table destination<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">–as-sequencefile Imp<wbr>orts da<wbr>ta to SequenceFiles<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">–as-textfile Imp<wbr>orts da<wbr>ta as plain text (default)<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">–all-tables Imp<wbr>ort all tables in database<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">(Ignores –table, –columns and –order-by)</wbr></wbr></wbr></wbr></wbr></wbr>
Co<wbr>de generation options:<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">–outdir (dir) Output directory for generated co<wbr>de<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">–bindir (dir) Output directory for compiled objects<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">–generate-on<wbr>ly Stop after co<wbr>de generation; do not imp<wbr>ort</wbr></wbr></wbr></wbr></wbr>
Additional commands:
–list-tables List tables in database and exit
–list-databases List all databases available and exit
–debug-sql (statement) Execute ’statement’ in SQL and exit
Generic Hadoop command-line options:
Generic options supported are
-conf <configuration file> specify an application configuration file
-D <property=value> use value for given property
-fs <local|namenode:port> specify a namenode
-jt <local|jobtracker:port> specify a job tracker
-files <comma separated list of files> specify comma separated files to be copied to the map reduce cluster
-libjars <comma separated list of jars> specify comma separated jar files to include in the classpath.
-archives <comma separated list of archives> specify comma separated archives to be unarchived on the compute machines.
The general command line syntax is
bin/hadoop command [genericOptions] [commandOptions]
At minimum, you must specify –connect and either –table or –all-tables.
Alternatively, you can specify –generate-on<wbr>ly or on<wbr>e of the additional<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">commands.</wbr></wbr>
可以通过apt-get install mysql-server 安装debian下的mysql进行联合测试。
测试体验见前面的文章，到这里大家可以进行一个完整的体验了。Cloudera的出现的确对hadoop的配置进行了极大的简化，推动了开源云计算的发展。
转自：http://rdc.taobao.com/blog/dw/archives/417