安装hadoop下的sqoop1.99.3及配置问题全解决

outlook

　　2016年8月27日，解决sqoop
　　先下载的是sqoop1.99.7，结果配置的时候，出现了没有server/conf目录，以及启动时出现无法配置错误
　　./bin/sqoop.sh server start
　　yq@ubuntu:/opt/sqoop-1.99.7$ ./bin/sqoop.sh server start
　　Setting conf dir: ./bin/../conf
　　Sqoop home directory: /opt/sqoop-1.99.7
　　Starting the Sqoop2 server...
　　Sqoop2 server started.
　　dyq@ubuntu:/opt/sqoop-1.99.7$ 0    [main] INFO  org.apache.sqoop.core.SqoopServer  - Initializing Sqoop server.
　　58   [main] INFO  org.apache.sqoop.core.PropertiesConfigurationProvider  - Starting config file poller thread
　　Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/configuration/Configuration
　　at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.(DefaultMetricsSystem.java:38)
　　at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.(DefaultMetricsSystem.java:36)
　　at org.apache.hadoop.security.UserGroupInformation$UgiMetrics.create(UserGroupInformation.java:120)
　　at org.apache.hadoop.security.UserGroupInformation.(UserGroupInformation.java:236)
　　at org.apache.sqoop.security.authentication.SimpleAuthenticationHandler.secureLogin(SimpleAuthenticationHandler.java:39)
　　at org.apache.sqoop.security.AuthenticationManager.initialize(AuthenticationManager.java:98)
　　at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:57)
　　at org.apache.sqoop.server.SqoopJettyServer.(SqoopJettyServer.java:67)
　　at org.apache.sqoop.server.SqoopJettyServer.main(SqoopJettyServer.java:177)
　　Caused by: java.lang.ClassNotFoundException: org.apache.commons.configuration.Configuration
　　at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
　　at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
　　at java.security.AccessController.doPrivileged(Native Method)
　　at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
　　at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
　　at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
　　at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
　　... 9 more
　　解决办法是下载sqoop1.99.3，然后哪找如下配置，搞掂：

• 　　2.解压文件到工作目录:
  
• 　　hadoop@hadoopMaster:$ sudo tar -xvf /opt/hn/hadoop_family/sqoop-1.99.3-bin-hadoop200.tar.gz
  
• 　　hadoop@hadoopMaster:mv /opt/hn/hadoop_family/sqoop-1.99.3-bin-hadoop200 /usr/local/sqoop
  
• 　　3.修改环境变量：
  
• 　　hadoop@hadoopMaster:~$ vim /etc/profile
  
• 　　添加如下内容：
  
• 　　#sqoop
  
• 　　export SQOOP_HOME=/usr/sqoop-1.99.3
  
• 　　export PATH=$SQOOP_HOME/bin:$PATH
  
• 　　export CATALINA_HOME=$SQOOP_HOME/server
  
• 　　export LOGDIR=$SQOOP_HOME/logs
  
• 　　保存退出即时生效：
  
• 　　source /etc/profile
  
• 　　4.修改sqoop配置：
  
• 　　hadoop@hadoopMaster:~$ vim /usr/local/sqoop/server/conf/sqoop.properties
  
• 　　#修改指向我的hadoop的配置文件目录
  
• 　　org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/usr/local/hadoop-2.6.2/etc/hadoop
  
• 　　#把hadoop目录下的jar包都引进来
  
• 　　hadoop@hadoopMaster:~$ vim /usr/local/sqoop/server/conf/catalina.properties
  
• 　　common.loader=/usr/local/hadoop/share/hadoop/common/*.jar,/usr/local/hadoop/share/hadoop/common/lib/*.jar,/usr/local/hadoop/share/hadoop/hdfs/*.jar,/usr/local/hadoop/share/hadoop/hdfs/lib/*.jar,/usr/local/hadoop/share/hadoop/mapreduce/*.jar,/usr/local/hadoop/share/hadoop/mapreduce/lib/*.jar,/usr/local/hadoop/share/hadoop/tools/*.jar,/usr/local/hadoop/share/hadoop/tools/lib/*.jar,/usr/local/hadoop/share/hadoop/yarn/*.jar,/usr/local/hadoop/share/hadoop/yarn/lib/*.jar,/usr/local/hadoop/share/hadoop/httpfs/tomcat/lib/*.jar
  
• 　　5.下载mysql驱动包
  
• 　　mysql-connector-java-5.1.16-bin.jar
  
• 　　6.启动/停止sqoop200
  
• 　　hadoop@hadoopMaster:/usr/local/sqoop/bin$ ./sqoop.sh server start/stop
  

　　跑起来试试：
　　dyq@ubuntu:/opt/sqoop-1.99.3$  ./bin/sqoop.sh server start
　　Sqoop home directory: /opt/sqoop-1.99.3
　　Setting SQOOP_HTTP_PORT:     12000
　　Setting SQOOP_ADMIN_PORT:     12001
　　Using   CATALINA_OPTS:
　　Adding to CATALINA_OPTS:    -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001
　　Using CATALINA_BASE:   /opt/sqoop-1.99.3/server
　　Using CATALINA_HOME:   /opt/sqoop-1.99.3/server
　　Using CATALINA_TMPDIR: /opt/sqoop-1.99.3/server/temp
　　Using JRE_HOME:        /usr/java/jdk1.7.0_79/jre

　　Using>