hadoop mapred(hive)执行目录 文件权限问题

gtdfe

在配置其他linux用户使用hive 的cli客户端时，发现该用户没有对/tmp目录的写权限，于是将/tmp目录下权限设置为777：bin  /hadoop fs -chmod -R 777 /tmp，然而第二天来时发现hadoop的文件rush任务失败，报错的日志信息如下：Job Submission failed with exception 'java.io.IOException(The ownership/permissions on the staging directory /tmp/hadoop-hadoop-user1/mapred/staging/hadoop-user1/.staging is not as expected. It is owned by hadoop-user1 and permissions are rwxrwxrwx. The directory must be owned by the submitter hadoop-user1 or by hadoop-user1 and permissions must be rwx------)
   原来hadoop mapred作业的运行用户在/tmp目录下创建了自己的作业空间：/tmp/hadoop-rsync，且该作业空间的权限必须为700，所以赶紧更改该目录权限： bin/hadoop fs -chmod -R 700 /tmp/hadoop-hadoop-user1 ，权限更改之后mapred作业运行正常。
   在修改之前，我将/tmp的目录修改为644，结果其他用户在使用hive是报错：
FAILED: Hive Internal Error: java.lang.RuntimeException(org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security.AccessControlException: Permission denied: user=hadoop-user2, access=EXECUTE, inode="tmp":hadoop-user1:supergroup:rw-rw-rw-)
java.lang.RuntimeException: org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security.AccessControlException: Permission denied: user=hadoop-user2, access=EXECUTE, inode="tmp":hadoop-user1:supergroup:rw-rw-rw-
        at org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:151)
        at org.apache.hadoop.hive.ql.Context.getMRScratchDir(Context.java:190)
        at org.apache.hadoop.hive.ql.Context.getMRTmpFileURI(Context.java:247)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:900)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:6594)
        at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security.AccessControlException: Permission denied: user=hadoop-user2, access=EXECUTE, inode="tmp":hadoop-user1:supergroup:rw-rw-rw-
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
        at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:57)
        at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:994)
        at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:329)
        at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1050)
        at org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:147)
        ... 15 more
Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.security.AccessControlException: Permission denied: user=hadoop-user2, access=EXECUTE, inode="tmp":hadoop-user1:supergroup:rw-rw-rw-
        at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:199)
        at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkTraverse(FSPermissionChecker.java:155)
        at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:125)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:4811)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkTraverse(FSNamesystem.java:4790)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:1904)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:1886)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.mkdirs(NameNode.java:716)
        at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:523)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1383)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1379)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1377)

        at org.apache.hadoop.ipc.Client.call(Client.java:1030)
        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
        at $Proxy5.mkdirs(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
        at $Proxy5.mkdirs(Unknown Source)
        at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:992)
        ... 18 more
   在将/tmp 和/tmp/hive-wangqiang的全新修改为777后：
                          bin/hadoop fs -chmod 777 /tmp/hive-hadoop-user2
                          bin/hadoop fs -chmod 777 /tmp/
   错误信息为：Failed with exception java.io.IOException:org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security.AccessControlException: Permission denied: user=hadoop-user2, access=EXECUTE, inode="dw":hadoop-user1:supergroup:rw-r--r--
    最后将hive external表数据目录权限改为777后，查询才恢复正常。
    但是该用户在执行需要mapred的hql时，报错信息如下：
org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security.AccessControlException: Permission denied: user=hadoop-user2, access=EXECUTE, inode="hadoop-hadoop-user1":hadoop-user1:supergroup:rwx------
   上述日志信息的意思是mapred作业在/tmp/目录下的文件夹hadoop-rsync权限拒绝用户wangqiang的执行访问。但是将该文件下面的权限更改到其他权限组合是，以rsync用户提交的mapred作业报错：permissions must be rwx------
    在仔细查看了/tmp/hadoop-rsync文件夹下面的目录结构后，发现了问题的关键所在：
[hadoop-user1@oser-624 hadoop-0.20.203.0]$ bin/hadoop fs -ls /tmp/hadoop-hadoop-user1/mapred/staging
Found 2 items
drwx------   - hadoop-user1     supergroup          0 2011-10-19 18:18 /tmp/hadoop-hadoop-user1/mapred/staging/hadoop-user1
drwx------   - hadoop-user2 supergroup          0 2011-10-27 18:38 /tmp/hadoop-hadoop-user1/mapred/staging/hadoop-user2
   原来不同用户提交的作业是在 /tmp/hadoop-hadoop-user1/mapred/staging/目录下以用户名区分，而之前的修改是直接使用-R选项直接修改/tmp/hadoop-rsync目录下的所有权限导致的错误，执行以下权限修改命令：
[hadoop-user1@oser-624 hadoop-0.20.203.0]$ bin/hadoop fs -chmod 777  /tmp/hadoop-hadoop-user1/mapred/
[hadoop-user1@oser-624 hadoop-0.20.203.0]$ bin/hadoop fs -chmod 777  /tmp/hadoop-hadoop-user1/mapred/staging
[hadoop-user1@oser-624 hadoop-0.20.203.0]$ bin/hadoop fs -chmod 777  /tmp/hadoop-hadoop-user1/
[hadoop-user1@oser-624 hadoop-0.20.203.0]$ bin/hadoop fs -chmod 777  /tmp
  hive查询正常。
    最近又遇上了这个问题，但是按照上面的方法修改后没能解决，于是查看namenode的日志：
2011-11-29 15:57:09,921 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 9000, call mkdirs(/opt/data/hive-zhaoxiuxiang/hive_2011-11-29_15-57-08_094_4199830510252920639, rwxr-xr-x) from 192.168.1.187:18457: error: org.apache.hadoop.security.AccessControlException: Permission denied: user=zhaoxiuxiang, access=WRITE, inode="data":rsync:supergroup:rwxr-xr-x
    发现原来出现权限错误的是目录：/opt/data ，将该目录权限修改为777后错误解决。



附注：不同用户的mapred执行目录必须是700（rwx------）权限