hadoop streaming方式下的参数传递
In hadoop streaming, when run a map/reduce job, we may want to get some running parameter to known the statues of job. Many thess parameters, configuration and running parameters of job can be obtained from os.environ in python, i.e., the name of file input split, the job> os.environ is the dictionary to store the environment variables, Hadoopwill pass the parameter to each task of map/reduce by setting the
environment variable on each host
In map/reduce step, we can use the function of os.environ.get()
Also, you can pass the parameters of configuration explictly to your script, i.e., -mapper "python map.py -i 1"
页:
[1]