When using HDFS on Isilon, only one user needs to be created for hadoop to access HDFS, “mapred”. This user is used by the jobtraker and used when mapredcuce jobs are submitted. Interesting thing is that if you run a
hadoop fs –ls /
command, this is done under the credentials you are logged in with. Mapred is only used when jobs are submitted. To create the user and add it to the wheel group on Isilon:
This is done from the CLI of the Isilon cluster. The other interesting thing is that when you submit a mapreduce job the first time, a /tmp directory is created in the root HDFS directory.
It is created with the mapred user as the owner and wheel as the group. I have verified that Hortonworks, Apache, and Cloudera all create this file structure properly. Cloudera needs this directory structure setup. Here is how you do it:
Comments