While writing this post, I am assuming that you have
- Installed oozie on your linux machine
- Installed hadoop-0.20.1+
If not, I have already mentioned the Oozie Installation in my previous post.
Steps to get an Oozie app running
Having started hadoop and the oozie server, follow the steps below to get an sample oozie application running :
- In case Oozie installation has been done using debian packag, you can find the oozie examples tar.gz at /etc/oozie/doc/oozie else it can be located in the oozie setup folder.
- Extract this and the obtained /examples folder would contain apps, input-data and src sub-directories.
- Add the following properties to the conf/core-site.xml of your hadoop setup
| <property> <name>hadoop.proxyuser.oozie.hosts</name> <value>*</value> </property> <property> <name>hadoop.proxyuser.oozie.groups</name> <value>*</value> </property> |
- In order to run any of the apps, remember to edit the port nos. of jobtracker and the namenode in the job.properties file of the app depending upon your hadoop configuration
JobTracker port no. is set in: /conf/mapred-site.xml
and NameNode port no. is set in : /conf/core-site.xml
Accordingly replace the 'JTPortNo' and 'NNPortNo' in job.properties as below :
| oozie.wf.application.path=hdfs://localhost:NNPortNo/path_to_examples/apps/map-reduce jobTracker=localhost:JTPortNo nameNode=hdfs://localhost:NNPortNo queueName=default outputDir=map-reduce |
- Now its time to copy the examples dir to hdfs, but if there is already an examples dir in hdfs you must delete it else the files are not copied. Here's the command :
| /path_to_hadoopdir/bin/hadoop fs -put /path_to_egdir/examples examples |
For a confirmation, you can check if the copy has been successful at
http://localhost:50070
- Run the following command to get the example running
In case of debian package used for installation of Oozie
| /usr/lib/oozie/bin/oozie job -oozie http://localhost:11000/oozie -config /path_to_egdir/examples/apps/map-reduce/job.properties -run |
else
/path_to_oozie/bin/oozie job -oozie http://localhost:11000/oozie -config /path_to_egdir/examples/apps/map-reduce/job.properties -run |
Here an important note is that you need to specify the local system path to job.properties and not that of hdfs in the command.
If the application has started off successfully, a job id would be returned in response to the above command, something like this :
job: 14-20090525161321-oozie-tucu
If you have the web console installed, you can view the status of the job on
http://localhost:11000/oozie
else the following command will do
/path_to_oozie/bin/oozie job -oozie http://localhost:11000/oozie -info 14-20090525161321-oozie-tucu |
That's it … You can apply the same steps for running any of the documented examples. Well, if the things have not worked as smoothly as they seem, my next post on 'Errors while installation and running Oozie' could be an answer.
