Running Mahout's Logistic Regression

Logistic Regression(SGD) is one the algorithms available in Mahout. This blog post lists and provides all that is required for the same. To install Mahout on your machine, you can refer to my previous post.

Logistic Regression executes in two major phases:

• Train the model: This step is about creating a model using some train data, that can further be used for the classification of any input data rather I would say test data.

• Test the model: This step tests the generated model in step 1 by evaluating the results of classification of test data, and measuring the accuracy, scores and confusion matrix.

Steps for running Mahout’s LR

Step-I: Get the input data file called donut.csv, which is present in the mahout setup. But for your ready reference I have also shared it. You can download it from here.

Step-II: Next cd to the MAHOUT_HOME. Here we would be running the “org.apache.mahout.classifier.sgd.TrainLogistic” class that would train the model for us using the “donut.csv” file what we would be providing as train data. Here’s the command to be run from within MAHOUT_HOME:

bin/mahout org.apache.mahout.classifier.sgd.TrainLogistic --passes 1 --rate 1 --lambda 0.5 --input loc_of_file/donut.csv --features 21 --output any_loc_on_your_machine/donut.model --target color --categories 2 --predictors x y xx xy yy a b c --types n n

If the Mahout version is 0.7 you are likely to face the error below:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/util/ProgramDriver     at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:96) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.util.ProgramDriver     at java.net.URLClassLoader$1.run(URLClassLoader.java:366)     at java.net.URLClassLoader$1.run(URLClassLoader.java:355)     at java.security.AccessController.doPrivileged(Native Method)     at java.net.URLClassLoader.findClass(URLClassLoader.java:354)     at java.lang.ClassLoader.loadClass(ClassLoader.java:423)     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)     at java.lang.ClassLoader.loadClass(ClassLoader.java:356)     ... 1 more

Don’t worry, all you need to do is:

export CLASSPATH=${CLASSPATH}:your_MAHOUT_HOME/mahout-distribution-0.7/lib/hadoop/hadoop-core-0.20.204.0.jar

After editing the CLASSPATH as mentioned above the command should run successfully and print something like:

color ~ -0.016*Intercept Term + -0.016*xy + -0.016*yy       Intercept Term -0.01559                   xy -0.01559                   yy -0.01559     0.000000000     0.000000000     0.000000000     0.000000000     0.000000000     0.000000000     0.000000000     0.000000000     0.000000000     0.000000000    -0.015590929     0.000000000     0.000000000     0.000000000     0.000000000     0.000000000     0.000000000     0.000000000     0.000000000     0.000000000     0.000000000  13/05/26 02:14:02 INFO driver.MahoutDriver: Program took 588 ms (Minutes: 0.0098)

The most important parameters influencing the execution of the Training process are:

"--passes": the number of times to pass over the input data
"--lambda": the amount of coefficient decay to use
"--rate": the learning rate

You can vary the values of these 3 variables and see the change in performance of the algorithm. Also, you can now see that the model would have been created at the location which you had specified in the command.

Step-III: Now it’s time to run the classifier using the model that has been trained in Step-II. As the test data we would be using the same donut.csv file that we used for training or you can even split the file in some ratio for eg. 70-30 and use the 70% file for training of the model and 30% file for testing. Here’s the command for testing the model and running the classifier:

bin/mahout org.apache.mahout.classifier.sgd.RunLogistic --input loc_of_file/donut.csv  --model loc_of_model/donut.model --auc --scores --confusion

which should print an output something like:

"target","model-output","log-likelihood" 0,0.496,-0.685284 0,0.490,-0.674055 0,0.491,-0.675162 1,0.495,-0.703361 1,0.493,-0.706289 0,0.495,-0.683275 0,0.496,-0.685282 0,0.492,-0.677191 1,0.494,-0.704222 0,0.495,-0.684107 0,0.496,-0.684765 1,0.494,-0.705209 0,0.491,-0.675272 1,0.495,-0.703438 0,0.496,-0.685121 0,0.496,-0.684886 0,0.490,-0.672500 0,0.495,-0.682445 0,0.496,-0.684872 1,0.495,-0.703070 0,0.490,-0.672511 0,0.495,-0.683643 0,0.492,-0.677610 1,0.492,-0.708915 0,0.496,-0.684744 1,0.494,-0.704766 0,0.492,-0.677496 1,0.492,-0.708679 0,0.496,-0.685222 1,0.495,-0.703604 0,0.492,-0.677846 0,0.490,-0.672702 0,0.492,-0.676980 0,0.494,-0.681450 1,0.495,-0.702845 0,0.493,-0.679049 0,0.496,-0.684262 1,0.493,-0.706564 1,0.495,-0.704016 0,0.490,-0.672624 AUC = 0.52 confusion: [[27.0, 13.0], [0.0, 0.0]] entropy: [[-0.7, -0.4], [-0.7, -0.5]] 13/05/26 02:16:19 INFO driver.MahoutDriver: Program took 474 ms (Minutes: 0.0079)

Similarly, you can try on a variety of data sets that you might have. I have seen upto 93% accuracy of results of classification on a different data set.

All the best !!!