This page provides some miscellaneous tools based on LIBSVM. Roughly they include
Disclaimer: We do not take any responsibility on damage or other problems caused by using these software and data sets.
For some applications, data instances are strings. SVM trains a model using some string kernels. This experimental code (download zip file here) allows string inputs and implements one string kernel. Details are in README.
Feature selection tool
This is a simple python script (download here) to use F-score for selecting features. To run it, please put it in the sub-directory "tools" of LIBSVM.
Usage: ./fselect.py training_file [testing_file]
Output files: .fscore shows importance of features, .select gives the running log, and .pred gives testing results.
More information about this implementation can be found in Y.-W. Chen and C.-J. Lin, Combining SVMs with various feature selection strategies. To appear in the book "Feature extraction, foundations and applications." 2005. This implementation is still preliminary. More comments are very welcome.
Author: Yi-Wei Chen
Multi-class classification (and probability output) via error-correcting codes
Note: libsvm does support multi-class classification. The code here implements some extensions for experimental purposes.
This code implements multi-class classification and probability estimates using 4 types of error correcting codes. Details of the 4 types of ECCs and the algorithms can be found in the following paper:
T.-K. Huang, R. C. Weng, and C.-J. Lin. Generalized Bradley-Terry Models and Multi-class Probability Estimates. Journal of Machine Learning Research, 7(2006), 85-115. A (very) short version of this paper appears in NIPS 2004.
The code can be downloaded here. The installation is the same as the standard LIBSVM package, and different types of ECCs are specified as the "-i" option. Type "svm-train" without any arguments to see the usage. Note that both "one-againse-one" and "one-against-the rest" multi-class strategies are part of the implementation.
If you specify -b in training and testing, you get probability estimates and the predicted label is the one with the larget value. If you do not specify -b, this is classification based on decision values. Now we use the "exponential-loss" method in the paper:
Allwein et al.: Reducing multiclass to binary: a unifying approach for margin classifiers. Journal of Machine Learning Research, 1:113--141, 2001,
to predict class label. For one-against-the rest (or called 1vsall), this is the same as the commonly used way
argmax_{i} (decision value of ith class vs the rest).
For one-against-one, it is different from the max-win strategy used in libsvm.
MATLAB code for experiments in our paper is available here
Author: Tzu-Kuo Huang
SVM with Precomputed Kernel Matrices
Note: this feature has been included in the core LIBSVM (after version 2.82)
The input format is slightly different. Please check README for details.
Special kernels are used for some problems. Users can calculate and store the kernel matrix first. Then this code will directly use them without needing further kernel evaluations.
Please note that this is suitable for special kernels and data with very many features. For those with few features, direct kernel evaluations can be much faster than reading the kernel matrices.
Author: Pei-Chin Wang
SVM Multi-class Probability Outputs
This code implements different strategies for multi-class probability estimates from in the following paper
T.-F. Wu, C.-J. Lin, and R. C. Weng. Probability Estimates for Multi-class Classification by Pairwise Coupling. Journal of Machine Learning Research, 2004. A short version appears in NIPS 2003.
After libsvm 2.6, it already includes one of the methods here. You may directly use the standard libsvm unless you are interested in doing comparisons. Please download the tgz file here. The data used in the paper is available here. Please then check README for installation.
Matlab programs for the synthetic data experiment in the paper can be found in this directory. The main program is fig1a.m
Author: Tingfan Wu (svm [at] future.csie.org)
ROC Curve for Binary SVM
This tool which gives the ROC (Receiver Operating Characteristic) curve and AUC (Area Under Curve) by ranking the decision values. Note that we assume labels of two classes are +1 and -1. Multi-class is not supported yet.
Please download the plotroc.py file here. You need to
- Download libsvm and make the libsvm python interface
- Edit the path of gnuplot in plotroc.py in necessary
- Put plotroc.py into the python directory of libsvm package.
The usage is
plotroc.py [-t kern_type][-c Cost][-g gamma][-m cache_size][-v cv_fold][-T testing_file] training_file
If there is no test data, "validated decision values" from cross-validation on the training data are used. Otherwise, we consider decision values of testing data using the model from the training data (without cross-validation).
Author: Tingfan Wu (svm [at] future.csie.org)
Grid Parameter Search for Regression
This file is a slight modification of grid.py in the libsvm package. In addition to parameters C, gamma in classification, it searches for epsilon as well.
Usage: grid.py [-log2c begin,end,step] [-log2g begin,end,step] [-log2p begin,end,step] [-v fold]
[-svmtrain pathname] [-gnuplot pathname] [-out pathname] [-png pathname]
[additional parameters for svm-train] dataset
Author: Hsuan-Tien Lin (initial modification); Tzu-Kuo Huang (the parameter epsilon).
Radius Margin Bounds for SVM Model Selection
This is the code used in the paper: K.-M. Chung, W.-C. Kao, T. Sun, L.-L. Wang, and C.-J. Lin. Radius Margin Bounds for Support Vector Machines with the RBF Kernel. Please download the tar.bz2 file here. Details of using this code are in the readme.txt file. Part of the optimization subroutines written in Python were based on the module by Travis E. Oliphant.
Author: Wei-Chun Kao with the help from Leland Wang, Kai-Min Chung, and Tony Sun
Primal variable w of linear SVM and feature selection
In the following directory there are two files. svm-weight.cpp calculates the primal variable w using a model trained by libsvm (multi-class supported). Note that this program is for LINEAR SVM only! The output is a file containing the decison functions. If the data has k classes, the decision functions of all 1vs1 sub-problems are placed in the order 1 vs 2, ..., 1 vs k, 2 vs 3, ..., k-1 vs k.
The file linear-feasel.cpp conducts feature selection by considering indices with larger components of w. Please use the makefile in the same directory to build them. Note that this file works for two-class problems only.
Author: Tzu-Kuo Huang
Reduced Support Vector Machines Implementation
This is the code used in the paper: K.-M. Lin and C.-J. Lin. A study on reduced support vector machines. IEEE Transactions on Neural Networks, 2003.
Please download the .tgz file here. After making the binary files, type svm-train to see the usage. It includes different methods to implement RSVM.
To speed up the code, you may want to link the code to optimized BLAS/LAPACK or ATLAS.
Author: Kuan-Min Lin
DAG approach for multiclass classification
In svm.cpp, please replace the lines from
double *dec_values = Malloc(double, nr_class*(nr_class-1)/2);
to
return model->label[vote_max_idx];
in the subtoutine svm_predict() with this segment of code.
This follows from the code used in the paper: C.-W. Hsu and C.-J. Lin. A comparison of methods for multi-class support vector machines , IEEE Transactions on Neural Networks
, 13(2002), 415-425.
LIBSVM介绍及使用方法
