A FEAture Selection Toolbox for C/C+, Java, and Matlab/Octave.
BSD-3-CLAUSE License
A FEAture Selection Toolbox for C/C++ & MATLAB/OCTAVE, v2.0.0.
FEAST provides implementations of common mutual information based filter feature selection algorithms, and an implementation of RELIEF. All functions expect discrete inputs (except RELIEF, which does not depend on the MIToolbox), and they return the selected feature indices. These implementations were developed to help our research into the similarities between these algorithms, and our results are presented in the following paper:
Conditional Likelihood Maximisation: A Unifying Framework for Information Theoretic Feature Selection
G. Brown, A. Pocock, M.-J. Zhao, M. Lujan
Journal of Machine Learning Research, 13:27-66 (2012)
The weighted feature selection algorithms are described in Chapter 7 of:
Feature Selection via Joint Likelihood
A. Pocock
PhD Thesis, University of Manchester, 2012
If you use these implementations for academic research please cite the relevant paper above. All FEAST code is licensed under the BSD 3-Clause License.
Contains implementations of: mim, mrmr, mifs, cmim, jmi, disr, cife, icap, condred, cmi, relief, fcbf, betagamma
And weighted implementations of: mim, cmim, jmi, disr, cmi
References for these algorithms are provided in the accompanying feast.bib file (in BibTeX format).
FEAST works on discrete inputs, and all continuous values must be discretised before use with FEAST. In our experiments we've found that using 10 equal width bins is suitable for many problems, though this is data set size dependent. FEAST produces unreliable results when used with continuous inputs, runs slowly and uses much more memory than usual. The discrete inputs should have small cardinality, FEAST will treat values {1,10,100} the same way it treats {1,2,3} and the latter will be both faster and use less memory.
MATLAB Example (using "data" as our feature matrix, and "labels" as the class label vector):
>> size(data)
ans =
(569,30) %% denoting 569 examples, and 30 features
>> selectedIndices = feast('jmi',5,data,labels) %% selecting the top 5 features using the jmi algorithm
selectedIndices =
28
21
8
27
23
>> selectedIndices = feast('mrmr',10,data,labels) %% selecting the top 10 features using the mrmr algorithm
selectedIndices =
28
24
22
8
27
21
29
4
7
25
>> selectedIndices = feast('mifs',5,data,labels,0.7) %% selecting the top 5 features using the mifs algorithm with beta = 0.7
selectedIndices =
28
24
22
20
29
The library is written in ANSI C for compatibility with the MATLAB mex compiler, except for MIM, FCBF and RELIEF, which are written in MATLAB/OCTAVE script. There is a different implementation of MIM available for use in the C library.
MIToolbox v3.0.0 is required to compile these algorithms, and these implementations supercede the example implementations given in that package (they have more robust behaviour when used with unexpected inputs).
MIToolbox can be found at: http://www.github.com/Craigacp/MIToolbox/
The C library expects all matrices in column-major format (i.e. Fortran style). This is for two reasons, a) MATLAB generates Fortran-style arrays, and b) feature selection iterates over columns rather than rows, unlike most other ML processes.
Compilation instructions:
CompileFEAST.m
in the matlab
folder.make x86
or make x64
for a 32-bit or 64-bit library.make x64_win
.make x64
, sudo make install
to build and install the C library.make java
to build the JNI wrapper.mvn package
in the java
directory to build the jar file.Update History