Search My Blog

Monday, March 15, 2010

Nick's Machine Perception Toolbox

These Face Recognition Apps look very interesting. I'm gonna try them out later...

Nick's Machine Perception Toolbox

0.2

Introduction

The NMPT package consists of an API and a set of libraries for Machine Perception that were developed by Nicholas Butko. It can be obtained via the Download NMPT section. Directions for compiling the software (including platform-specific directions for installing OpenCV) are in the Installation section. The central philosophy of this package is three-fold:

Easy to Install
Easy to Learn API
Easy to Run

The code is meant to stand alone under any standard C++ compiler, with the exception that it requires the OpenCV libraries. The only platform-specific (or user specific) portion of the compilation should be correctly locating the OpenCV libraries on your system. A few standard configurations are supported, but porting the code to a new system is as simple as modifying the Makefile to reflect the location of the OpenCV header files (includes) and libraries.

The core of the library is an API for Machine Perception Primitives. There is also a separate API for the Auxilliary Tools used internally, which may also be useful to others.

The following Machine Perception Primitives are currently implemented in this library / API:

FastSaliency: An implementation of the "Fast Saliency Using Natural-statistics" algorithm from Butko et al., 2008. If this code is used in your research, please cite both the paper and this NMPT package. FastSUN is an efficient implementation of Zhang et al.'s SUN algorithm (see Related Publications).
MIPOMDP: An implementation of the "Multinomial Infomax-POMDP" algorithm from Butko and Movellan, 2009. If this code is used in your research, please cite both the paper and this NMPT package. MIPOMDP is an extension of the IPOMDP Infomax Model of Eye-movment in Butko and Movellan, 2008; Najemnik and Geisler, 2005 (see Related Publications).

The Example Programs are meant to be both illustrative of code usage, and valuable stand-alone tools for research for those who don't wish to code their own programs using the API. The following examples are included:

Examples of FastSaliency:

SimpleSaliencyExample - The simplest program that illustrates the use of the FastSaliency class. One of the included movies is loaded into memory and processed frame-by-frame for saliency. The result is displayed. To run:

>>bin/SimpleSaliencyExample

FastSUN - A more advanced program that illustrates the range of parameters that can control the FastSaliency class. This program can take input from any video that OpenCV can read, or from an attached camera. It displays the input/output of the saliency program, and timing information. It has a user-interface where all of the parameters to the saliency algorithm can be modified manually. To run:

>>bin/FastSUN [optional-path-to-movie-file]

FastSUNImage - A similar example to the one above, this program analyzes a static image for saliency. To run:

>>bin/FastSUNImage [required-path-to-image-file]

Examples of MIPOMDP:

SimpleFaceTracker An example the simplest program using of the MIPOMDP class. Loads a video, and produces one fixation per video frame, tracking the face across video frames. To run:

>> bin/SimpleFaceTracker

FoveatedFaceTrackerImage A more complete example of the MIPOMDP algorithm,: takes any image OpenCV reads as input, and animates visual search. To run:

>> bin/FoveatedFaceTracker [optional-path-to-movie-file]

FoveatedFaceTracker A more complete example of the MIPOMDP algorithm, displaying visualizations of the algorithm internals, and taking multiple input sources (camera, video). To run:

>> bin/FoveatedFaceTrackerImage [required-path-to-image-file]

CVPRTestSpeed - Reproduce the speed results from Butko and Movellan, CVPR 09 on your own machine. To run:

(1) Uncompress and Expand the included GENKI R2009a dataset. Make sure the GENKI-R2009a folder is in the data directory:

         >> tar -xzvf data/GENKI-R2009a.tgz -C data/

(2) Run the program.
>> bin/CVPRTestSpeed

CVPRTrainModels - Reproduce the Multinomial Observation Models used to generate results in Butko and Movellan, CVPR 09 on your own machine. This file is included for instructional purposes -- the files that it creates are already included in the data directory (data/MIPOMDPData-21x21-4Scales-*.txt). To run:

(1) Uncompress and Expand the included GENKI R2009a dataset. Make sure the GENKI-R2009a folder is in the data directory:
>> tar -xzvf data/GENKI-R2009a.tgz -C data/

(2) Run the program.
>> bin/CVPRTrainModels

TrainNarrowFOVModel - Reproduce the Multinomial Observation Model used in the FoveatedFaceTracker example. This model has a narrow field of view, so that it cannot access the whole image at once. This illustrates how MIPOMDP can be used to simulate an active camera. This file is included for instructional purposes -- the files that it creates are already included in the data directory (data/MIPOMDPData-21x21-3Scales-*.txt). To run:

(1) Uncompress and Expand the included GENKI R2009a dataset. Make sure the GENKI-R2009a folder is in the data directory:
>> tar -xzvf data/GENKI-R2009a.tgz -C data/

(2) Run the program.
>> bin/TrainNarrowFOVModel