Wednesday, November 04, 2009

Performance benchmark script for MATLAB and Octave

The benchmark script evaluates MATLAB and Octave not only in execution times for matrix manipulation, but also for integration, 2D interpolation, solving nonlinear equation, reading/writing files, plot creating and saving, image manipulation (e.g. rotation, filtering, erosion).

The execution times of each test (27 tests for now) are normalized against times obtained using MATLAB 7.4.0.287 (R2007a) on Intel Mac OS X 10.4.11 (Intel Core Duo
2GHz, 2GB RAM).

As a results the script generates txt file with the time values (normalized and not normalized) obtained for each test.

The evaluate overall performance of MATLAB/Octave geometric and arithmetic means of the individual time values obtained are used.

The header
function res = mybench(varargin)
% res = mybench('noOfRepeats', 3,...
% 'normalize',true,...
% 'onlyTests',[],...
% 'imagep',true)
%
% Benchmark script for MATLAB and Octave.
% Tested on Matlab 2009a and Octave 3.2.2 for Windows.
%
% Execution time of basic matrix manipulation function is tested along with
% integration, solving nonlinear equation, image processing functions ( if
% available), saving/loading matrices to a file system, for loop,
% binary operation,etc. In total 27
% tests are performed (less if image processing functions not available).
%
% All results are normalized against the results obtained
% using MATLAB 7.4.0.287 (R2007a) on Intel Mac OS X 10.4.11 (Intel Core Duo
% 2GHz, 2GB RAM)
%
% At the end, arithmetic and geometric means of the times obtained are
% calculated. All results obtained are stored in a txt file named
% results_.txt.
%
% INPUT
% noOfRepeats - int - number of times each test is executed (default 3)
% normalize - boolean - normalize results (default true).
% onlyTests - int vector - do only tests given (default [], i.e. do all tests)
% imagep - boolean - do or not image processing testing
%
% OUTPUT
% res - struct - normalized geometric mean .
%
% EXAMPLES
% res = mybench(); %perform all tests with default settings.
% res = mybench('noOfRepeats',10); %perform all tests 10 times.
% res = mybench('onlyTests',[1,5,8]); %perform only tests 1,5 and 8.
% res = mybench('noOfRepeats', 1,'normalize',false); % repeat 1 time
% %each tests and do not
% %normalize results.
%
% Site: http:\\shortrecipes.blogspot.com
% Date: Nov 2009

The full code is here.
Example output from the script is as follows: