Paired t-test

I always forget what P value must be to reject or not to reject null hypothesis in paired t-test. So lets explain by the example.

H0 - null hypothesis - there is no
significant difference
between method A and B
H1 - alternative hypothesis - there is difference
(two tail test)

For example [with 5% (a=0.05) level of significance ]:

Based on the above results I can say that: since P=0.009103483 and this is lover than a=0.05 (P<0.05), I can claim that:

I'm 95% (a=0.05) sure that there is significant
difference between A and B, because (P<0.05).


On the other hand, we can have:

In this case P=0.649507752, hence I can claim that:

I'm 95% (a=0.05) sure that there is no significant
difference between A and B, because (P>0.05).

Above paired t-test was performed in Excel. Read More!

Octave is getting faster!


GNU Octave "is a high-level language, primarily intended for numerical computations. It provides a convenient command line interface for solving linear and nonlinear problems numerically, and for performing other numerical experiments using a language that is mostly compatible with Matlab. It may also be used as a batch-oriented language. "

In numerical calculations speed is very important. I remember that Octave 2.x, although very useful, used to be slow. What about now when there is Octave 3.2? To answer to this question I decided to compare the time performance of this new Octave release with the performances of versions 3.0 and 2.9 using custom made benchmark script. The tests were run on Intel Mac OS X 10.4.11 (Intel Core Duo 2GHz, 2GB RAM).

 The results of 21 small tests for matrix and I/O operations are below or can be downloaded from here (gnumeric file):

The bar plot of normalized geometric mean of the 21 tests is as follows (lower is better):

The graph shows that Octave 3.2 is a bit faster than the two previous versions! Additionally, since the time values used were normalized against times obtained using MATLAB 7.4.0.287 (R2007a) on Intel Mac OS X 10.4.11 (Intel Core Duo 2GHz, 2GB RAM), it can be seen that Octave 3.2 is faster than the MATLAB R2007a. However, based on the data in the table it can be seen that the lowest performance of the Octave, as compared to Matlab is, for loop (row 13 in the table). The Octave 3.2 was nearly 700 times slower than the Matlab in execution of nested for loops! At the same time Matlab was about 1000 times slower than Octave 3.2 (but not Octave 2.9 and 3.0) in creation of sparse matrices (row 14 in the table)!!!

Should we be very excited about this! Probably not very much, since my analysis has a number of limitations. The main one is that the question whether Octave 3.2 is faster than the older versions should be answered using some real statistical approach e.g. Mann-Whiten U tests or Ordinary least products (OLP) regression analysis, influence of outliners (row 13 and 14) should be investigated etc. There is also a question whether the differences in the performance obtained are statistically significant, or whether Octave 3.2 is statistically significantly faster/slower that Matlab?

Nonetheless, I think that the results obtained clearly show a trend of increased performance for Octave.

I'm also planning to repeat the above tests for Octave 3.2, but this time with inclusion of image processing functions such as: image creation, rotation, resize, image saving etc. I could not do this for is post, because there were problems compiling image package for Octave 2.9 and 3.0 on my Mac.
Read More!

Linux Mint: Positive first impression


Today, my Arch Linux died on a PC at work. Since I had not time to play with it, I decided to install something new. First I wanted to install Ubuntu 9.10, but usually after the installation of Ubuntu I have to spent time installing audio/video codes, flash player etc. I've heard that in Linux Mint all this stuff is pre-installed. I've also heard positive comments about this Ubuntu-base distro, so today I decided to give it a try.

I downloaded Mint 7 Gloria, made backup of my home folder form Arch and I did a fresh installation on my dual-screen PC.

In short: installation was very smooth and quite fast. After the installation, I could play my video and audio files, DVDs without any tweaking or installing any packages. The one thing that required installation was nvidia driver, but there were no problems with it. As a gnome user, the only thing that I found strange was the mintMenu at the bottom panel, no top panel and very dark default Mint theme. So I had to manually adjust these things. It just took few minutes.

Some screens:









In conclusion, my first impression of Linux Mint is very positive. Now I look forward to Mint 8! Read More!

Effect size of matched and unmatched data in Maltab

Calculation of effect size for matched (paired) and unmatched (unpaired) data as described in [1].function res = effect_size(X,Y)
% Calculation of effect size for matched and unmatched data
% as described in [1].
%
% EXAMPLE from [1]:
%
% E=[27 25 30 29 30 33 31 35];
% C=[21 25 23 26 27 26 29 31];
%
% res = effect_size(E,C)
%
% res =
%
% unmatched_es: 1.2649
% unmatched_tstat: 2.5298
% matched_es: 1.2649
% matched_tstat: 4.5126
% corrcoef: 0.6857
%
% The above results agree with those in [1].
%
% NOTE
% Requires statistics toolbox in matlab/Octave
%
% REFERENCES
% Dunlop, W. P., Cortina, J. M., Vaslow, J. B., & Burke, M. J. (1996).
% Meta-analysis of experiments with matched groups or repeated measures designs. Psychological Methods, 1, 170-177.
%

[h1,p1,ci1,stats1]=ttest2(X,Y);
[h2,p2,ci2,stats2]=ttest(X-Y);

n = length(X);

r = corr2(X,Y);


d1 = stats1.tstat * sqrt(2/n);
d2 = stats2.tstat * sqrt(2/n);
d3 = stats2.tstat * sqrt(2*(1-r)/n);


res.unmatched_es = d1;
res.unmatched_tstat = stats1.tstat;
res.matched_es = d3;
res.matched_tstat = stats2.tstat;
res.corrcoef = r;
res.unmatched_tstat;
Example >> E=[27 25 30 29 30 33 31 35];
>> C=[21 25 23 26 27 26 29 31];
>> res = effect_size(E,C)

res =

unmatched_es: 1.2649
unmatched_tstat: 2.5298
matched_es: 1.2649
matched_tstat: 4.5126
corrcoef: 0.6857

>>
So, if C and E sets are matched than effect size d is 1.265. If C and E sets are unmatched than effect size d is also 1.265. The two sizes are equal for this example only.

References

[1] Dunlop, W. P., Cortina, J. M., Vaslow, J. B., & Burke, M. J. (1996).
Meta-analysis of experiments with matched groups or repeated measures designs. Psychological Methods, 1, 170-177. Read More!

Useful Matlab / Octave script to make default function arguments easier


The lack of an easy way of making default function arguments in MATLAB (I have R2007a) was annoying me for a long time. Octave (current version 3.2), which is an open source equivalent of MATLAB is much better in terms of handling default function arguments.

What I mean, can be explained best by simple example.

So, lets define a very simple m file, called testDefaultArguments.m with the following function.
function testDefaultArguments(a=1,b=2)
fprintf(1,'a=%d, b=%d\n',a,b);
We see that we have a function called testDefaultArguments with two arguments a and b, and we assign default values to them, i.e. 1 and 2, respectively. Such syntax seems to be helpful, because you can execute testDefaultArguments function as follows:octave-3.2.3:40> testDefaultArguments()
a=1, b=2
octave-3.2.3:41> testDefaultArguments(3)
a=3, b=2
octave-3.2.3:42> testDefaultArguments(a=43)
a=43, b=2
octave-3.2.3:43> testDefaultArguments(a=43,5)
a=43, b=5
octave-3.2.3:44> testDefaultArguments(a=43,b=55)
a=43, b=55
So everything seems good, but what if you want to assign only value to b, and not to a?Lets tryoctave-3.2.3:47> testDefaultArguments(b=55)
a=55, b=2 %!!! a is 55 and not b !!! This is strange
So, this is not only strange, but can easily leads to bugs in one's code.

So this was Octave. What about MATLAB? The anwser is: Error! The function testDefaultArguments can't be executed in Matlab (at least in R2007a and R2008a) due to syntax errors!

So, to help myself with making default function arguments in both MATLAB and Octave I made getfunargs.m script, which makes working with default function arguments little easier (To be honest, this function is a modification of getargs.m script). I'm not going to explain how it works, instead I'm just going to show how it can be used.

Lets define function called testDefaultArguments2 which uses getfunargs:function testDefaultArguments2(varargin)

defaults = struct(...
'a',1,...
'b',2 ...
);

args = getfunargs(defaults, varargin);

fprintf(1,'a=%d, b=%d\n',args.a,args.b);
Now, the testDefaultArguments2 can be executed both in MATLAB and Octave as follows:octave-3.2.3:52> testDefaultArguments2()
a=1, b=2
octave-3.2.3:53> testDefaultArguments2('a',3)
a=3, b=2
octave-3.2.3:54> testDefaultArguments2('b',5)
a=1, b=5
octave-3.2.3:55> testDefaultArguments2('b',5,'a',2)
a=2, b=5
It can bee seen now, that the default arguments work as expected. Read More!

Install Octave 3.2 with Image Processing package on Arch Linux


Octave is a great tool for programming mathematics. However, when I wanted to install it and its Image processing package on Arch Linux I got a problem: missing lgfortranbegin library.

To begin with I installed Octave using pacman as a root or sudo user (I prefer sudo)sudo pacman -Sy; pacman -S octave There was no problems with this.
Now, to install Image processing package it is necessary to do as follows:
To install a package, download the package file, and install it from the Octave prompt by typing pkg install package_file_name.tar.gz
where package_file_name.tar.gz is the name of the file you downloaded.
In my case the package_file_name.tar.gz was image-1.0.10.tar.gz and it can be downloaded form here

So, when I tried to install it I got the following erroroctave:1> pkg install image-1.0.10.tar.gz
warning: creating installation directory /usr/share/octave/packages
/usr/bin/ld: cannot find -lgfortranbegin
collect2: ld returned 1 exit status
make: *** [__spatial_filtering__.oct] Error 1
'make' returned the following error: make: Entering directory `/tmp/oct-9oWZxI/image-1.0.10/src'
mkoctfile __spatial_filtering__.cc
make: Leaving directory `/tmp/oct-9oWZxI/image-1.0.10/src'
error: called from `pkg>configure_make' in file /usr/share/octave/3.2.3/m/pkg/pkg.m near line 1253, column 2
error: called from:
error: /usr/share/octave/3.2.3/m/pkg/pkg.m at line 714, column 5
error: /usr/share/octave/3.2.3/m/pkg/pkg.m at line 287, column 7
As it can be seen lgfortranbegin is missing. The library belongs to fortran compiler and in Arch Linux it can be installed simply by sudo pacman -S gcc-fortranTo make sure that the library was installed I performed a search and I found it[marcin@arch ~]$ sudo find / -name "*fortranbegin*"
/usr/lib/gcc/i686-pc-linux-gnu/4.4.2/libgfortranbegin.a
After this there was no problems. I installed Image package as previously described and I checked if I got functions that I needed (e.g. imrotate, imresize) by: octave:1> pkg install image-1.0.10.tar.gz
octave:2> imr<tabulator>
imread imresize imrotate_Fourier
imremap imrotate
octave:2>
The installation was almost finished. I needed one more package, i.e. gnuplot to be able to display figuressudo pacman -S gnuplotFinally, I did some simple test to create an image, make a rotated version of it and display both of themoctave:8> I=randn(256);
octave:9> I2=imrotate(I,30);
octave:10> figure,imshow(I);
octave:11> figure,imshow(I2);


Conclusion

So in conclusion, to install octave along with Image package I needed to have gcc-fortran and gnuplot installed first:sudo pacman -S gcc-fortran gnuplot octaveThen follow general instruction on how to install Octave packages from here. Read More!

Performance benchmark script for MATLAB and Octave

The benchmark script evaluates MATLAB and Octave not only in execution times for matrix manipulation, but also for integration, 2D interpolation, solving nonlinear equation, reading/writing files, plot creating and saving, image manipulation (e.g. rotation, filtering, erosion).

The execution times of each test (27 tests for now) are normalized against times obtained using MATLAB 7.4.0.287 (R2007a) on Intel Mac OS X 10.4.11 (Intel Core Duo
2GHz, 2GB RAM).

As a results the script generates txt file with the time values (normalized and not normalized) obtained for each test.

The evaluate overall performance of MATLAB/Octave geometric and arithmetic means of the individual time values obtained are used.

The header
function res = mybench(varargin)
% res = mybench('noOfRepeats', 3,...
% 'normalize',true,...
% 'onlyTests',[],...
% 'imagep',true)
%
% Benchmark script for MATLAB and Octave.
% Tested on Matlab 2009a and Octave 3.2.2 for Windows.
%
% Execution time of basic matrix manipulation function is tested along with
% integration, solving nonlinear equation, image processing functions ( if
% available), saving/loading matrices to a file system, for loop,
% binary operation,etc. In total 27
% tests are performed (less if image processing functions not available).
%
% All results are normalized against the results obtained
% using MATLAB 7.4.0.287 (R2007a) on Intel Mac OS X 10.4.11 (Intel Core Duo
% 2GHz, 2GB RAM)
%
% At the end, arithmetic and geometric means of the times obtained are
% calculated. All results obtained are stored in a txt file named
% results_.txt.
%
% INPUT
% noOfRepeats - int - number of times each test is executed (default 3)
% normalize - boolean - normalize results (default true).
% onlyTests - int vector - do only tests given (default [], i.e. do all tests)
% imagep - boolean - do or not image processing testing
%
% OUTPUT
% res - struct - normalized geometric mean .
%
% EXAMPLES
% res = mybench(); %perform all tests with default settings.
% res = mybench('noOfRepeats',10); %perform all tests 10 times.
% res = mybench('onlyTests',[1,5,8]); %perform only tests 1,5 and 8.
% res = mybench('noOfRepeats', 1,'normalize',false); % repeat 1 time
% %each tests and do not
% %normalize results.
%
% Site: http:\\shortrecipes.blogspot.com
% Date: Nov 2009

The full code is here.
Example output from the script is as follows:



Read More!

Comparing two methods of measurement in Matalb or Octave

How to compare two methods that measure the same variable? How to check if one method produces results constantly higher than the other method and if the difference is significant? How to determine whether new method, can replace the old one? The anwser to these question lies in the analysis of bias between the two method of measurement [1]. Below is a Matlab implementation of Ordinary least products (OLP) regression analysis [1] that can be used to anwser to these questions.

function res = mgr(X,Y,varargin)
% res = mgr(X,Y)
% Mean Geometric Regression (i.e. Ordinary least products (OLP) regression)
%
% Used to evaluate two measurement methods in terms of fixed and
% proportional bias as described in [1].
%
% INPUT
% X - vector of values calculated by the first method.
% Y - vector of values calculated by the second method.
% boolean - plot or not scattergram of the values
% string - label of x axis of scattergram
% string - label of y axis of scattergram
%
%
% EXAMPLE from [1]:
%
% M1=[132 138 144 146 148 152 158 130 162 168 172 174 180 180 188 194 194 200 200 204 210 210 216 220 220];
% M2=[130 134 132 140 150 144 150 122 160 150 160 178 168 174 186 172 182 178 196 188 180 196 210 190 202];
%
% res=mgr(M1,M2)
%res =
%
% a: 13.9506
% b: 0.8611
% a_CI95: [-6.9476 32.3364]
% b_CI95: [0.7576 0.9788]
%
% res=mgr(M1,M2,true); % produces scattergram
% res=mgr(M1,M2,true,'SBP(M1)','SBP(M2)'); % produces scattergram with x and y axis
% % labels of 'SBP(M1)' and 'SBP(M2)'
%
% NOTE:
% The above calculated values of a and b agree with the values in [1]
% Values of CI95 are marginally different, becouse, as expalined in [1],
% they are calculated using approximate formulas.
%
%REFERENCE:
%[1] J Ludbrook, Comparing methods of measurement, Clinical and
% Experimental Pharmacology and Physiology, 1997:24,193-203
%
%
%
Full code of the above script is here.
Script was tested on Matlab 2007a and in Octave 3.2. En example of the scattergram that is generated by the script is below:

References

[1] J Ludbrook, Comparing methods of measurement, Clinical and Experimental Pharmacology and Physiology, 1997:24,193-203. Read More!

Ten clicks to install Ubuntu 9.04

Update: My honest mistake: It should be Ubuntu 9.10, not 9.04. Thanks Tom!



Click 1

Click Enter for English language.

Clicks 2 and 3

Go to Install Ubuntu and Click Enter.

Click 4

Forward.


Click5

Forward.



Click 6

Forward


Click 7

Forward



Click 8

Forward (Typing username and password were not counted).


Click 9

Install


Waiting for installation to complete


Click 10

Restart Now and Ubuntu installation is finished.


So 10 clicks is enough to install Ubuntu 9.04 9.10. Is it much or not? You can compare it with the number of clicks in other systems [Image taken from Benchmarked: Ubuntu vs Vista vs Windows 7]:

Read More!

How ImageMagick can save you time


ImageMagick is a great tool! Many times it has saved me from programming. Here is just one example of many applications of the ImageMagick that can save one's time.

Recently, I was asked to check whether a set of images (few hundreds of them) contains one particular image. So, basically I was given and image (called testImage.tiff), and I needed to check whether this image belongs to a set a few hundreds of images. En example of the images from the set is below:


So how to anwser this? You can do manual (i.e. visual comparison), you can also write some simple script in Python with PIL. All would take some time. But why to do this, if you can just write one for loop in bash with compare command from ImageMagick?

So first thing I did is to create the following folders:

The input folder contains all images from the set, the output folder will be where the results of my search will be stored, and the testImage.tiff is my test image.

So thanks to ImageMagick's, the only thing I had to do is to execute its compare command on all images from the set (i.e. input folder).  In other words, I compared, in a loop, an image from the set with the testImage.tiff and the results of the comparison went to output folder. Below is the bash for loop that was executed inside the input folder.
for f in *.tiff ; do
compare $f ../testImage.tiff ../output/$f ;
done
After loop finished, I could go to output folder, and quickly find out whether testImage.tiff is in the set:


As can be seen in the above picture, the testImage.tiff belongs this this set.

One limitation of this procedure is that I had to manually go through the output folder. But because all not matching images are in red, it was very quick to find, the one image that was not red. (if any). Additionally, if the testImage.tiff and images from the set were of different size or type additional code would be necessary.

So thanks to the ImageMagick, I could do the job in just a few minutes.
Read More!