Linux: Copy all files from subdirectories into one folder

find zfiles/* -type f -mindepth 2 -print0 | xargs -0 cp -t ./sub1

Sourcing bash completion in .bashrc

Add the following into your .bashrc:
source /etc/profile.d/bash_completion.sh

Bash: reset bash conosole

tput reset

Linux: mkdir and cd in one command

mkcd () {
case "$1" in */..|*/../) cd -- "$1";; # that doesn't make any sense unless the directory already exists
/*/../*) (cd "${1%/../*}/.." && mkdir -p "./${1##*/../}") && cd -- "$1";;
/*) mkdir -p "$1" && cd "$1";;
*/../*) (cd "./${1%/../*}/.." && mkdir -p "./${1##*/../}") && cd "./$1";;
../*) (cd .. && mkdir -p "${1#.}") && cd "$1";;
*) mkdir -p "./$1" && cd "./$1";;
esac
}

From: http://unix.stackexchange.com/a/9124/90216

rsync: copy files with a given extensions

rsync -zarv --include="*/" --include="*.h" --exclude="*" --prune-empty-dirs /home/marcin/cpp/bitmonero ./test

xfce4: Lunch desktop file from command line

Example: exo-open /usr/share/applications/firefox.desktop

bash: rename files to numbers

find . -name '*.jpg' | gawk 'BEGIN{ a=1 }{ printf "mv \"%s\" %04d.jpg\n", $0, a++ }' | bash

sed: remove some lines except first line

sed -e '2,${/^File.*/d}' in.csv > out.csv

Git: Show updated tree of branches in console

To do this, just add the following to ~/.gitconfig[alias]
  tree = log --graph --all --decorate --pretty=oneline --abbrev-commit Then git tree will give text based tree of branches:

To run git tree in console showing e.g. 20 latest commits we can run git tree in a while loop as follows: while true; do clear; git tree -20; sleep 2; done

bash: send a command to a screen session

"Screen is a full-screen window manager that multiplexes a physical terminal between several processes, typically interactive shells." It is a very useful tool, that I often use, as it allows me to run "multiple terminals in one" even when I log out from the system.

So how to send command to screen session from other terminal. In other words how to remote-control a screen session from within a script. This post is useful for this.

In particular, I wanted to execute matlab scripts in screen windows. To do this I used the following:
1. Create screen sessionscreen -S matlab2. Create few windows in the screen session using Ctr-a c
3. Send matlab command to each as followsfor ((i=0;i<$NO_WINDOWS;i+=1)); do
echo $i
sleep 2
screen -S matlab -p $i -X exec matlab -nodesktop -r "bashCalculations('part',$i+1)"
donewhere bashCalculations is my matlab script bashCalculations.m that takes part argument. So,, each screen window executes different part of my script.

bash: split string to array

There are few possible ways of splitting a string with delimiter-separated values to an array in Bash. Here I present five of them.

1. Using tr command

Output:> [123]
> [456]
> [567]
> [5]
> [343]This method will produce incorrect array due to space in "567 5", but it works fine if there are no spaces in the $STR variable.

2. Using IFS (Internal Field Separator) variable

Output:> [123]
> [456]
> [567 5]
> [343]This method works fine even if there are spaces.

3. Using read command

Output:> [123]
> [456]
> [567 5]
> [343] This method also works fine even though there are spaces in the $STR variable.

4. Using sed command

Output:> [123]
> [456]
> [567]
> [5]
> [343]This method will also produce incorrect array due to space in "567 5", but it works fine if there are no spaces in the $STR variable.

5. Using set command

Output:> [123]
> [456]
> [567 5]
> [343] This method also works fine.

References

This post was mainly inspired by this StackOverflow question and my need to perform string 2 array conversions in bash.

bash: read a specific line or/and a column from a file

If we have column-row wise txt files, we can use awk and sed programs to read the rows and columns of such files.

Read specific line

To read a specific line from a txt file, the following commands can be usedawk 'NR==12' file.txt sed -n '12p' file.txt In both cases, line 12 will be returned.

Read specific column

awk '{print $1, $2}' file The code will return first and second columns from file.txt

Read specific column and line

Additionally, using awk we can get a specific column(s) in of a given line as followsawk 'NR==12 {print $1, $2}' file.txtThe Code will return first and second columns form a line no. 12.

bash: check size of all folders in a given directory

To list all folders in a given path

this can be usedfor f in ./* ; do if [ -d "$f" ]; then echo $f ; fi done;

To list sizes of all folders in a given path

To check the size of all folders in the current directory the above can be modified to use du -sh "$f" command: for f in * ; do if [ -d "$f" ]; then du -sh "$f" ; fi done;The command goes in a loop on every file in a current dir and if a given file is a directory, than the size is calculated.

To sort by folder size

for f in * ; do if [ -d "$f" ]; then du -s "$f" ; fi done | sort -nNote that in du command there is no -h option!

bash: extract all lines containing a string in many files

The following bash snippets scans through all txt files in current directory, searches for lines containing string "something" and saves these lines in "out" directory (the directory must be created prior to running this command) in new files that end with "-something.txt"str="something"; for f in *.txt ; do cat $f | grep $str > out/$f-$str.txt; done
Or extract last name in many txt files and save them in new file along with the name of the orginal filesfor a in *.txt; do echo $a `tail -1 $a`; done > out.txt

bash: using find plus grep command

The problem is to scan all files of some name (.e.g. *.css) and to find only those that contain specific string (e.g. 702px). find ./ -name "*.css" -exec grep '702px' {} \; -print
The command finds all .css files that contain string "702px".

bash: simple loop through files

for f in *.tiff ; do echo $f; done
Example with ImageMagick's convert program: for f in *.tiff; do convert -crop 384x384+48+35 $f out/$f; done

bash: use sed and awk to calculate an awarage value in the file

#!/bin/bash
FD=`cat $1 | sed -e 's/^[0-9.]*//g' -e 's/^\t//g' -e 's/^M$//g' \
| awk '{for (i=1; i<=NF; i++) s=s+$i}; END{print s/9/12}'`
echo $1 $FD
The script calculates avarage of values in a txt without values from the first column.
Example txt file:0 2.7792 2.7876 2.8023 2.8248 2.8413 2.8551 2.8679 2.8593 2.8537
18.281 2.7619 2.7723 2.7864 2.7888 2.8105 2.8314 2.8697 2.8695 2.8615
26.719 2.749 2.7464 2.7622 2.7811 2.8145 2.8276 2.8359 2.8438 2.8482
45 2.7596 2.7601 2.7614 2.779 2.803 2.803 2.8207 2.8391 2.8374
63.281 2.778 2.7809 2.8013 2.8071 2.8408 2.8423 2.8478 2.8514 2.826
71.719 2.7871 2.7927 2.8171 2.819 2.8366 2.8581 2.876 2.8826 2.8648
90 2.8007 2.8038 2.8219 2.8371 2.851 2.8628 2.8683 2.879 2.8719
108.28 2.781 2.7835 2.7901 2.7982 2.8214 2.8314 2.8488 2.852 2.8416
116.72 2.7731 2.7749 2.7784 2.7852 2.8077 2.8155 2.8226 2.8274 2.832
135 2.7744 2.7826 2.7854 2.7998 2.8153 2.8154 2.8358 2.8627 2.8775
153.28 2.7866 2.8011 2.8079 2.8253 2.8517 2.862 2.8727 2.8728 2.8644
161.72 2.7906 2.8071 2.8278 2.8424 2.8674 2.8679 2.8709 2.8716 2.8635


Ruby: bash image processing

As working mainly with images and image processing, sometimes I have to do one or two simple operations on many files at once like resizing, converting RGB two Gray, changing format of images, mirroring, blurring, etc. To this time I was always using Matlab to do it, because I could not find any free software on Mac X that can do such simple operations. Nonetheless, there are some issues with Matlab that annoy me, specifically: there is no native Matlab for Intel Mac X 10.4. Matlab works on Mac X 10.4 only by emulation mode, which is extremely slow and most importantly crashes very often. I had to find some more reliable solution, and I find it in Ruby.

Ruby, has very good image processing package called RMagick (interface between the Ruby programming language and the ImageMagick and GraphicsMagick image processing libraries). Installation is quite trivial. Explanation how to install RMagick on Mac X is here. I fallowed those instructions and had no problem.

#!/opt/local/bin/ruby
#convertFiles.rb
require 'rubygems'
require 'RMagick'

class ConvertFiles

 def initialize inDir='inputDir/', outDir='out/'
   raise 'No INPUT dir' if !File.exist? inDir   
   @inDir = inDir
   @outDir = outDir
   @files = Array.new  
   readFiles
   createOutDir     
 end

 def readFiles
   Dir.glob(@inDir+'*.tiff').each { |l|
     @files.push File.basename(l)
   }
 end

 def createOutDir
   Dir.mkdir(@outDir) if !File.exist? @outDir
 end

 def singleImgProcess img
   return  img.scale!(0.50)
 end

 def bashProcess
   @files.each { |fn|
     puts @inDir+fn
     img=Magick::Image.read(@inDir+fn).first
     img = singleImgProcess img
     img.write(@outDir+fn)     
   }
 end

end

Example usage is also quite simple:

c = ConvertFiles.new
c.bashProcess

Moreover, if I need some other operation or operations, I can just create new class, inheriting from the above one e.g.:

class Enlarge < ConvertFiles
def singleImgProcess img
 return  img.scale!(1.50)
end 
end

e = Enlarge.new
e.bashProcess

Or if I want to add some text to all images, I just can do:

class AddText < ConvertFiles

def setText t
  @txt = t
end

def singleImgProcess img
  @txt='Default' if @txt.nil?
  text = Magick::Draw.new
  text.annotate(img, 0, 0, 0, 60, @txt) {
    self.gravity = Magick::SouthGravity
    self.pointsize = 48
  }
  return  img
end
end


a=AddText.new
a.setText 'My pictures'
a.bashProcess

This simple code, shows how powerful and convenient Ruby can be.

Mac X: Changing bash prompt in X11's xterm

Default bash prompt in my X11's xterm was:PS1="\h:\w \u"This was very annoying as the prompt contained all the path to the current directory, which was especially bed if I went dip into the directory tree. In such a case it is better to use '\W' option instead of '\w', as this option shows only current directory name, and not full path to it. To change bash prompt use:export PS1="\h:\W \u$ "To make it permanent just add this line to ~/.bashrc.