Bash Recipes Part 2 - Process a string character by character

For splitting a string character by character, or doing any sort of operation per each character use the following code snippet

safeer@penguinpower:~$ NAME=safeer;for((i=0;i<${#NAME};i++)); do echo ${NAME:$i:1}; done

s

a

f

e

e

r

${#NAME} is the length of the variable $NAME

${NAME:$i:1} - This is a bash parameter substitution technique. U can extract a substring of a variable by using the syntax "${parameter:offset:length}" - This expands to up to "length" characters of parameter starting at the character specified by "offset" - The offset starts at zero.

In our case we use $i as the offset and in every iteration of the for loop we increment $i from zero to length of the string minus one ( which is the list of offsets for every single character in the string ). The length of substring is always 1 ( hence single character ). So ultimately per every iteration we extract one character from the string advancing from the beginning to the end.

Now if you want to extract the character in the reverse order, start the index from -1 and down to negative of string length.

safeer@penguinpower:~$ A=safeer;for((i=-1;i>=-${#A};i--)); do echo ${A:$i:1}; done

r

e

e

f

a

s

Managing ISO files in Linux

ISO (International organizations for standardization) format is the archive file format for optical discs. The name is derived from the file system format for optical discs, known as ISO 9660 File System. This format is also known as CDFS (Compact Disc File System). ISO files are used to either create the image of an optical disc, or to write data to it.

Here are a few tricks to create and manage ISO files.

Creating an ISO image out of a CD ROM mounted in your machine.

We can use the dd (Disc Dump) command to take the copy of an optical disc. For this first you have to insert the disc in the drive, then find out the device name of your CD-ROM.

Once the CD is mounted, you can find your CD drive in this way:

safeer@penguinpower:~$ mount|grep -i iso
/dev/sr0 on /media/UBUNTU_10 type iso9660(ro,nosuid,nodev,uhelper=udisks,uid=1000,gid=1000,iocharset=utf8,mode=0400,dmode=0500)

Here my CD device is /dev/sr0. In most of the systems where you have a single CD drive, this will be symlinked to /dev/cdrom, so you can even use that in place of /dev/sr0 ( or whatever your device is).

safeer@penguinpower:~$ ls -l /dev/cdrom
lrwxrwxrwx 1 root root 3 2010-11-06 10:23 /dev/cdrom -> sr0

Now that we know our CD device name, let us create the ISO image of the mounted disc.

safeer@penguinpower:~$ dd if=/dev/sr0 of=/home/safeer/ISO/ubuntu10.iso
1365028+0 records in
1365028+0 records out
698894336 bytes (699 MB) copied, 267.224 s, 2.6 MB/s

Where "/dev/sr0" is my CD device and "/home/safeer/ISO/ubuntu10.iso" is the name of the iso image that will be created from the CD.

Mounting an ISO file as if you are mounting a CD-ROM.

ISO file can be mounted in a UNIX file system just like inserting a CD to your CD drive. We make use of the loop device system for this purpose. Loop device is a pseudo file system available in Linux/UNIX that makes a file accessible as a block device.

First, create a mount point

safeer@penguinpower:~$ sudo mkdir /mnt/iso

Then mount the ISO file as loop device

safeer@penguinpower:~$ sudo mount -o loop -t iso9660 /home/safeer/ISO/ubuntu10.iso /mnt/iso/
safeer@penguinpower:~$ mount |grep iso
/dev/loop0 on /mnt/iso type iso9660 (rw)

Now you can browse the contents of iso /home/safeer/ISO/ubuntu10.iso via the directory /mnt/iso/.

To unmount the device:

safeer@penguinpower:~$ sudo umount /mnt/iso

Creating an ISO file from a directory.

You can use mkisofs command to create an iso. In my system mkisofs is a symlink to the program genisoimage. You can use either command for this purpose.

Here I will create an iso image of all softwares that I downloaded.

safeer@penguinpower:~$ mkisofs -o software.iso /home/safeer/Downloads/
I: -input-charset not specified, using utf-8 (detected in locale settings)
Using VPNC_000.RPM;1 for /vpnc-0.5.3-8.fc13.src.rpm (vpnc-0.5.3-6.fc11.x86_64.rpm)
Using VPNC_001.RPM;1 for /vpnc-0.5.3-6.fc11.x86_64.rpm (vpnc-0.5.3-8.fc13.x86_64.rpm)
1.03% done, estimate finish Sun Nov 7 18:28:59 2010
2.06% done, estimate finish Sun Nov 7 18:28:59 2010
................................
................................
98.68% done, estimate finish Sun Nov 7 18:29:33 2010
99.71% done, estimate finish Sun Nov 7 18:29:34 2010
Total translation table size: 0
Total rockridge attributes bytes: 0
Total directory bytes: 4914
Path table size(bytes): 36
Max brk space used 1b000
486421 extents written (950 MB)

Let us check the iso image by mounting it.

safeer@penguinpower:~$ sudo mount -t iso9660 -o loop software.iso /mnt
safeer@penguinpower:~$ cd /mnt/
safeer@penguinpower:/mnt$ ls
a0654873.tor google_c.deb skype_de.deb torrentd.tor vpnc_000.rpm vpnc_0_5.deb vpnc_con.rpm
adberdr9.deb opera_10.deb skype_ub.deb virtualb.deb vpnc_001.rpm vpnc_0_5.rpm vpnc_scr.rpm

The problem with using the default settings for mkisofs is that it truncates the filenames to be compatible with DOS file system. A few options that will help us create better image is shown below.

safeer@penguinpower:~$ mkisofs -r -R -J -l -o software.iso /home/safeer/Downloads/
I: -input-charset not specified, using utf-8 (detected in locale settings)
1.03% done, estimate finish Sun Nov 7 18:56:33 2010
....................................................
99.71% done, estimate finish Sun Nov 7 18:57:00 2010
Total translation table size: 0
Total rockridge attributes bytes: 5152
Total directory bytes: 11248
Path table size(bytes): 36
Max brk space used 1c000
486433 extents written (950 MB)

Checkout the man pages for more details on mkisofs

Perl Snippets Part 2 - CSV to XLS

This program will convert a CSV ( Comma Separated Value ) file to an XLS ( Windows Excel ) spreadsheet. The script should be provided with the CSV file name as the first command line argument. It takes an optional second argument as the output XLS file name. If no output filename is given, the output will be saved to out.xls in the current directory.

This script requires Spreadsheet::WriteExcel perl module. On my Ubuntu box, this was installed with the command:
safeer@penguinpower:~$sudo apt-get install libspreadsheet-writeexcel-perl

#!/usr/bin/perl -w

use Spreadsheet::WriteExcel;

if (!defined $ARGV[0]) { die "You must provide a csv file"; }
else {$CSV_FILE = $ARGV[0];}

if ( defined $ARGV[1] ) { $XLS_FILE = $ARGV[1]; }
else { $XLS_FILE = "out.xls"; }

unless ( open ( CSV,$CSV_FILE))
{ die "Unable to open ".$CSV_FILE;}

my $workbook = Spreadsheet::WriteExcel->new($XLS_FILE);
if ( !defined $workbook ) {
             die "Unable to create excel file\n";
         }
$worksheet = $workbook->add_worksheet();
$col = $row = 0;

while (<CSV>)
{
chomp;
my @line = split(",",$_);
for $cell (@line){
      $worksheet->write($row, $col,$cell);
      $col++;
      }
$row++; $col = 0;
}
$workbook->close();
close (CSV);

Bash Recipes Part 1 - Changing extensions for multiple file names

I have came across many situations where a bunch of files with one filename extension has to be renamed to another extension. Here is the way to do it in bash.

We will rename a number of post script files to PDF files ( .ps to .pdf )

safeer@penguinpower:~/Documents$ ls
book0.ps book1.ps book2.ps book3.ps book4.ps
safeer@penguinpower:~/Documents$ for file in $(ls *.ps);
do
mv $file ${file/%.ps/.pdf};
done
safeer@penguinpower:~/Documents$ ls
book0.pdf book1.pdf book2.pdf book3.pdf book4.pdf

Perl Snippets Part 1 - XLS to CSV

For parsing Microsoft excel files (.xls) using Perl, we can make use of the module Spreadsheet::ParseExcel.
On my Ubuntu system this module is installed by:

safeer@penguinpower:~$sudo apt-get install libspreadsheet-parseexcel-perl

Following program will take an XLS file as input and print the csv format to standard out.

#!/usr/bin/perl -w

use strict;
use Spreadsheet::ParseExcel;

my $parser = Spreadsheet::ParseExcel->new();
if ( !defined $ARGV[0] ) { die "Please provide an excel file"; }
my $workbook = $parser->parse($ARGV[0]);
if ( !defined $workbook ) { die $parser->error(), ".\n"; }

for my $worksheet ( $workbook->worksheets() ) {

my ( $row_min, $row_max ) = $worksheet->row_range();
my ( $col_min, $col_max ) = $worksheet->col_range();

for my $row ( $row_min .. $row_max ) {
for my $col ( $col_min .. $col_max ) {

my $cell = $worksheet->get_cell( $row, $col );
if ( defined $cell ) { print $cell->value(); }
if ( $col < $col_max ) { print ","; }        
}   
print "\n";
}
}