Category Archives: Linux

Wayfinder open sourced!

Posted on by
3

I got involved in a final project at Wayfinder Systems after we got permission to release a majority of our source code as open source. We’ve spent the last months cleaning up code and writing documentation and now it has finally been released! Have a look at the blog post over on the Wayfinder OSS site for more details. In the released code you will find our location based server platform and map & navigation clients for the major mobile platforms.
Vodafone has also published a press release!

I spent most of my time cleaning up and documenting the operations aspect of the server, and this includes releasing a lot of tools and scripts that we used for operating the Wayfinder server clusters but are generic tools. The two most interesting ones are WFConf; a minimal but still powerful Configuration Management tool (but I still recommend Puppet if you want to start using a CM tool) and a web based network kickstart file generator.  WFConf is documented here, and if you’re interested go ahead and look at it in the repository here. There you’ll find the less documented kickstart files as well and there is some sparse documentation you can have a look at.  The kickstart stuff is quite easy to use. You start out with a normal kickstart file and then you cut out the sections you need to vary across different servers and put the snippets  in separate files. You then create new nippets for different configurations, typically things like root passwords, disk partitioning, the post section etc. When you PXE boot your server the kickstart file is fetched from a Perl CGI which looks in a config file to determine which snippets to include for this particular server and pastes them into a complete file which is returned. In the config file you setup defaults and then specify the individual snippet configuration based on the ethernet address of the server. Here’s what it might look like:

%ks_ethers = (
   # defaults
   'default' => {
      'grubfix'      => 'nop',
      'rootpw'       => 'example',
      'partitioning' => 'one_disk_sda',
   },
   # mgmt1
   '00:aa:bb:cc:dd:ee'   => {
      'partitioning' => 'md_raid1_sda_sdb',
      'grubfix'      => 'md_raid1_sda_sdb',
   },
   # node1
   '00:aa:bb:cc:dd:ef'   => {
   },
   # node2
   '00:aa:bb:cc:dd:f0'   => {
   },
);

Combine this with a configuration management tool that setups all of your services and you can have a server with a specific configuration and role re-installed and ready to run in less than 10 minutes

Convenient interactive renaming in zsh

Posted on by
Reply

While risking turning this into the renaming files blog I just used another one of my favorite functions that I’ve named iren and realized it could be worth sharing as well. This one is really simple to put together when using zsh. It will allow you to type iren file1 file2 ... and for each file name specified you will get to interactively edit the file name using the normal line editing features in zsh. When you’re done you press enter and the file is renamed/moved. If you haven’t tried it before I can really recommend zsh, it used to be the best shell by far, although bash has caught up on many areas. iren uses the built-in zsh function vared that can be used to interactively edit any environment variable. Here’s the iren function, put it in your .zshrc file:

function iren ()
{
        while [ "$1" != "" ]
        do
                source=$1
                dest=$1
                vared dest
                if [ "$source" != "$dest" ]
                then
                        mv "$source" "$dest"
                fi
                shift
        done
}

Rename files using your favorite text editor

Posted on by
2

When renaming lots of files you usually resort to your typical for f in … command line with a sed command or similar included, or perhaps you use rename(1) but sometimes it’s just easier if you can use your normal editor. In my case this is vim, where I can then use various nice things such as changes on blocks, or just ad hoc changes to the file names. To accomplish this in an easy way I use an old quick hack; which is the following script that I’ve named viren:

#!/bin/sh

# viren
# Rename files using an editor (vim in this case)

FNEW=`mktemp -t`
FORIG=`mktemp -t`
FCMDS=`mktemp -t`
for i in $*
do
        echo "\"$i\"" >> $FORIG
done
cat $FORIG > $FNEW

vim $FNEW

wcn=`wc -l $FNEW | cut -d\  -f1`
wco=`wc -l $FORIG | cut -d\  -f1`

# go ahead if the line count is the same
if [ $wcn -eq $wco ]
then
        paste $FORIG $FNEW | sed -e 's/^/mv /g' > $FCMDS
        source $FCMDS
        echo Renamed/moved files
else
        echo Aborted rename/move
fi

rm $FNEW $FORIG $FCMDS

Move files to directories based on modification time

Posted on by
Reply

Here’s a really simple shell script to move files to directories named based on the modification time of each file. It uses only the date and the script also accepts an optional first argument to set a suffix for each directory. I sometimes use this as a first step when I organize photos and videos, the suffix is set to the source of the files (user-device in my case, eg ckk-40d if it’s from my Canon 40D), I then manually add a short description to the end based on the directory contents. Most commonly applied when I empty the card from my point-and-shoot.

#!/bin/sh

# Put files on command line in one directory per date (based on mtime) with the
# suffix given by the first optional argument on the command line

if [ ! -n "$1" ]; then
   echo "Usage:"
   echo "$0 [suffix]  ... "
   exit
fi

if [ ! -f "$1" ]; then
  suffix=$1
  shift
fi

while [ -n "$1" -a -f "$1" ]; do
  mdate=`stat --printf "%y" $1 | cut -d\  -f1`
  mkdir -p $mdate$suffix
  mv -v $1 $mdate$suffix
  shift
done