Suppose you have some process that produces output to both stdout and stderr, like curl, for example, and you need to be able to parse both streams simultaneously.
Normally this is a pain with bash. Bash doesn’t have the handy ‘|&’ redirection that the c shells have. You can still accomplish your task, but it won’t be pretty:
res=$((((curl -sLv -X GET http://ubergibson.com \
|sed 's/^/OUT /' && echo) 3>&2 2>&1 1>&3) \
|sed 's/^/ERR /' && echo) 2>&1)
Basically what is going on here is this: the first layer of parentheses runs curl, processing stdout with sed to prepend ‘OUT ‘ to each line of output. Then stderr and stdout file descriptors are swapped, which allows us to pipe the old stderr (new stdout) to the second sed process, which prepends ‘ERR ‘ to each line.
Finally, stderr is duped to stdout and the whole mess is stored in the ‘res’ variable. Extracting the stderr and stdout from this variable is straightforward and left as an exercise to the reader ;)
Comment [3]
A continual pain when dealing with Java software is managing the CLASSPATH environment variable. I’m dealing with HtmlUnit, which depends on 12 jars. You can include everything in a big jar file if you’re deft enough with Ant, but a lot of projects – to include my personal ones – depend simply on a colon-delimited CLASSPATH of jars.
So here’s a little ditty that, fed a directory, does some magic with find(1) and searches the directory for any jar file, appending it to CLASSPATH. Behold:
MYJARS=/Users/alan/jars
export CLASSPATH=$CLASSPATH$(find $MYJARS
-name *.jar -exec printf :{} ';')
You can get fancy with find’s options if you want; there’s a shitload of them and the sky is the limit. I use printf(1) instead of echo(1), because it reliably prints its input sans-newline across most platforms.
Comment [3]
Suppose you have a directory of files, and all of the files have a common prefix that is a certain number of characters long, and you want to take the prefix off. Your directory looks like this:
[alan@shitfit testdir]$ ls
000-lol.zip 001-rofl.txt 002-roflmfao.jpg
So to get rid of the numbers and dashes in all of the files, you can do something like this:
for F in *; do mv $F ${F:4}; done
The ${F:4} bit tells Bash to print the string $F starting at the 4th character. It turns out Bash does all kinds of things with strings, as you can see here.
So, your result will look like this:
[alan@shitfit testdir]$ ls
lol.txt rofl.txt roflmfao.jpg
To tune your renaming to a certain file extension, you might use:
for F in *.txt; do mv $F ${F:4}; done
You have a number of options available, and all of them involve a loop, some kind of formatting approach, and the mv command. Micha suggested using find and piping the results to while, read, sed, and mv. This guy does something like that to match the beginnings of filenames to a pattern.
Presto.
Comment [5]
MacFUSE is an OS X kernel extension that can load usermode filesystem modules. One of the filesystems you can integrate directly into the Finder is really a pseudo filesystem: SSH. This is really handy especially since the Finder’s FTP functionality is limited to read-only. Most people who want to use the Finder or something like it for web work either buy Fetch, use Fugu, or opt for ftp, ncftp, or sftp on the command line. But MacFUSE is free, easy, secure, and rocks.
Once you have MacFUSE Core and the SSHFS application installed, you’re cooking. But how do you connect to and mount SSHFS volumes automatically?
Well, it turns out inside the sshfs.app application bundle, there’s a CLI flavor of the sshfs mounter. It’s located here:
/Applications/sshfs.app/Contents/Resources/sshfs-static-10.5
Steps for Beginners. Guru? Skip to 4
1. If you don’t already, it’s not a bad idea to take advantage of a /usr/local directory hierarchy for incidental or home-rolled command line binaries and jazz. We’re about to make a symlink to the sshfs-static-10.5 binary, and we need a spot to put the target. Below is a command which creates the specified directory as well as directories above it, as required, as instructed with the ‘p’ flag:
sudo mkdir -p /usr/local/bin
2. So now we have a spot for a symbolic link to the sshfs.app CLI that we can update in the future (for instance, for sshfs-static-10.6). We just need to add /usr/local/bin to the list of directories searched for executables. This line will do the trick:
export PATH=$PATH:/usr/local/bin
You can stick that in ~/.profile or ~/.bash_profile and it will be executed every time you open a shell.
3. I have a symbolic link from /usr/local/bin/sshfs to the above program, which I created with this line:
sudo ln -s /Applications/sshfs.app/Contents/Resources/sshfs-static-10.5 /usr/local/bin/sshfs
4. Now that we have an environment that knows about the command line version of sshfs, consider the following, an excerpt from my .bashrc:
#mount remote home directory
MOUNTPOINT=/Volumes/uberhome
REMOTE=alan@ubergibson.com:/home/alan
OPTIONS=-oping_diskarb,volname=uberhome
TESTHOST=ubergibson.com
#test if sshfs already mounted
if ! [ -e $MOUNTPOINT ]
then
#test if connected to internet
if ! [ -z "`ping -c 1 $TESTHOST 2>/dev/null | grep "time="`" ]
then
mkdir $MOUNTPOINT
sshfs $REMOTE $MOUNTPOINT $OPTIONS
echo "$REMOTE mounted on $MOUNTPOINT"
else
echo "Network down, unable to mount $REMOTE"
fi
fi
The variable assignments are self-explanatory, and are unique to my setup.
The first if statement checks if the file /Volumes/uberhome exists with the -e flag. If it doesn’t exist, meaning the volume has not yet been mounted, move on to the next test.
The next test tries to ping ubergibson.com. Successful runs of ping always result in output that contains the string “time=”, so we grep for it. The -z flag tests the result of the ping/grep for whether or not it is null. If it isn’t, mkdir a mount point and run sshfs. Then, say it worked.
My script gets run every time I open a shell. There are fancier ways of going about automounting, like customizing your login and logout with scripts and applications
There are a billion ways to skin a cat. But I hope you have fun with this one. Happy computations.
Comment [3]
The “while” loop is an ubiquitous feature of most programming languages. In languages with C-like syntax like Java and PHP it takes the form:
while( expression )
statement
In languages that support the evaluation of strings as source code, you can, using recursion, construct your own whiledo() function. Consider the following PHP function:
function whiledo($while, $do)
{
if(eval("return ".$while)) {
eval($do);
whiledo($while, $do);
}
}
$while is the expression to evaluate, and if it evaluates to true (in PHP this is any nonzero integer, any nonempty string, or the NULL constant) eval($do) is called and a recursive call to whiledo() is made.
Should eval(“return “.$while) evaluate to anything non-true, the block is skipped and the function stops calling itself.
Bizarre, no? To test the function it gets a bit nasty because the inner-function eval() call necessitates global variables. Try this:
<?
/*
* The "while, do" function
* Inner-function eval() necessitates globals
*/
global $i, $lim;
function whiledo($while, $do) {
if(eval("return ".$while)) {
eval($do);
whiledo($while, $do);
}
}
$i = 0;
$lim = 10;
whiledo('$GLOBALS[\'i\'] < $GLOBALS[\'lim\'];', '
++$GLOBALS[\'i\'];
echo $GLOBALS[\'i\']."\n";
');
?>
Comment [4]