Hi! Please consider following me on twitter: @hanekomu.

2010年05月28日

bash brace expansion

Last time we looked at bash parameter expansion. Now I would like to talk about bash brace expansion. This is mainly a blog about Perl, but Unix tools like shells and editors are part of a Perl programmer's life as well, so from time to time I'll write about those tools too.

Brace expansion in bash is a way of generating strings. The most general form is a list of strings within braces, separated by commas. There can be other strings before and/or after the braces; they will be included in all variations. For example:

$ echo {foo,baz,baz}
foo baz baz

$ echo prefix-{foo,bar,baz}-suffix
prefix-foo-suffix prefix-bar-suffix prefix-baz-suffix

$ for i in {foo,bar,baz}; do echo $i; done
foo
bar
baz

Brace expansion can be turned on and off using bash's B option. +B turns brace expansion off, -B turns it back on.

$ set +B

$ echo {foo,baz,baz}
{foo,baz,baz}

$ set -B

$ echo {foo,baz,baz}
foo baz baz

Here are some easy examples that show what is possible:

$ echo foo{,,,}
foo foo foo foo

$ echo {1..3}{1..3}{1..3}
111 112 113 121 122 123 131 132 133 211 212 213 221 222
223 231 232 233 311 312 313 321 322 323 331 332 333

$ echo {foo,bar,baz}{1..3}
foo1 foo2 foo3 bar1 bar2 bar3 baz1 baz2 baz3

$ echo {foo{1..3},bar{4..6},baz{7..9}}
foo1 foo2 foo3 bar4 bar5 bar6 baz7 baz8 baz9

$ cp /some/very/long/path/to/my.conf{,.bak}
cp /some/very/long/path/to/my.conf /some/very/long/path/to/my.conf.bak

$ mkdir -p /Foo-Bar/{bin,lib,eg,t}
mkdir -p /Foo-Bar/bin /Foo-Bar/lib /Foo-Bar/eg /Foo-Bar/t

$ mv foo{,-}bar
mv foobar foo-bar

You can also specify ranges using the {from..to} syntax, and ranges with steps using the {from..to..step} syntax. These ranges can be numeric or alphabetic.

$ echo {1..5}
1 2 3 4 5

$ echo {5..1}
5 4 3 2 1

$ echo {3..-3}
3 2 1 0 -1 -2 -3

$ echo {1..10..2}
1 3 5 7 9

$ echo {a..e}
a b c d e

$ echo {e..a}
e d c b a

$ echo {a..z..2}
a c e g i k m o q s u w y

One typical use for brace expansion is to tell wget or curl to download several files whose URLs conform to a certain pattern:

$ wget http://example.com/path/to/{014..017}.{html,png}
wget http://example.com/path/to/014.html http://example.com/path/to/014.png
http://example.com/path/to/015.html http://example.com/path/to/015.png
http://example.com/path/to/016.html http://example.com/path/to/016.png
http://example.com/path/to/017.html http://example.com/path/to/017.png

(bash version 4 retains leading zeros; earlier bash versions omitted them.)

I also use brace expansion in my ~/.bashrc to inspect certain directories to find out whether to they contain bin/, sbin/ or man/ subdirectories that should be added to $PATH and $MANPATH:

for d in {/usr,/opt,~}{,/{local,share,local/share,perl,perl/5.*.*}}
do
    test -d "$d/bin" && PATH="$d/bin:$PATH"
    test -d "$d/sbin" && PATH="$d/sbin:$PATH"
    test -d "$d/man" && MANPATH="$d/man:$MANPATH"
done

This effectively iterates over all of these directories:

/usr
/usr/local
/usr/share
/usr/local/share
/usr/perl
/usr/perl/5.*.*
/opt
/opt/local
/opt/share
/opt/local/share
/opt/perl
/opt/perl/5.*.*
~
~/local
~/share
~/local/share
~/perl
~/perl/5.*.*

Do you have interesting examples of using the bash brace expansion mechanism?

Write a comment | Bookmark and Share

posted at: 10:12 | path: /dev | permalink | 1 comment | 0 trackbacks

2010年05月27日

Aspect-Oriented Programming, Reloaded

Back in 2001, I wrote Aspect.pm, which brought Aspect-Oriented Programming (AOP) to Perl. It was a neat toy, but to be honest, for me that's all it ever was. I never used it even in tests, much less in production.

Now that Adam Kennedy++ has taken over maintenance of the Aspect distribution, he has rewritten its core, introduced new pointcuts and join point types and wrote a blog post in which he details the future plans for AOP in Perl.

Write a comment | Bookmark and Share

posted at: 12:02 | path: /dev | permalink | 0 comments | 0 trackbacks

2010年05月25日

bash parameter expansion

The bash shell, especially in version 4, has useful parameter expansion features. I've only just started to really use them, so I wanted to make a note of the ones that seem most interesting.

$ foo="hello world goodbye world"
$ echo "${foo/world/perl}"
hello perl goodbye world
$ echo "${foo//world/perl}"
hello perl goodbye perl
$ echo "${foo^}"
Hello world goodbye world
$ echo "${foo^^}"
HELLO WORLD GOODBYE WORLD

bar="HELLO"
$ echo "${bar,}"
hELLO
$ echo "${bar,,}"
hello

$ path="/path/to/the/script"
$ echo "dirname  = ${path%/*}"
dirname  = /path/to/the
$ echo "basename = ${path##*/}"
basename = script

As you can see, ${foo/bar/baz} takes the contents of the variable foo, substitutes the first occurrence of bar with baz and returns the result. The contents of foo are unaltered. If you want to replace all occurrences, not just the first, use ${foo//bar/baz}.

Starting with bash version 4, ${foo^} uppercases the first character, while ${foo^^} uppercases the whole string. Likewise, {$foo,} lowercases the first character and {$foo,,} lowercases the whole string.

${foo#pattern} tries to match the pattern from the start of the string and deletes the matching part of the string. ${foo##pattern} is the greedy version. In the example above, to get the directory path, we greedily delete everything from the start of the string to the final slash: ${path##*/}.

There is also ${foo%pattern}, which also deletes, but starts looking from the end of the string. ${foo%%pattern} is the greedy version. In the example above, to get the filename, we delete everything from the final slash — that is, we start looking from the end until we find a slash: ${path%/*}.

Using the above techniques you might not need to invoke the external dirname and basename programs, as the shell built-in features are powerful enough.

There is a lot more to parameter expansion. See man bash for more details.

Write a comment | Bookmark and Share

posted at: 23:31 | path: /dev | permalink | 1 comment | 0 trackbacks