Total size of all files with a certain extension

In our product, we have code that traverses a directory and its subdirs and totals the sizes of all files with a particular extension. I wrote a unit test for this code, but needed to independently verify the size that I was looking for to make sure the code was giving me the right answer. In our code, we use the File.length() method, which returns a long representing the amount of bytes taken up by the file, so I needed to calculate the total in bytes.

After some digging around on Google, I came up with the following bash command line expression:

find . -name *.ext | xargs du -b | cut -f 1 | awk '{total+=$0}END{print total}'

See the command man pages for details, bus as a quick summary, the find command finds all files with extension ext, which get fed into the du command that prints out the size, which gets picked up out of the du output and fed to awk, which totals everything (I am, admittedly, not initiated into the mysteries of awk, I just know that it works in this case ;).

Works like a charm! Except on Solaris, where du doesn't have a -b switch, so you get a little less detail (Solaris du reports sizes in 512-byte blocks).

Installing openldap on Opensolaris Indiana

I've used one of the old boxes here to install Opensolaris as a testing environment. The idea behind it is that, because so much of our code is Java, it'd be interesting to see if we could tackle performance problems using dtrace, which isn't available under any non-solaris OS yet.

For the testing environment to get up and running, I needed to install an ldap server to feed authentication and identity info to our product. Opensolaris comes with a bunch of ldap client software installed, but like I said, I needed a server. Sun has a product called Directory Server, but that was way too much for my needs, and besides, I don't even know if that would work on Opensolaris.

On a bunch of linux machines we had been running OpenLDAP to act as an ldap server, and that seemed to work well enough. You can download the source and compile it yourself, but I'd rather have a packaged install. Luckily, one of the Opensolaris repositories, Blastwave, had a package that sounded promising: IPSoldap@0.5.11,5.10-2.6:20080514T173322Z.

By default, the Opensolaris package manager, IPS, isn't configured to use the Blastwave repository, you have to add it. This is accomplished by executing (as root) pkg set-authority -O http://blastwave.network.com:10000 Blastwave. The last argument is an arbitrary name that you can choose yourself, the url is the important thing. The docs then say to execute pkg refresh, but it's completely unclear to me what that does. You'd expect it to update the local catalog, but issuing a search command for openldap, without the -r switch, turns up nothing. A little disappointing and unintuitive.

Anyway, if you issue the same search command with the -r switch (pkg search -r openldap) you get the following results:

basename dir opt/csw/etc/openldap pkg:/IPSoldaprt@0.5.11-2.6
basename dir opt/csw/share/openldap pkg:/IPSoldapclient@0.5.11-2.6
basename dir opt/csw/var/openldap pkg:/IPSoldap@0.5.11-2.6
basename dir usr/sfw/lib/webmin/caldera/openldap pkg:/SUNWwebmin@1.340-0.86
basename dir usr/sfw/lib/webmin/caldera/openldap pkg:/SUNWwebmin@1.340-0.75
basename dir usr/sfw/lib/webmin/caldera/openldap pkg:/SUNWwebmin@1.340-0.86
basename dir usr/sfw/lib/webmin/caldera/openldap pkg:/SUNWwebmin@1.340-0.79

The first three results are from the Blastwave repo. To install, enter pkg install IPSoldap on the command line.

I was expecting that to be it, but my system hit a snag and returned OSError: [Errno 1] Not owner: '//opt/csw/lib/i386'.
To be honest, I don't know what the hell caused that, but after several unsuccessful attempts to find an answer on several opensolaris irc channels, I just delete the dir and everything worked fine.

In summary, IPS works, but, in my opinion, it has a long way to go before it can stand comparison with linux package management systems like yum. Under Centos, installing OpenLDAP was a matter of running yum install openldap, and that was it. No repositories to add, no obscure package names to figure out, just a completely intuitive command. The documentation for IPS also isn't all that great. The man page describes the basic commands well enough, but I've yet to figure out why pkg search -r openldap turns up the OpenLDAP packages, whereas pkg search -r oldap doesn't, even though the package is called IPSoldap. You would expect that last search to return the package. Searches are apparently performed over some sort of metadata that isn't explained anywhere. Also, the pkg info command doesn't list dependencies even though the repository must know about them.