TAR-ing ginormous amounts of data
James Lorenzen posted a quick how-to on his blog about compressing huge amounts of data. His post can be found here. There's one small piece i'd like to add.
If you're tar-ing up 10Gb of data, you need 10Gb of free space. What if you don't have the free space? Well, if you're moving the data (not wanting to keep the original files), here's a tip.
Just add the --remove-files parameter to your tar command. This will remove files as they're added to the archive. So you can tar up the 10Gb of data w/o having 10Gb free!
tar --remove-files -cf filename.tar directory/*
