Tuesday, January 12, 2010

Make a big file into many small files (and back again)

Adapted from this slashdot thread

Create many little files called "chunks00000" etc. from bigfile that are all just under 10MB in size: split -a 5 -b 10MB -d bigfile chunks

Put the chunks back together: cat * > newbigfile

Do the same, except with compression: tar -zc bigfile | split -a 5 -b 10MB -d - chunks

Put the compressed chunks back together: cat chunks000* | tar -zxO > newbigfile