Welcome to WebmasterWorld Guest from 54.234.129.215

Forum Moderators: bakedjake

Message Too Old, No Replies

Grouping files and folders into one file

Something like tar or gzip...

     

sunroof

7:55 pm on Mar 21, 2010 (gmt 0)

5+ Year Member



Hello guys,

Do you have any suggestions how can I group 900K different files and insert them into one single file?

I used tar but it's too slow. It takes 8 hours to complete...

I don't need any compression.

lammert

7:39 pm on Mar 22, 2010 (gmt 0)

WebmasterWorld Senior Member lammert is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Are these small or large files? And are you writing the archive to the same disk, or to another disk in the system.

If the result file is on the same disk as the source files, the heads still have to move between the write location and the read location. If you have a second disk in the system (not a partition, but a real physical disk) one disk will write continuously and the other read, which will give a significant speed improvement.

Furthermore on fast processors compressing your output will actually help in speed because there are less bytes to write to disk. You could do this with tar:

tar --create --verbose --gzip --file output.tgz sourcedir

This will create a gzipped tar file called output.tgz containing the files in sourcedir.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month