Welcome to WebmasterWorld Guest from

Forum Moderators: bakedjake

Message Too Old, No Replies

Grouping files and folders into one file

Something like tar or gzip...

7:55 pm on Mar 21, 2010 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 8, 2005
posts: 83
votes: 0

Hello guys,

Do you have any suggestions how can I group 900K different files and insert them into one single file?

I used tar but it's too slow. It takes 8 hours to complete...

I don't need any compression.
7:39 pm on Mar 22, 2010 (gmt 0)

Senior Member from KZ 

WebmasterWorld Senior Member lammert is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 10, 2005
posts: 2947
votes: 30

Are these small or large files? And are you writing the archive to the same disk, or to another disk in the system.

If the result file is on the same disk as the source files, the heads still have to move between the write location and the read location. If you have a second disk in the system (not a partition, but a real physical disk) one disk will write continuously and the other read, which will give a significant speed improvement.

Furthermore on fast processors compressing your output will actually help in speed because there are less bytes to write to disk. You could do this with tar:

tar --create --verbose --gzip --file output.tgz sourcedir

This will create a gzipped tar file called output.tgz containing the files in sourcedir.

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members