homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Hardware and OS Related Technologies / Linux, Unix, and *nix like Operating Systems
Forum Library, Charter, Moderators: bakedjake

Linux, Unix, and *nix like Operating Systems Forum

Grouping files and folders into one file
Something like tar or gzip...

5+ Year Member

Msg#: 4101980 posted 7:55 pm on Mar 21, 2010 (gmt 0)

Hello guys,

Do you have any suggestions how can I group 900K different files and insert them into one single file?

I used tar but it's too slow. It takes 8 hours to complete...

I don't need any compression.



WebmasterWorld Senior Member lammert us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

Msg#: 4101980 posted 7:39 pm on Mar 22, 2010 (gmt 0)

Are these small or large files? And are you writing the archive to the same disk, or to another disk in the system.

If the result file is on the same disk as the source files, the heads still have to move between the write location and the read location. If you have a second disk in the system (not a partition, but a real physical disk) one disk will write continuously and the other read, which will give a significant speed improvement.

Furthermore on fast processors compressing your output will actually help in speed because there are less bytes to write to disk. You could do this with tar:

tar --create --verbose --gzip --file output.tgz sourcedir

This will create a gzipped tar file called output.tgz containing the files in sourcedir.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Hardware and OS Related Technologies / Linux, Unix, and *nix like Operating Systems
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved