Question about getting a script to compare file sizes

[smithaa02]

I have about a 100 sites on a server with each path specifing their respective mail queue directory in a text file like:

/home/virtual/mysite.com/var/spool/mqueue.site
/home/virtual/anothersite.com/var/spool/mqueue.site
/home/virtual/site3.com/var/spool/mqueue.site
...

My question is, based on my list, is there an easy way to:

A) Determine the number of files in the mqueue.site directory for each website and list them in descending order.

eg mysite.com = 40 emails
site3.com = 32 emails
anothersite.com = 22 emails
etc...

B) Determine and order the websites by the size of their mqueue.site directorys
eg mysite.com = 30 meg
site3.com = 12 meg
anothersite.com = 4 meg
etc...

[physics]

For A try something like:

cd /home/virtual/
for x in `find $1 -type d -name "*mqueue.site*"`; do echo `ls -1 $x ¦ wc -l` $x ; done ¦ sort -nr

For B try something like:

cd /home/virtual/
find $1 -type d -name "*mqueue.site*" ¦ xargs du -sm ¦ sort -gr

(don't just copy and paste though as the pipe ¦ characters will be 'broken')

Your mileage may vary but hope it gets you started anyway. You can use awk to pretty up the output.

Viva la command line :)

[smithaa02]

Thanks for the reply. Unfortunately both scripts just hang and I have to kill them (each site is loaded with a huge amount of content).

Would there be a way to optimize the script?

For example what if the script found all sites in /home/virtual, added '/home/virtual/' + each result + '/var/spool/mqueue.site' to create a master list like:

/home/virtual/mysite.com/var/spool/mqueue.site
/home/virtual/anothersite.com/var/spool/mqueue.site
/home/virtual/site3.com/var/spool/mqueue.site

Then this would be piped to a listing to check number of files and sizes.

This way the initial find wouldn't have to plow through the entire subcontents of /home/virtual/mysite.com/* and the innards of the other 100 sites, which should be very efficient.

Anybody have an idea of how to implement this?

[physics]

Sure, make a file named dirs.txt that has the list you mentioned. Then do something like:

for x in `cat dirs.txt`; do echo `ls -1 $x ¦ wc -l` $x ; done ¦ sort -nr 

for x in `cat dirs.txt`; do echo `du -sm $x` $x ¦ sort -gr

[smithaa02]

Thanks for the advice. The first script worked like a charm and was very helpful.

The second disk usage script however didn't work for some reason.

When I typed in the following:

for x in `cat dirs.txt`; do echo `du -sm $x` $x ¦ sort -gr

I just got a carot prompt, which I had to kill. Is something missing from the script?

[physics]

Sorry, it should have been (was missing the done):

for x in `cat dirs.txt`; do echo `du -sm $x` $x; done ¦ sort -gr 

[smithaa02]

That did the trick. Thanks for all your help.

[physics]

Glad to help. Here's a tidbit: add

¦ mail -s"mail report" [e-mail]

at the end to have the results mailed to you.

[belliott]

To compare file sizes I would use some third-party compare utility. Bacause ... if I would need more functionality (for instance not just compare by size, but compare by content) I could do this easily. There are a lot of freeware tools avaliable or commercial if you like it :).

[edited by: tedster at 4:50 am (utc) on Oct. 27, 2006]