Welcome to WebmasterWorld Guest from 54.205.251.179

Forum Moderators: bakedjake

Message Too Old, No Replies

Cleaning the disk - getting rid of old files

   
3:41 pm on Aug 22, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



For some reason, my disks seem to have accumulated a lot of files. 20Gb, 11Gb in use. I'm sure when I first had this, only 2Gb was in use.

Where has all the bloat come from?

1. What directories require regular pruning?
I guess the log files is one directory. I have notice that they are all on rotation and old files are gzip'd but theres still loads of them in there

I notice in /tmp that there is an mm_apache_session.mem or something file. This is over 800Mb in size. What the heck is this?

2. Is there like a defrag or empty trash routine?
Like windows, are deleted files held in a trash until purged?
Is there a compact routine to clear free space?

3. I think the bloat could have come from all the dist-upgrades I have done recently.
If the updates and upgrades backup existing files then surely it is ok to delete these backups after the upgrades have been running for a while. Where are the backup files? Which are safe to delete?

Some say dont bother cleaning out - just leave it. I'm not like that. I strip Windows 98 to the bone and just leave essential files. But the main reason for the Linux server is that I have a dual disk system which copies 1 complete disk onto the backup disk each night. Copying 11GB must surely take some time and slow things down for the twilight visitors.

12:55 am on Aug 23, 2002 (gmt 0)

10+ Year Member



1. I find that the directories that need more pruning are the ones where the videogame and movie trailers end up :) But you are right, you should move away the logs, maybe burning on CD when they, even compressed, begin to take too much space.

I don't know about that mm_apache_session.mem, but sounds like it is some kind of buffer or cache or something. If it is not being used it should be deletable; try 'lsof ¦ grep mm_apache_session.mem' and see if something is using it.

2. There is not defrag; the ext2 filesystem defrags itself all the time (I think). Up to my knowledge, the deleted files are gone forever, unless you are using KDE, for example, that has its own "recycle bin".

3. You have done dist-upgrades... Is that Debian? If it is, look at /var/cache/apt/archives/; if it is too heavy do a 'apt-get clean' to get rid of them. You would like to burn them, too, if you're planning to reinstall something you uninstalled.

1:49 pm on Aug 23, 2002 (gmt 0)

10+ Year Member



Do a telinit 1 and then clean up everything in /tmp that seems temporary. You can also delete some stuff from /var.
4:12 pm on Aug 23, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



hey guys, some good tips there. I'll do some pruning later this evening.

I'm getting worried though, I just checked the disk space and I'm now using 13Gb, so thats 2Gb added in a couple of days.

WTH is going on?

I dont think there is anything suspicious going on because the logs ok to me so far.

Any ideas why 2Gb is being added in a couple of days?

4:53 pm on Aug 23, 2002 (gmt 0)

10+ Year Member



Thats a bunch for two days. While I dont have the answer one of the local experts should be able to help with a command to show you all modified files from yesterday.

That kind of file growth sounds like a error log or a program stuck in a loop.

At that rate you will be out of space in a week or so.

8:17 pm on Aug 23, 2002 (gmt 0)

10+ Year Member



This comes from the back of my head and the 'find' man page...

find / -name '*' -ctime 1

will find all the files whose status was changed on the last 24 hours.

<added> I think that it was more like -ctime 0, and -ctime 1 gave you the updated files from the day before yesterday... play with the values.</added>

8:38 pm on Aug 23, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



arggh, that produces a list of thousands of files!

I have spotted that there is a /backup directory. I think this is where the backup operation thing goes. it should mount itself as /backup on the second disk. But I think its screwed.

There is a /backup directory with a complete copy of the / and subdirs. But also in this is /backup/backup and /backup/backup/backup

so it looks as if the backup is backing up itself and has done so for a few days.

This could explain why 2Gb a day is dissapearing - the backup is copying itself to a directory called /backup on disk 1 and not onto the mount of disk 2 if you know what I mean.

I just tried mounting the second disk via

mount /dev/hdc3

and

mount /backup

but it comes back with an error

wrong fs type, bad option, bad superblock on /dev/hdc3, or too many mounted file systems

I think this is one for the webhost support droids.

9:31 pm on Aug 23, 2002 (gmt 0)

10+ Year Member



so it looks as if the backup is backing up itself and has done so for a few days.

That's why I like coming here. Sheer comedy value ;)

Sidenote: You can't mount a filesystem just like that unless there is an entry at /etc/fstab; to mount anything else the mantra is 'mount -t filesystem-type device-address mount-point; for example, to mount a cdrom you'd do 'mount -t iso9660 /dev/cdrom /mnt/cdrom'.

10:02 pm on Aug 23, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Ok in fstab I got:

-------------------------------------------

# <file system> <mount point> <type> <options> <dump> <pass>
/dev/hda1 /boot ext3 defaults 0 1
/dev/hda2 none swap sw 0 0
/dev/hda3 / ext3 defaults 0 1

proc /proc proc defaults 0 0

/dev/fd0 /floppy auto defaults,user,noauto 0 0
/dev/cdrom /cdrom iso9660 defaults,ro,user,noauto 0 0

# Backup drive
/dev/hdc1 /backup/boot ext3 defaults,noauto 0 1
/dev/hdc2 none swap sw 0 0
/dev/hdc3 /backup ext3 defaults,noauto 0 1

-----------------------------------------

So whats that all about eh?

In the past I have just typed in

mount backup

Ahhh, JEEEEZE, I just remembered what it could be! I changed some disk settings the other day as in this thread:

[webmasterworld.com...]

hdparm -c3 -m16 /dev/hda
hdparm -X34 -d1 -u1 /dev/hda

[expletive mode off] Man, what do I do now? I better fix it before those support droids find out and charge me for fixing it :)

I think I'll try a reboot first as I didnt save the settings so a reboot should put it back to the slow settings right?

8:30 am on Aug 24, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Ok,

I dont think it is to do with me meddling with the speed up of the disk functions. I did that around 1-Aug but the disk2 /backup problem started around 18-Aug.

The webhosters droids have confirmed that disk2 has a hardware fault and they will swap it out.

Amazes me that the dumb backup script doesnt detect the failure and dumps the backup files on the disk it is trying to backup!

Thanks for your help though guys.

9:12 am on Aug 25, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



try
du -h -d 1

while in /

(not sure if there is a "-d 1" option in the gnu version of du, but try it).

That will print out the size of all top level directories.
If the "-d" option is present - try "-d 2", that will give you a good breakdown on what's taking up space.

If you don't have the "-d" option - then check the man page and see how to set the depth in your verson of the utility.

The app might take a while to execute.
Be sure you are logged in as root.