Square DBA – Oracle DBA Talk

Finding Large Directories

Recently I was looking to clear some space on one of my Oracle servers. I looked for large files with this command:

find . -type f -size +100000k -exec ls -lh {} \; | awk ‘{ print $9 “: ” $5 }’

This looks for files over 100MB, but other than datafiles there were very few files found. I knew I needed to look for large directories. Directories that had a large amount of small files. This command worked nicely:

du -h / | grep ^[1-9][0-9][0-9.]*G | sort -rn

From there I was able to find the directories that I needed to clear out.

Finding large files in Linux

It’s easy to track down your Oracle datafiles.

select BYTES, NAME from v$datafile;

Will tell the size of each datafile and it’s location on the server. If your server is like mine you will large non-Oracle files on the server as well. From time to time I need to find these large files to see if they can either be compressed, backed up and removed, or deleted all together. Here is a handy little statement I use on my Redhat servers.

find {/path/to/directory/} -type f -size +{size-in-kb}k
-exec ls -lh {} \; | awk '{ print $9 ": " $5 }'

so when I am looking for 200+ MB files on a specific mount I use

find /old_array -type f -size +200000k
-exec ls -lh {} \; | awk '{ print $9 ": " $5 }' 

or if I want to find files over 500MB in the current directory I will use

find . -type f -size +500000k
-exec ls -lh {} \; | awk '{ print $9 ": " $5 }' 

“Argument list too long” on Linux

Ever get the error “Argument list too long” when working with files on Linux? I deal with a lot of files from telecommunication switches. These files contain call detail records (CDRs). I generally load these files into an Oracle database using sqlldr.

These files ad up quickly and space is valuable on the server. I need to get these files off to tape. Moving these files around I run into the error “Argument list too long”. This error will affect all regular system commands (ls *, cp *, rm *, etc…).

I filter the list of files through the find command, instructing it to properly handle each file based on a specified set of command-line parameters. Due to the built-in flexibility of the find command, this workaround is easy to use. It allows you to selectively work with subsets of files based on their name patterns, date stamps, permissions and even inode numbers. In addition, and perhaps most importantly, you can complete the entire task with a single command.

The main drawback to this method is the length of time required to complete the process. This procedure actually inspects the individual properties of each file before performing the designated operation. The overhead involved can be quite significant, and moving lots of files individually may take a long time. Here is the command I use to move the files.

[user@server1 archive]$ find $archive -type f -name '*'
-exec mv {} /old_array/backup/acme/. \; 

Reclaim lost space with tune2fs

File system Reserved Block Count is supposed to reduce Linux file system defragmentation, to allow root user login for maintenance or to allow Linux system logging facility to function properly in case file system running low of free disk space.

File system utility called tune2fs can be used to tune Linux ext2 / ext3 file system parameters, such as adjusting file system reserved block count, frequency of file system force-check based, maximal time between two file system checks, behavior of the kernel code when errors are detected, overriding the default ext3 journal parameters, etc.

tune2fs -l /dev/hda

or

dumpe2fs -h /dev/vg0/lvol1

could be used to inspect a file system superblock information. Take note on these fields of both tune2fs and dumpe2fs command output:

Reserved block count: 3399024
Free blocks: 4997248
Free inodes: 42473835
First block: 0
Block size: 4096

The block size is measured in byte unit. In this case, it’s 4,096 byte or 4KB. So, the Reserved Block Count: 3399024 is equal to 13,922,402,304 byte or roughly equivalent to 14GB.

Now, if execute the command

tune2fs -m 0 /dev/vg0/lvol1

will set the percentage of file system reserved block count to 0% for /dev/vg0/lvol1 file system (the first column of df command output). It’s OK to totally disable file system reserved block count, if the file system is not used by root user account or storing system log and system/program temp files (e.g. /var and /tmp). For example, file system dedicated to Oracle datafiles are safe to disable file system reserved block count.

So, if you’re running a Linux machine with 320GB storage array and haven’t tune the file system reserved block count, you may tune it now to claim back as much as 16GB of free disk space!