PostgreSQL Log Compressing and Moving Script

This scripts are using to tar(zip compress) and remove more than 1 month old postgresql log and linux log file
1.This script used for moving and compressing  older than 30 days log files:

vi /home/postgres/eds/tarlog.sh
find /data/emut16_slavedata/pg_log/ -mtime +30 | xargs  tar -czvPf /pg_xlog/older_log.tar.gz

vi /home/postgres/eds/removelogs.sh
find /data/emut16_slavedata/pg_log/ -type f -name "*.log" -mtime +30 -exec rm {} \;


but before that get a listing to see what you are about to delete and tar
find /data/emut16_slavedata/pg_log/ -type f -name "*.log" -mtime +30 -exec ls -tr {} \;

2.This script used for moving and compressing  older than 30 minutes and hours  log files:

vi /home/postgres/eds/tarlog.sh
find /data/emut16_slavedata/pg_log/ -cmin +30 | xargs  tar -czvPf /pg_xlog/older_log.tar.gz

vi /home/postgres/eds/removelogs.sh
find /data/emut16_slavedata/pg_log/ -type f -name "*.log" -cmin +30 -exec rm {} \;


but before that get a listing to see what you are about to delete and tar
find /data/emut16_slavedata/pg_log/ -type f -name "*.log" -cmin +30 -exec ls -tr {} \;

If you want to give 3 hours mention like -cmin+180 

Three times tracked for each file in Unix are these:

1. access time – atime
2. change time – ctime
3. modify time – mtime

1.atime

• This is the Access time :
• atime is updated whenever file’s data is accessed (a read, write or any other access); this could be done by a system process, interactively by CLI or by a script.
• copy (cp) time will be updated

2.mtime

• This is the Modification time :
• mtime is updated whenever the file’s content changes. This time stamp is not updated by a change of files permissions (e.g : through a chown command). It is usually used for tracking the file content changes (see the Linux time related tools section below for more infos).
• mtime changing implies a ctime change too.
• So basically mtime represents the file’s data age.
• updating file contents time will be updated

3.ctime

• This is the Change time :
• ctime is updated when  file’s ownership, access permissions or file contents are modified. As stated in ls manual (man ls) : time of last modification of file status information. ctime is updated when the inode data changes.
• Move(mv),updating file contents,changing file owner time will be updated

Example:
If we want to see the last access time for this file, atime – you need to use -lu options

ls -lu /tmp/nijam.txt
-rw-r--r-- 1 root root 9 2008-04-05 07:27 /tmp/nijam.txt

ls -lc will show you the last time our file was changed, ctime:

ls -lc /tmp/nijam.txt
-rw-r--r-- 1 root root 9 2008-04-05 07:35 /tmp/nijam.txt

By default ls used with the “-l” option displays the mtime.

ls -l /tmp/nijam.txt
-rw-r--r-- 1 root root 9 2008-04-05 07:10 /tmp/nijam.txt

stat command, which can be used to show all of the times in a more convenient way

# stat /tmp/nijam.txt
File: `/tmp/nijam.txt'
Size: 9             Blocks: 8          IO Block: 4096   regular file
Device: 811h/2065d    Inode: 179420      Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2008-04-05 07:27:51.000000000 +0100
Modify: 2008-04-05 07:10:14.000000000 +0100
Change: 2008-04-05 07:35:22.000000000 +0100

File Systems and time stamping

FileSystems	atime	ctime	mtime	Creation time
ext2	yes	yes	yes	no
ext3	yes	yes	yes	no
ext4	yes	yes	yes	yes
btrfs	yes	yes	yes	yes
reiserfs	yes	yes	yes	no
zfs	yes	yes	yes	yes
ntfs	yes	yes	yes	yes

More info about ctime,mtime,atime,cmin,amin,mmin
Find:
-type f  :Find file in a directory.
-type d :Find directories in a directory.
-name  :Find all log file whose file end with .log in a directory.
-size     :size of files/Dirrectories
-perm   :File/Directory permission
-user    :username      
-group  :group name
  !        : not in example for ! -name,! -perm,! -user

Find all the files whose permissions are 766:
find -type f -perm 0766 -print

Find all the files without permission 766:
find -type f ! -perm 766

Find Read Only Files:
find / -perm /u=r

Find all Executable files:
find / -perm /a=x

Find Directories with 777 Permissions and Chmod to 766:
find / -type d -perm 777 -print -exec chmod 766 {} \;

Find files with 777 Permissions and Chmod to 766:
find / -type d -perm 777 -print -exec chmod 766 {} \;

To file all empty directories under certain path:
find /tmp -type d -empty

Find Single File Based on User:
find / -user nijam -name callmegirls.txt

Find all Files Based on Group:
find / -user nijamgroup

To find all .txt files of user nijam under /home directory:
find /home -user tecmint -iname "*.txt"

To find all the files which are modified 30 days back:
find -mtime 30

Find Last 30 Days Accessed Files:
find -atime 30

Find Last 30 Days changed Files:
find -ctime 30

To find all the files which are modified more than 30 days back and less than 60 days:
find / -mtime +30 –mtime -60

Find Changed Files in Last 1 Hour:
find / -cmin -60

To find all 30MB files:
 find / -size 30M

To find all the files which are greater than 30MB and less than 100MB(30MB – 100MB):
find / -size +30M -size -100M

To find all 50MB files and delete them using one single command:
find / -size +50M -exec rm -rf {} \;

Find all .log files with more than 10MB and delete them using one single command:
find / -type f -name *.log -size +10M -exec rm {} \;

Search specific directory or path:
find ./nijam
./nijam
./nijam/call.txt
./nijam/subdir

Find largest and smallest files:
The following command will display the 10 largest file in the current directory and its subdirectory.
$ find . -type f -exec ls -s {} \; | sort -n -r | head -10

Similary when sorted in ascending order, it would show the smallest files first
$ find . -type f -exec ls -s {} \; | sort -n | head -10

Limit depth of directory traversal:

• For example we don't want to go more than 2 or 3 levels down in the sub directories. This is done using the maxdepth option.
• The second example uses maxdepth of 1, which means it will not go lower than 1 level deep, either only in the current directory.

$ find ./test -maxdepth 2 -name *.php
./test/subdir/how.php
./test/cool.php

$ find ./test -maxdepth 1 -name *.php
./test/cool.php


To search for files that do no match a given name or pattern:
$ find ./test -not -name *.php
./test
./test/abc.txt
./test/subdir

To use multiple criterias when specifying name and inverting:
$ find ./test -name 'abc*' ! -name '*.php'
./test/abc.txt
./test/abc

To search inside 2 separate directories:
$ find ./test ./dir2 -type f -name abc*
./test/abc.txt
./dir2/abcdefg.txt

Find hidden files:
find ~ -type f -name ".*"