Drop Down MenusCSS Drop Down MenuPure CSS Dropdown Menu

PostgreSQL Log Compressing and Moving Script

This scripts are using to tar(zip compress) and remove more than 1 month old postgresql log and linux log file
1.This script used for moving and compressing  older than 30 days log files:
vi /home/postgres/eds/tarlog.sh
find /data/emut16_slavedata/pg_log/ -mtime +30 | xargs  tar -czvPf /pg_xlog/older_log.tar.gz

vi /home/postgres/eds/removelogs.sh
find /data/emut16_slavedata/pg_log/ -type f -name "*.log" -mtime +30 -exec rm {} \;


but before that get a listing to see what you are about to delete and tar
find /data/emut16_slavedata/pg_log/ -type f -name "*.log" -mtime +30 -exec ls -tr {} \;
2.This script used for moving and compressing  older than 30 minutes and hours  log files:
vi /home/postgres/eds/tarlog.sh
find /data/emut16_slavedata/pg_log/ -cmin +30 | xargs  tar -czvPf /pg_xlog/older_log.tar.gz

vi /home/postgres/eds/removelogs.sh
find /data/emut16_slavedata/pg_log/ -type f -name "*.log" -cmin +30 -exec rm {} \;


but before that get a listing to see what you are about to delete and tar
find /data/emut16_slavedata/pg_log/ -type f -name "*.log" -cmin +30 -exec ls -tr {} \;
If you want to give 3 hours mention like -cmin+180 

Three times tracked for each file in Unix are these:
  1. access time – atime
  2. change time – ctime
  3. modify time – mtime
1.atime
  • This is the Access time :
  • atime is updated whenever file’s data is accessed (a read, write or any other access); this could be done by a system process, interactively by CLI or by a script.
  • copy (cp) time will be updated

2.mtime
  • This is the Modification time :
  • mtime is updated whenever the file’s content changes. This time stamp is not updated by a change of files permissions (e.g : through a chown command). It is usually used for tracking the file content changes (see the Linux time related tools section below for more infos).
  • mtime changing implies a ctime change too.
  • So basically mtime represents the file’s data age.
  • updating file contents time will be updated

3.ctime
  • This is the Change time :
  • ctime is updated when  file’s ownership, access permissions or file contents are modified. As stated in ls manual (man ls) : time of last modification of file status information. ctime is updated when the inode data changes.
  • Move(mv),updating file contents,changing file owner time will be updated

Example:
If we want to see the last access time for this file, atime – you need to use -lu options
ls -lu /tmp/nijam.txt
-rw-r--r-- 1 root root 9 2008-04-05 07:27 /tmp/nijam.txt

ls -lc will show you the last time our file was changed, ctime:
ls -lc /tmp/nijam.txt
-rw-r--r-- 1 root root 9 2008-04-05 07:35 /tmp/nijam.txt

By default ls used with the “-l” option displays the mtime.
ls -l /tmp/nijam.txt
-rw-r--r-- 1 root root 9 2008-04-05 07:10 /tmp/nijam.txt

stat command, which can be used to show all of the times in a more convenient way
# stat /tmp/nijam.txt
File: `/tmp/nijam.txt'
Size: 9             Blocks: 8          IO Block: 4096   regular file
Device: 811h/2065d    Inode: 179420      Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2008-04-05 07:27:51.000000000 +0100
Modify: 2008-04-05 07:10:14.000000000 +0100
Change: 2008-04-05 07:35:22.000000000 +0100
File Systems and time stamping
FileSystemsatimectimemtimeCreation time
ext2yesyesyesno
ext3yesyesyesno
ext4yesyesyesyes
btrfsyesyesyesyes
reiserfsyesyesyesno
zfsyesyesyesyes
ntfsyesyesyesyes

More info about ctime,mtime,atime,cmin,amin,mmin
Find:
-type f  :Find file in a directory.
-type d :Find directories in a directory.
-name  :Find all log file whose file end with .log in a directory.
-size     :size of files/Dirrectories
-perm   :File/Directory permission
-user    :username      
-group  :group name
  !        : not in example for ! -name,! -perm,! -user

Find all the files whose permissions are 766:
find -type f -perm 0766 -print

Find all the files without permission 766:
find -type f ! -perm 766

Find Read Only Files:
find / -perm /u=r

Find all Executable files:
find / -perm /a=x

Find Directories with 777 Permissions and Chmod to 766:
find / -type d -perm 777 -print -exec chmod 766 {} \;

Find files with 777 Permissions and Chmod to 766:
find / -type d -perm 777 -print -exec chmod 766 {} \;

To file all empty directories under certain path:
find /tmp -type d -empty

Find Single File Based on User:
find / -user nijam -name callmegirls.txt

Find all Files Based on Group:
find / -user nijamgroup

To find all .txt files of user nijam under /home directory:
find /home -user tecmint -iname "*.txt"

To find all the files which are modified 30 days back:
find -mtime 30

Find Last 30 Days Accessed Files:
find -atime 30

Find Last 30 Days changed Files:
find -ctime 30

To find all the files which are modified more than 30 days back and less than 60 days:
find / -mtime +30 –mtime -60

Find Changed Files in Last 1 Hour:
find / -cmin -60

To find all 30MB files:
 find / -size 30M

To find all the files which are greater than 30MB and less than 100MB(30MB – 100MB):
find / -size +30M -size -100M

To find all 50MB files and delete them using one single command:
find / -size +50M -exec rm -rf {} \;

Find all .log files with more than 10MB and delete them using one single command:
find / -type f -name *.log -size +10M -exec rm {} \;

Search specific directory or path:
find ./nijam
./nijam
./nijam/call.txt
./nijam/subdir

Find largest and smallest files:
The following command will display the 10 largest file in the current directory and its subdirectory.
$ find . -type f -exec ls -s {} \; | sort -n -r | head -10

Similary when sorted in ascending order, it would show the smallest files first
$ find . -type f -exec ls -s {} \; | sort -n | head -10

Limit depth of directory traversal:

  • For example we don't want to go more than 2 or 3 levels down in the sub directories. This is done using the maxdepth option.
  • The second example uses maxdepth of 1, which means it will not go lower than 1 level deep, either only in the current directory.

$ find ./test -maxdepth 2 -name *.php
./test/subdir/how.php
./test/cool.php

$ find ./test -maxdepth 1 -name *.php
./test/cool.php


To search for files that do no match a given name or pattern:
$ find ./test -not -name *.php
./test
./test/abc.txt
./test/subdir

To use multiple criterias when specifying name and inverting:
$ find ./test -name 'abc*' ! -name '*.php'
./test/abc.txt
./test/abc

To search inside 2 separate directories:
$ find ./test ./dir2 -type f -name abc*
./test/abc.txt
./dir2/abcdefg.txt

Find hidden files:
find ~ -type f -name ".*"

Comments

Popular posts from this blog

PostgreSQL pgBadger

PostgreSQL Pgbadger Installation On Linux

How to configure Replication Manager (repmgr) ?

What's is the difference between streaming replication Vs hot standby vs warm standby ?

Script to kill ALL IDLE Connection In postgreSQL