To install Nginx from source

./configure --prefix=/u/pandev/opt/nginx --without-http_rewrite_module

netstat + lsof = my friend

linux

1. "netstat -anpt | grep -i listen | grep 8888" will find out the process id is running on that port.
2. "/usr/sbin/lsof -p " find out what file and sockets it opens.
3. "lsof -P -n -i TCP" will list all the TCP sockets with unresolved (numeric) IP and port

Database connection pooling with firewall troubleshooting tip: Use Netcat

Database connection pooling is useful and common feature for performance. However there are some cavets.
Sometimes your sql query just got timed out, and you wonder why.
If you suspect the TCP connection between your application hosted in DMZ to the database (which is usually sits behind a firewall) got dropped by the firewall. It is very likely firewall has a policy to terminate any idle connection.

Let's go back to the basics to use Netcat.

Server(internal machine):
nc -v -l 9999

Client (DMZ machine):
nc 9999

Assuming port 9999 is allowd on the firewall.

Let's assume the firewall drops the connection after 30 minutes idle time, then the nc will be broken after 30 minutes.

Remote disk space monitor

The following is a poor man's disk space monitor script.
"ssh id@host df -h " is the core of the script. It relies on SSH RSA authentication so that the remote "df" can work like local.


#!/bin/ksh
homedir=`dirname $0`
RUN_DATE_LONG=$(date +'%Y%m%d')
LOGFILE=${homedir}/diskmon_log.$RUN_DATE_LONG

HOSTS="192.168.1.1 192.168.1.2 192.168.1.3 192.168.1.4 "

alert=90
alert_email="admin@foo.com"
typeset -i email=0

function rdf
{
echo "$1 is sshing into $2 doing a df......." >> $LOGFILE
ssh $1@$2 df -hP | grep -vE '^Filesystem' | awk '{print $4 " " $5 " " $1 " " $2}' | while read line;
do
typeset -i used=$(echo $line | awk '{print $1}' | cut -d "%" -f 1)
if [[ $used -ge $alert ]]; then
email=1
echo $line >> $LOGFILE
fi
done
echo "ssh done......." >> $LOGFILE
}



for h in $HOSTS
do
rdf sshid $h
done


if [[ ${email} -eq 1 ]]; then
echo "alert email will be setnt" >> $LOGFILE
# put whatever mailer you use here
fi

Dtrace could be the secret weapon

I had this vision of setting up a high end consulting firm which only serves some hard problem such as performance. One of secret weapon is Dtrace.

DTrace for linux: http://www.crisp.demon.co.uk/tools.html

DTrace with Python: http://blogs.sun.com/levon/entry/python_and_dtrace_in_build

The power of rsync with hardlink

rsync -aHvz -e ssh /var/src /var/src-hardlinks
remote_id@192.168.1.1:/home/users/foo/

The above would copy the directory "src" and "src-hardlinks" (which holds all the hardlinks) recursively to 1921.68.1.1's folder.

find and cpio to back up hardlinks and archive

It can be used for backup with cpio to maintain the hardlinks, permission and modified timestamp.

find d1 d2 -type f -mtime +${period} | cpio -Bpdmv backup_dir


Archive the data to a single .cpio file.

http://bradthemad.org/tech/notes/cpio_directory.php