It’s often not possible or wanted to take down your rails application for updates. Even if it is, I still regard downtime as something that should be avoided if possible.
When using the absolutely fantastic HAProxy to load-balance between your mongrels, you can take down a mongrel process without any adverse effects. When a mongrel process is not answering anymore, HAProxy just forwards the request to the next available mongrel and no harm is done.
So all you need is a way to stop and restart each mongrel, one after the other. I cooked up a bash script to do this. It’s not perfect, but I’ve been using it on several projects. The script stops each mongrel, waits until it’s really stopped, then restarts and checks if it’s really running.
Why a bash script ? I wanted to sharpen my (limited) bash scripting skills (so if you have any tips: feel free to comment).
Customize where needed. (properly indented version here)
If you are running monit, remember to ‘unmonitor’ before and ‘monitor’ after.
#!/usr/bin/env bash
#rails project home dir
#!!Change acc to your needs
cd /home/rails/le_test/current
#Ports mongrels are running on
#!!Change acc to your needs
for PORT in 8000 8001 8002
do
#the location of our pid-files
#!!Change acc to your needs
PF=/var/run/mongrel/mongrel.$PORT.pid
#pidfile exists ?
if [ -e $PF ]; then
PR=$(cat $PF)
#process still running ?
if ps $PR > /dev/null 2>&1; then
echo "$PF $PR Running - killing"
kill $PR
while ps $PR > /dev/null 2>&1; do
echo "$PR still running"
sleep 1
done
echo "$PR killed"
else
echo "$PF $PR Not Running - deleting pidfile"
rm -f $PF
fi
else
echo "$PF does not exist"
fi
#check if a command containing $PORT.pid is still running
#in case process is still active but pidfile was not found
if ps ax|grep $PORT.[p]id > /dev/null 2>&1; then
echo "!!! But process containing $PORT.pid running !!!"
echo "!!! NOT auto-restarting, Manual intervention necessary !!!"
else
echo "Restarting port $PORT"
#mongrel start cmd
#!!Change acc to your needs
mongrel_rails start -d -e production -p $PORT -P $PF
sleep 1
#check if process is actually running now (could also be done by pidfile based check)
while ! ps ax|grep $PORT.[p]id > /dev/null 2>&1; do
echo "Not Running"
done
fi
done
This should work with other load-balancers too, but since HAProxy had all features I expected to find and is very resource friendly and has a tiny footprint (and irons a shirt in 2 mins), I didn’t look any further (after looking at Pound, which only advantage is that it’s easy to install/configure and cisco css switches which are just a tiny bit expensive for my needs)