Archive for 九月, 2009


REHL: rsync setup & ssh keys

Have you ever wanted to know how to easily synchronize the data between multiple servers automatically?

rsync is the anwser. And here’s some note. We will use rsync, ssh key authentication, and a cron job.

Let’s call the 2 servers “SOURCESERVER" and “DESTSERVER" for:
SOURCESERVER = Source server (the server we’re connecting from to upload the data)
DESTSERVER = Destination server (the server we’re connecting to receive the data)

Part 1 – Setting up SSH key authentication
First, we need to make sure the DESTSERVER has the ability to use key authentication enabled. Find your sshd configuration file (usually /etc/ssh/sshd_config) and enable the following options if they are not already set.

RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys

If you edit the file be sure to restart sshd afterwards.

#service sshd restart

Next, on the SOURCESERVER we will create the public & private key pair to be used for authentication with the following command:

ssh-keygen -t dsa

*Note 1: Do not enter a passphrase for this, just hit enter when prompted.
*Note 2: if SOURCESERVER doesn’t have key yet, do the keygen as well.

This should create 2 files in ~/.ssh folder, a public key file (id_dsa) and a private key file (id_dsa.pub).

The private key file (~/.ssh/id_dsa) we will keep on the SOURCESERVER.

*Be sure to keep this private key safe. With it anyone will be able to connect to the DESTSERVER that contains the public key.

chmod 700 ~/.ssh
chmod -R 600 ~/.ssh/*

Now we will add the public key we created on to the DESTSERVER.
Choose the user account which you will use to connect to on DESTSERVER, we’ll call this user ‘destuser’ for now.

The public key file (~/.ssh/id_dsa.pub) we will upload to the DESTSERVER, and call it SOURCESERVER.pub.

scp ~/.ssh/id_dsa.pub DESTUSER@DESTSERVER:~/.ssh/SOURCESERVER.pub

On the DESTSERVER, in the DESTUSER’s home directory, in the .ssh folder, create a new text file called “authorized_keys".
touch ~/.ssh/authorized_keys

If it already exists, great, we will use the existing authorized_keys file to add the SOURCESERVER’s public key.

cat ~/.ssh/SOURCESERVER.pub >> ~/.ssh/authorized_keys

Be sure the permissions for key files are 600 and 700 for the ‘.ssh’ folder.

Now to test that the keys are working. From the SOURCESERVER try logging in as normal using ssh to the DESTSERVER.

# ssh destuser@DESTSERVER

If all is working you should not be prompted for a password and able to connected directly to a shell on the DESTSERVER.

Part 2 – Creating the rsync script
Now for the rsync script. I use a simple script such as the following

#!/bin/bash

SOURCEPATH=’/source/directory’
DESTPATH=’/destination’
DESTHOST=’123.123.123.123′
DESTUSER=’destuser’
LOGFILE=’rsync.log’

echo $’\n\n’ >> $LOGFILE
rsync -av –rsh=ssh $SOURCEPATH $DESTUSER@$DESTHOST:$DESTPATH 2>&1 >> $LOGFILE
echo “Completed at: `/bin/date`” >> $LOGFILE

Copy this file into the home directory of the sourceuser on the SOURCESERVER and modify the first 4 variables in the file.

SOURCEPATH (Source path to be synced)
DESTPATH (Destination path to be synced)
DESTHOST (Destination IP address or host name)
DESTUSER (User on the destination server)

Save it as something like ‘rsync.sh’

Set the permissions on the file to 700.
# chmod 700 rsync.sh

Now you should be able to run the script, have it connect to the DESTSERVER, and transfer the files all without your interaction.

The script will send all output to the ‘rsync.log’ file specified in the script.

Part 3 – Setting up the cron job

Assuming everything has worked so far all that’s left is to setup a cron job to run the script automatically at a predefined interval.

As the same sourceuser use the ‘crontab’ command to create a new cron job.

# crontab -e

This will open an editor where you can schedule the job. Enter the following to have the script run once every hour:

# Run my rsync script once every hour
0 * * * * /path/to/rsync.sh

Your 2 servers should now be syncing the chosen directory once every hour.

廣告

Compiling MySQL on Snow Leopard

Here’s my version based on HiveLogic’s.

Reference: http://hivelogic.com/articles/compiling-mysql-on-snow-leopard/

Set the PATH

vi ~/.profile

# export PATH="/usr/local/bin:/usr/local/sbin:/usr/local/mysql/bin:$PATH"

source ~/.profile

Download the source

1. Open the Terminal application from the Utilities folder which is nested in the Applications folder

2. In the newly opened terminal type:

defaults write com.apple.desktopservices DSDontWriteNetworkStores true

3. Restart the computer for the change to take effect

Reference: http://support.apple.com/kb/HT1629

The boot camp VM is broken because missing files. How do you redo it all?

It’s extremely annoying that one cannot remove the Boot Camp partition entry on the Virtual Machine Library without going through some hoops and jumps…

If you do the following this will stop the Boot Camp partition entry from showing on the Virtual Machine Library…

1. Close Fusion

2. In a Terminal (/Applications/Utilities/Terminal) copy and paste the following command, as is, and then press Enter and type in your password then press Enter.

sudo mv "/Library/Application Support/VMware Fusion/vmware-rawdiskCreator" "/Library/Application Support/VMware Fusion/vmware-rawdiskCreator.bak"

3. Open Fusion and then ctrl-click the Boot Camp partition entry on the Virtual Machine Library and select Delete.

Note: If you have Imported the Boot Camp partition as a normal file based Virtual Machine you could have a Boot Camp partition entry on the Virtual Machine Library that you do not want to delete so before deleting the Boot Camp partition entry on the Virtual Machine Library you can ctrl-click the Boot Camp partition entry on the Virtual Machine Library and select Show in Finder and then make sure this is the one in your “/Users/${USER}/Library/Application Support/VMware Fusion/Virtual Machines/Boot Camp" folder before removing it from the Virtual Machine Library.

To restore the Boot Camp partition entry on the Virtual Machine Library…

1. Close Fusion

2. In a Terminal (/Applications/Utilities/Terminal) copy and paste the following command, as is, and then press Enter and type in your password then press Enter.

sudo mv "/Library/Application Support/VMware Fusion/vmware-rawdiskCreator.bak" "/Library/Application Support/VMware Fusion/vmware-rawdiskCreator"

Reference: http://communities.vmware.com/thread/200649

In 10.5.x or earlier version of OSX, you use Directory Utility to enable root user and reset its password. In Snow Leopard, the utility is removed under the Application/Utility folder. So what do you do?

sudo passwd

Alternately, you can navigate to /System/Library/CoreServices/Directory Utility.app to do it in the old way.

Moved from windows to mac, the old “D drive" staff need a bit of re-org. One big trouble I found is that OSX doesn’t deal with file start with “dot" well. For example, the “.svn" folders.

So how to get rid of these? On any *nix machine (Mac included) you can run the following command:

rm -rf 'find . -type d -name .svn'

PHP 5.3 is not supported on current Drupal (6.14), and CentOS’s PHP version is still on 5.1.9. So the drupal installation needs a little bit of work.

1. Install MySQL 5.1
cd /etc/yum.repos.d
wget wget http://rpms.famillecollet.com/enterprise/remi-release-5.rpm
yum --enablerepo=remi install mysql mysql-server mysql-devel mysql-libs

2. Install PHP 5.2.9 from CentOS testing repository:
cd /etc/yum.repos.d
wget http://dev.centos.org/centos/5/CentOS-Testing.repo
yum --enablerepo=c5-testing install php php-devel php-pear php-pecl php-mcrypt php-xml php-xml php-imap php-soap php-mbstring php-mysql php-cli php-mysql

3. drupal clean URL config (sample)
there are many ways to turn on clean URL. Here’s how I do it:
AllowOverride All
Options Indexes FollowSymLinks MultiViews
Order allow,deny
Allow from all

RewriteEngine on
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php?q=$1 [L,QSA]
Alias /drupal /var/www/drupal

4. PHP.ini
modify the following keys, this is my settings:

max_input_time = 100 ; Maximum amount of time each script may spend parsing request data
memory_limit = 100M ; Maximum amount of memory a script may consume (16MB)
error_reporting = E_ALL
memory_limit = 100M ; Maximum amount of memory a script may consume (16MB)
upload_max_filesize = 100M

5. upload progress

add the extension to php.ini:
if you don’t have gcc installed:
yum -y groupinstall "Development Tools"

install pecl related if you don’t have them yet:

yum -y install php-devel php-pecl php-pear

install the upload progress:

pecl install uploadprogress

edit your /etc/php.ini and add this line under extension section:

extension=uploadprogress.so

6. GD library

yum --enablerepo=c5-testing install gd gd-devel php-gd

6. Install drupal

PHP 5.3 is not supported on current Drupal (6.14), and CentOS’s PHP version is still on 5.1.9. So the drupal installation needs a little bit of work.

1. Install MySQL

wget http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-3.noarch.rpm
rpm -ivh epel-release-5-3.noarch.rpm

wget http://rpms.famillecollet.com/enterprise/remi-release-5.rpm
rpm -ivh remi-release-5.rpm

yum --enablerepo=remi install mysql mysql-server

2. Install PHP 5.2.9 from CentOS testing repository:

cd /etc/yum.repos.d
wget http://dev.centos.org/centos/5/CentOS-Testing.repo
yum --enablerepo=c5-testing install php php-devel php-pear php-pecl php-mcrypt php-xml php-xml php-imap php-soap php-mbstring php-mysql php-cli php-mysql

3. drupal clean URL config (sample)
there are many ways to turn on clean URL. Here’s how I do it:

AllowOverride All
Options Indexes FollowSymLinks MultiViews
Order allow,deny
Allow from all

RewriteEngine on
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php?q=$1 [L,QSA]

Alias /drupal /var/www/drupal

4. PHP.ini

5. GD library

yum --enablerepo=c5-testing install gd gd-devel php-gd

6. Install drupal