This issue drove me nuts today for several hours today, to be honest I was pretty close to breaking something! Earlier in the day I had the NFS mount working fine then I created an AMI and booted up another instance from the newly created AMI but in the new instance the mount kept failing. The error looks something like this:

[root@server]# mount -t nfs 192.168.2.1:/dbshare /mnt/dbshare
mount: 192.168.2.1:/dbshare failed, reason given by server: Permission denied

According to the error you would think that I have a configuration issue so I changed everything that I could think of within /etc/exports

My /etc/exports originally looked like this:

/dbshare 192.168.2.2(rw,sync) (where 192.168.2.2 is the client where I am performing the mount)

I changed it to something more open like this with no luck:

/dbshare 192.168.0.0/255.255.0.0(rw,sync)

I started looking around the logs on the server in /var/log/messages and found that it was authenticating fine
Jun 11 19:04:00 servername mountd[5222]: authenticated mount request from 192.168.2.2:736 for /dbshare (/dbshare)

I was really frustrated at this point and I had already spent an hour on Google looking for the answer. I found another answer but the website was down, luckily the cached version on Google came to the rescue.

The ANSWER:

The problem was that the special nfsd file system that mounts to /proc/fs/nfsd wasn't mounted. I'm not sure how it gets mounted (maybe rc.sysinit does it?), but I tool the advice from the forum entry and added an entry to /etc/fstab

none /proc/fs/nfsd nfsd auto,defaults 0 0

then ran mount -a

After this the mount worked fine. I hope that someone finds this helpful.

I often need to copy code from a SVN repository to a location where I have files deployed or I may pull code from an SVN repository on the web and want to check it in locally. Both of these actions require the stripping out of all .svn directories spread throughout the directory structure. On Linux or Mac this is pretty simple with the following command

sudo find -d . -name '.svn' -exec rm -rf '{}' \; -print

I ran across this today. Examples are great when trying to understand JQuery selectors.

In recent months I have been using PHP to extend functionality inside of WordPress for the SpatialKey site and working with JQuery as well. After working with ColdFusion for so long I have gotten spoiled by the CFDUMP tag . Here are a few options similar to CFDUMP for both PHP and Javascript.

CFDUMP for PHP - dBug

CFDUMP for Javascript

One challenge I have encountered with Amazon Ec2 is the sending of email from our web applications. If you try to send directly from sendmail or postfix then you might as well forget about guranteed delivery. A large amount of your email will end up in spam folders if it is even delivered at all.

There are a few problems with delivering email from Ec2

• MX record will not map to your IP and you are using dynamic IPs (you can address this with elastic IP's) adding a SPF DNS record can help as well
• Reverse DNS will map back to Amazon and not back to your Hostname
• Many of the Ec2 IP's have been blacklisted due to abusers of the service sending spam.

There are a few solutions to this problem and I will propose two of them.

Using a google apps account:

If you are using a solution like Google Apps and have them host your email accounts then you can use gmail as your SMTP server. You will need to create an account donotreply@mydomain.com and use authentication in your applications to send the email. With Google apps you cannot override the "from" address when you send email it wil always become whatever you account you are sending from. For example if you create the account donotreply@mydomain.com and attempt to set the "from" in your code to send from brandon@mydomain.com google will override it and send from donotreply. The only option is to set "replyto" in your code and when a user replys they will send to your replyto account. With google apps you are limited to a maximum amount of 500 emails a day per account as well and if you are sending a lot of emails this can quickly become a problem. This is a great solution for small volumes of email and you delivery rates are very good.

Relay from localhost through a third party:

This blog post outlines a set of steps to relay through a local Postfix instance to a third party SMTP service. The great thing about this solution is that you can send email from your application to localhost without storing the authentication parameters in your applications code and have Postfix handle it all. If you have many applications sending email this can greatly simplify things. Also it allows your application to hand off the emails quickly to another service that can handle queuing in case the third party email service is down at any time. You could combine the approach above with this one but you would still have the 500 email limitation. I am searching for a good third party SMTP service that is reliable, the author of the blog post recommends AuthSMP. I have not tried them and their prices are not too high but not cheap either. I am going to do a little more digging and test some of the options and will report back to this posting.

This is common knowledge if you have been using Linux for a while but I still find it a helpful resource to understand how you set what programs are running when Linux starts. This is mainly specific to Red Hat or CentOS which I use on a regular basis.

Running level
Running level is the current running functional level of the operation system, from level 1 to 6, possessing different functions.
Here are the different running levels:

• 0 - downtime or halt
• 1 – single user mode
• 2 – multi-user，no NFS
• 3 – entire multi-user mode(standard running level)
• 4 – not available
• 5 - X11 multi-user graphic mode（xwindow)
• 6 - restart （never set initdefault 6 ）

These levels are specified in the file /etc/inittab., which is the main file that the program init looks for, and the first running service is placed under the directory /etc/rc.d. For most Linux releases, startup scripts are all located in /etc/rc.d/init.d, which are all linked to the directory /etc/rc.d/rcn.d by ln command (here the n is the running level 0-6).

Setting services/applications to run at startup using chkconfig

chkconfig command (under redhat and centos)

Linux provides the command chkconfig to update and query system services of different running levels allowing you to set when certain process are started.

Syntax:
chkconfig --list [name]
chkconfig --add name
chkconfig --del name
chkconfig [--level levels] name
chkconfig [--level levels] name

example:

[user@server]# chkconfig --list
anacron        	0:off	1:off	2:on	3:on	4:off	5:on	6:off
atd            	0:off	1:off	2:off	3:on	4:on	5:on	6:off
autofs         	0:off	1:off	2:off	3:on	4:off	5:on	6:off
conman         	0:off	1:off	2:off	3:off	4:off	5:off	6:off
crond          	0:off	1:off	2:on	3:on	4:on	5:on	6:off
dc_client      	0:off	1:off	2:off	3:off	4:off	5:off	6:off
dc_server      	0:off	1:off	2:off	3:off	4:off	5:off	6:off
getsshkey      	0:off	1:off	2:off	3:off	4:on	5:off	6:off
haldaemon      	0:off	1:off	2:off	3:on	4:on	5:on	6:off
haproxy        	0:off	1:off	2:off	3:on	4:on	5:on	6:off
httpd          	0:off	1:off	2:off	3:off	4:off	5:off	6:off
ibmasm         	0:off	1:off	2:off	3:off	4:off	5:off	6:off
ip6tables      	0:off	1:off	2:on	3:on	4:on	5:on	6:off
iptables       	0:off	1:off	2:on	3:on	4:on	5:on	6:off
jexec          	0:on	1:on	2:on	3:on	4:on	5:on	6:on

chkconfig has five functions: add service, delete service, list service, change startup info and check the start state of specified service.

Option overview:

--level levels
specify running level, which is a character string composed of the number 0 to 7. For example:
--level 35 means to specify running level 3 and 5.
To stop the service nfs during running level 3,4,5, use the command next: chkconfig --level 345 nfs off

--add name
This option adds a new service, chkconfig ensures every running level an entrance to start (S) or to kill (K). if it is absent, then it would auto establish from default init script.

--del name
To delete service and delete related sign connections from /etc/rc[0-6].d.

--list name
List, if name is specified, then it only displays specified service name, otherwise, to list the state of all service at different running levels.

Usage examples:

As an example if you wanted mysql to run when the os starts you just need to do the following

• /etc/init.d/mysqld must exist and needs to be an executable (chmod +x)
• Add mysql - chkconfig --add mysqld
• setting the start level - chkconfig --level 345 mysqld on

This example applies to any server and to validate that it worked you can use chkconfig --list | grep mysql to see the changes

I ran across this article while debugging some .htaccess issues on our SpatialKey server and found some helpful information on .htaccess.

This is a good article on some monitoring tools for Linux. I had already used several of them before but found 'htop' and 'atop' in the comments and they are great

I had been looking for a simple way to get a thread dump from JBoss to see what was happening on each of the SpatialKey servers without actually logging onto any of them. After all reading through thread dumps is one of my favorite past times. I found a simple way to do that writes an HTML file containing the thread dump and I can access it by hitting the webserver directory at a hidden URL.

The command is rather simple to generate a thread dump:

/jboss/bin/twiddle.sh invoke "jboss.system:type=ServerInfo" listThreadDump > /dump.html

I modified the /etc/crontab file to create this file every minute

*/1 * * * * root //jboss/bin/twiddle.sh invoke "jboss.system:type=ServerInfo" listThreadDump > /apachewebroot/subdir/dump.html 2>&1

It is pretty simple but it worked for what I needed.

I found a helpful command today to help me search through many different file in linux. I have a bunch of log files and want to find a certain occurrence of an error.

for i in `find ./`; do grep -H "string to search" $i; done

Excel is a powerfull tool for sorting and analyzing data but what if you want to look at your data from another angle. What if you want to see the geographic aspect and understand the trends both over time and geographically? Geocoding your data can be a challenging task and getting it on a map can be even more of a pain. SpatialKey takes the pain out of mapping your excel data and makes it simple to get your data into a map and build engaging reports.

I have outlined this process in an article explaining how to take your excel data and get it into SpatialKey

With SpatialKey we create AMI's that are exact replicas of each other and can be scaled easily all of the persistent content is stored in a EBS volume allowing us to deploy a new instance from a Snapshot and easy backup with Snapshots.

This article is the best one I have seen highlighting performance with EBS volumes. He also highlights using software based RAID and the performance numbers with RAID are impressive. I am loving Amazon Ec2 more and more each day.

We recently moved our website for SpatialKey onto Amazon Ec2 which has worked great but there has been one items that has bugged me. If I needed to make a few small changes to the site I was using vi to make the inline edits. For larger changes I have scripts to scp the files up to the instance but the workflow for changes has become a pain.

I stumbled upon a great tool called ExpanDrive that allows me to use use sftp, basically ssh to manage my files remotely but leverage my Mac tools to do the editing. ExpanDrive provides a 30 day free trial but it is worth the price. The package runs $39.95. You can find answers to nearly all of your support questions on getsatisfaction.com. Right after installing I ran into an issue and found the answer right away.

If you are using Ec2 then you will probably experience this issue as well. Since Ec2 doesn't use a username/password combo but instead uses a cert/keypair for authentication. You need to use ssh-add to add the keypair then you can use it in ExpanDrive.

To do this open up a terminal session and run the following command - "ssh-add /Users/myusername/myec2keys/id_my_keypair"

Next I set up ExpanDrive with an empty password and it logged right in and mounted the drive on my Mac. I could then open up any of my Mac editing tools to edit on the server.

It has been a long time in the works but we are finally in a private beta for SpatialKey. This version has many more features and capabilities beyond our technology preview that was released in August of 2008 and provide a slick new ability to upload your own data.

Instead of me rambling on about what it does and doesn't do just check out the video!

Apply for the private beta if you are intersted in visualizing your own data.