I use SSH as my main form of connection, even with connecting to other boxes (I just create TCP Tunnels to get to RDP/VNC services). The problem with password based authentication is that social engineering and brute force CAN break through.
I prefer host based authentication - meaning that the server I am connecting to will only allow the server I'm connecting from to login, using DSA or RSA keys for the authentication process.
For clarity, Source means the server/workstation you are connecting FROM, Destination means the server you are connecting TO.
To setup the whole thing, you need to generate a private/public keypair on your Source server, do this with ssh-keygen:
ssh-keygen -t dsa
Follow the prompts and you DONT have to fill in a password if you dont want to, but it adds a level of security. The following files will be created in $HOME/.ssh :
Cat the id_dsa.pub file, and copy the output, this is going to need to be placed on the Destination server. Login to your destination server as the user you normally connect as. Make sure there is a $HOME/.ssh directory and then create a file called "authorized_keys" (Case and spelling sensitive). Paste the previous output of the id_dsa.pub file into this file and save it.
Now you might need to edit your /etc/ssh/sshd_config file to allow host-based key authentication, make sure the following parameters are set:
PermitRootLogin yes (ONLY if you NEED root login, rather su to root)
RSAAuthentication yes (Allow RSA as well as DSA Keys)
PubkeyAuthentication yes (Allow Public key authentication)
AuthorizedKeysFile .ssh/authorized_keys (Location of the Server authorized keys)
Restart sshd, or HUP it (pkill -HUP sshd).
For added security, I disable PAM authentication in the /etc/ssh/sshd_config file:
This prevents ANY type of password authentication, meaning bruteforce attacks are impossible to conduct against your SSH Server.
Should you wish to allow tunnelling through your SSH Server, set these parameters in your /etc/ssh/sshd_config:
Again, restart SSHD.
Thats it - a pretty secure SSH system, which is usually the first point of attack for UNIX Hackers.