repeated failing passwords in linux security log (/var/log/secure)

Recently, I opened up the SSH port through my firewalls (and redirecting to my server) so I could check on the (http) server while on the road. The first week or two there was nothing different. But now, three or four weeks later, I see lots of this:

Mar 20 08:38:28 localhost sshd[21895]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=mail.queued.net  user=root
Mar 20 08:38:31 localhost sshd[21895]: Failed password for root from 207.210.101.209 port 2854 ssh2
Mar 20 15:38:31 localhost sshd[21896]: Received disconnect from 207.210.101.209: 11: Bye Bye
Mar 20 08:38:32 localhost unix_chkpwd[21900]: password check failed for user (root)
Mar 20 08:38:32 localhost sshd[21898]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=mail.queued.net  user=root
Mar 20 08:38:34 localhost sshd[21898]: Failed password for root from 207.210.101.209 port 3729 ssh2
Mar 20 15:38:35 localhost sshd[21899]: Received disconnect from 207.210.101.209: 11: Bye Bye
Mar 20 08:38:36 localhost unix_chkpwd[21903]: password check failed for user (root)
Mar 20 08:38:36 localhost sshd[21901]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=mail.queued.net  user=root
Mar 20 08:38:38 localhost sshd[21901]: Failed password for root from 207.210.101.209 port 4313 ssh2
Mar 20 15:38:38 localhost sshd[21902]: Received disconnect from 207.210.101.209: 11: Bye Bye
Mar 20 08:38:40 localhost unix_chkpwd[21906]: password check failed for user (root)
Mar 20 08:38:40 localhost sshd[21904]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=mail.queued.net  user=root
Mar 20 08:38:42 localhost sshd[21904]: Failed password for root from 207.210.101.209 port 4869 ssh2
Mar 20 15:38:43 localhost sshd[21905]: Received disconnect from 207.210.101.209: 11: Bye Bye
Mar 20 08:38:44 localhost unix_chkpwd[21909]: password check failed for user (root)
Mar 20 08:38:44 localhost sshd[21907]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=mail.queued.net  user=root
Mar 20 08:38:46 localhost sshd[21907]: Failed password for root from 207.210.101.209 port 2512 ssh2
Mar 20 15:38:47 localhost sshd[21908]: Received disconnect from 207.210.101.209: 11: Bye Bye
Mar 20 15:38:57 localhost sshd[21912]: Connection closed by 207.210.101.209

There are about 1100 lines of these for March 20th, zero for the 19th, and 800 or so for the 18th—all related to the same IP.

What does it mean? What should I do? Why isn't it chronological?

From serverfault wallyk

• These are script kiddie attempts to get root via ssh on your box. The best way i've found to deal with these is:

  1. change the ssh port to anything but 22 (say 32999) or another random high port number
  2. use fail2ban or similar script. After a few failed login attempts it bans the ip with iptables for a specific period of time.

  There will be newbies chiming in on how setting the port to a different number is security through obscurity. Yes and no. It doesn't make your box any more secure but it will drastically cut down on the number of scripted attempts to crack ssh.

  Edit:

  Other good things to do are, disable root logins just in case they did guess your root password or disable password authentication altogether and use key based authentication.

  Chris Lercher : Agreed on 99% of your answer - but isn't using a high port dangerous? Note that non-privileged users can start services on ports >= 1024. So if your SSH server dies, you may log into a user's service instead of your machine. (This could also happen, if someone hacks e.g. your Apache, even though it runs as www-data)

  Chris Lercher : [It should be noted, that it's less dangerous in this case, since SSH is encrypted, but I still have a bad feeling about it]

  wallyk : Altering the port seems reasonable to conserve bandwidth, but will I have trouble with lame `ssh` clients not being able to specify or work successfully? Root login is not permitted from outside, so I'm not too worried about them finding the only two account names which are enabled. And they have 12+ character passwords. Well—not words, but character strings.

  Warner : It is security through obscurity. Changing a port is not securing anything nor is it a best practice. Filtering incoming traffic and implementing a strong password policy would be a better solution.

  anonymouse : If you actually read my comment you'll notice i didn't say changing the port secures anything. It does however drastically reduce the number of scripted attempts at ssh. If you have 50,000 login attempts a month compared to zero it does make a difference. It's not the be all end all solution and that's why listed it as just one out of a number of things that could be done. Others being fail2ban which filters incoming traffic and key based authentication. It might not be a "best practice" in your book but it's a real world practice in mine and countless other admins.
  From anonymouse

• A technique called port knocking can be used that allows you to keep your ssh port closed until requested (externally) by you.

  wallyk : Very interesting... and tempting. Alas, I don't think my Motorola Wimax modem will make this easy. Having a daemon check the modem log will encounter all kinds of complexity having to go through a password-protected html interface. I wouldn't be surprised if there's a solution for that, though the slowly changing passwords might be an issue.

  Dennis Williamson : You might consider leaving the port in the modem open and mapped only to the private IP of the server, but have it closed on the server and do the port knocking there using the server's logs. Or something to that effect.
  From Dennis Williamson

• Above answers explain why you're getting many authentication attempts to your SSH server.

  As to "Why isn't it chronological?", it looks like you may have a problem with your time zone setting. All the logs seem to be sequential, just at different hours (the minutes and seconds line up).

  Try running:

  sudo dpkg-reconfigure tzdata

  wallyk : I don't have any other issues with timekeeping. All the other logs show strict monotonic increasing timestamps. It's almost like a ssh connection opens and then closes 7 hours later, and then all of that session is logged at once. (Could that be?)
  From Jonathan Clarke