Raam Dev’s Weblog

Yahoo! appears to be inaccessible to people in the US. Visiting yahoo.com redirects to www.yahoo.com and fails to load. I confirmed it was at least somewhat limited to the US by trying the connection from a shell account on a server in Europe.

Using dig (a Unix DNS lookup utility), we can see from within the United States that there is a problem with DNS. There is no A record with an IP address listed in the ANSWERS section:

;; QUESTION SECTION:
;www.yahoo.com. IN A

;; ANSWER SECTION:
www.yahoo.com. 129 IN CNAME www.wa1.b.yahoo.com.

And from the server in Europe:

;; QUESTION SECTION:
;www.yahoo.com. IN A

;; ANSWER SECTION:
www.yahoo.com. 272 IN CNAME www.wa1.b.yahoo.com.
www.wa1.b.yahoo.com. 33 IN CNAME www-real.wa1.b.yahoo.com.
www-real.wa1.b.yahoo.com. 33 IN A 209.191.93.52

;; AUTHORITY SECTION:
wa1.b.yahoo.com. 273 IN NS yf2.yahoo.com.
wa1.b.yahoo.com. 273 IN NS yf1.yahoo.com.

If you try connecting directly to the missing IP address, you should at least be able to get the main Yahoo page: http://209.191.93.52. You might also try temporarily adding an entry to your /etc/hosts or C:\Windows\system32\drivers\etc\hosts if you want to continue being able to use the FQDN.

UPDATE: As of 15:50 EST, Yahoo appears to be working again. The outage appeared to start around 15:11 EST, so that’s a good 40 minutes of downtime.

Find all files in this directory, including the files in sub-directories, and exclude all files that start with a period (dot files) and any directories named .thumbs. Then pass the list of results to the wc command to get a total count:

find . ! -name ".*" ! -path "*.thumbs*" -type f | wc -l

When I decided to reformat and install my Mac Mini with the latest testing version of Debian (lenny, at the time of this writing) I discovered that I couldn’t mount my HFS+ OS X backup drive with write access:

erin:/# mount -t hfsplus /dev/sda /osx-backup
[ 630.769804] hfs: write access to a journaled filesystem is not supported, use the force option at your own risk, mounting read-only.

This warning puzzled me because I was able to mount fine before the reinstall and, since the external drive is to be used as the bootable backup for my MBP, anything with “at your own risk” was unacceptable.

I had already erased my previous Linux installation so I had no way of checking what might have previously given me write access to the HFS+ drive. A quick apt-cache search hfs revealed a bunch of packages related to the HFS filesystem. I installed the two that looked relevant to what I was trying to do:

hfsplus - Tools to access HFS+ formatted volumes
hfsutils - Tools for reading and writing Macintosh volumes

No dice. I still couldn’t get write access without that warning. I tried loading the hfsplus module and then adding it to /etc/modules to see if that would make a difference. As I expected, it didn’t. I was almost ready to give up but there was another HFS package in the list that, even though it seemed unrelated to what was trying to do, seemed worth a shot:

hfsprogs - mkfs and fsck for HFS and HFS+ file systems

It worked! I have no idea how or why (and I’m not interested enough to figure it out), but after installing the hfsprogs package I was able to mount my HFS+ partition with write access.

I have been using Linux for several years now and although I have looked at the load averages from time to time (either using top or uptime), I never really understood what they meant. All I knew was that the three different numbers stood for averages over three different time spans (1, 5, and 15 minutes) and that under normal operation the numbers should stay under 1.00 (which I now know is only true for single-core CPUs).

Earlier this week at work I needed to figure out why a box was running slow. I was put in charge of determining the cause, whether it be excessive heat, low system resources, or something else. Here’s what I saw for load averages when I ran the top command on the box:

load average: 2.86, 3.00, 2.89

I knew that looked high, but I had no idea how to explain what “normal” was and why. I quickly realized that I needed a better understanding of what I was looking at before I could confidently explain what was going on. A quick Google search turned up this very detailed article about Linux load averages, including a look at some of the C functions that actually do the calculations (this was particularly interesting to me because I’m currently learning C).

To keep this post shorter than the aforementioned article, I’ll simply quote the two sentences that gave me a clear-as-day explanation of how to read Linux load averages:

The point of perfect utilization, meaning that the CPUs are always busy and, yet, no process ever waits for one, is the average matching the number of CPUs. If there are four CPUs on a machine and the reported one-minute load average is 4.00, the machine has been utilizing its processors perfectly for the last 60 seconds.

The machine I was checking at work was a single-core Celeron machine. This meant with a continuous load of almost 3.00 the CPU was being stressed much higher than it should be. Theoretically, a dual-core machine would drop this load to around 1.50 and a quad-core would drop it to 0.75.

There is a lot more behind truly understanding the Linux load averages, but the most important thing to understand is that they do not represent CPU usage. Rather they represent the load on the CPU by processes waiting for their chance to use the CPU. If you still can’t get your brain away from thinking in terms of percentages, consider 1.00 to be 100% load for single-core CPU’s, 2.00 to be 100% load for dual-core CPUs, and so on.

I’ve had plans for a while now to set up a backup system using a Debian Linux server and rsync to back up my MacBook Pro laptop. At first glance, it seemed like it would be pretty straight forward. I’ve been able to make a bootable copy of my entire MBP using nothing but rsync (thanks to some very helpful directions by Mike Bombich, the creator of the popular, and free, Carbon Copy Cloner software). And by bootable copy I mean I could literally plug in the USB drive and boot my MBP from the drive (hold down the Alt/Option key while booting). Restoring a backup is as simple as running the rsync command again, but in the reverse direction. I know this solution works because I used it when I upgraded to a 320GB hard drive.

To start, I needed to create a big enough partition on the external USB drive using Disk Utility (formatted with Mac OS Extended (Journaled)). I then made a bootable copy of my MBP with one rsync command:

sudo rsync -aNHAXx --protect-args --fileflags --force-change \
--rsync-path="/usr/local/bin/rsync" / /Volumes/OSXBackup

But my dream backup system was more unattended. I wanted something that would periodically (a couple times a day) run that rsync command over SSH (in the background) and magically keep an up-to-date bootable copy of my MBP on a remote server.

I love Linux and I jump at any opportunity to use it for something new, especially in a heterogeneous network environment. So when I decided to set up a backup server, I naturally wanted to make use my existing Debian Linux machine (which just so happens to be running on an older G4 Mac Mini).

So, after making a bootable copy of my MBP using the local method mentioned above, I plugged the drive into my Linux machine, created a mount point (/osx-backup), and added an entry to /etc/fstab to make sure it was mounted on boot (note the filesystem type is hfsplus):

/dev/sda /osx-backup hfsplus rw,user,auto 0 0

All that’s left to do now is to run the same rsync command as earlier but this time specifying the remote path in the destination (root@myserver.example.com:/osx-backup/). This causes rsync to tunnel through SSH and run the sync. Unfortunately, this is where things started to fall apart.

OS X uses certain file metadata which must be copied for the backup to be complete (again, we’re talking about a true bootable copy that looks no different than the original). Several of the flags used in the rsync command above are required to maintain this metadata and unfortunately Linux doesn’t support all the necessary system calls to set this data. In particular, here are the necessary flags that don’t work when rsyncing an OS X partition to Linux:

-X (rsync: rsync_xal_set: lsetxattr() failed: Operation not supported (95))
-A (recv_acl_access: value out of range: 8000)
–fileflags (on remote machine: –fileflags: unknown option)
–force-change (on remote machine: –force-change: unknown option)
-N (on remote machine: -svlHogDtNpXrxe.iL: unknown option)

According to the man page for rsync on my MBP, the -N flag is used to preserve create times (crtimes) and the --fileflags option requires chflags system call. When I compiled the newer rsync 3.0.3 on my MBP, I had to apply two patches to the source that were relevant to preserving Mac OS X metadata:

patch -p1 <patches/fileflags.diff
patch -p1 <patches/crtimes.diff

I thought that maybe if I downloaded the source to my Linux server, applied those same patches, and then recompiled rsync, that it would be able to use those options. Unfortunately, those patches require system-level function calls (such as chflags) that simply don’t exist in Linux (the patched source wouldn’t even compile).

So I tried removing all unsupported flags even though I knew lots of OS X metadata would be lost. After the sync finished, I tried booting from the backup drive to see if everything worked. It booted into OS X, but when I logged into my account lots of configuration was gone and several things didn’t work. My Dock and Desktop were both reset and accessing my Documents directory gave me a “permission denied” error. Obviously that metadata is necessary for a viable bootable backup.

So, where to from here? Well, I obviously cannot use Linux to create a bootable backup of my OS X machine using rsync. I read of other possibilities (like mounting my Linux drive as an NFS share on the Mac and then using rsync on the Mac to sync to the NFS share) but they seemed like a lot more work than I was looking for. I liked the rsync solution because it could easily be tunneled over SSH (secure) and it was simple (one command). I can still use the rsync solution, but the backup server will need to be OS X. I’ll be setting that up soon, so look for another post with those details.

Sender verification is an important feature used by email servers to help prevent spam. When sender verification is enabled, the receiving email server checks to make sure the sender exists. Various email servers have different ways of handling this feature. Exim, for example, uses a mechanism called ’sender callouts’ or ‘callbacks’. (When the sending server does not accept a verification request, it does not comply with RFC 2821.)

However, in the event that the network route from the receiving email server to the originating email server is broken (or a firewall blocks the connection), the result can be a bit confusing. The receiving email server treats a failed verification as a failed verification, regardless of whether or not it could even connect to the originating server. This means the email never comes through to the recipient. After all, as far as the email server knows, it’s spam.

One of my hosting clients was experiencing this “lost email” problem and a quick grep at /var/log/exim_mainlog confirmed the problem (hosts and IPs changed for obvious reasons):


2008-11-17 15:02:27 [30121] H=relay1.example.com (qsv-spam1.example.com) [67.26.151.59]:36752 I=[69.161.211.25]:25 sender verify defer for : could not connect to customer.example.com [163.112.75.15]: Connection timed out
2008-11-17 15:02:27 [30121] H=relay1.example.com (qsv-spam1.example.com) [67.26.151.59]:36752 I=[69.161.211.25]:25 F=<administrator@customer.example.com> temporarily rejected RCPT <raam@mydomain.com>: Could not complete sender verify callout
2008-11-17 15:02:27 [30120] H=relay1.example.com (qsv-spam1.example.com) [67.26.151.59]:36751 I=[69.161.211.25]:25 incomplete transaction (RSET) from <administrator@customer.example.com>


As you can see, the email server was unable to connect to customer.example.com to verify the existence of the sender (administrator@customer.example.com). This doesn’t mean the sending server doesn’t verify callbacks, but rather that the network connection from my server to the sending server could not be established.

Most of the stuff I found online related to solving this problem on a server running WHM (here and here) explain how to modify exim.conf to add special whitelist rules. Luckily, my server is running WHM 11.23.2 and has a whitelist option that makes it really easy to exclude a particular IP address from sender verification without any manual changes to exim.conf:

1. Click Service Configuration -> Exim Configuration Editor
2. Under Access Lists, find “Whitelist: Bypass all SMTP time recipient/sender/spam/relay checks” and click [EDIT]
3. Add the IP address for the sending server for which you wish to skip sender verification (as the note at the bottom explains, hosts cannot be used in this list)
4. Click Save
5. Click Save again near the bottom of the Exim Configuration Editor page

That’s it! Now any emails from that IP that were failing to come through because of a sender verification failure will come through without a problem (again, you can watch /var/log/exim_mainlog to confirm).

In C programming, things as simple as variable assignment are not quite as simple as using an assignment operator—they sometimes require entire functions. For example, this code will not even compile:

#include        <stdio.h>
#include        <strings.h>

int main()
{
        char    a[10], b[10];

        a = "hello";
        b = "world!";

        printf("%s %s", a, b);

        return 0;
}


$ cc test.c
test.c: In function ‘main’:
test.c:8: error: incompatible types in assignment
test.c:9: error: incompatible types in assignment

In C all strings are arrays. To create a string variable, you must create an array. The variable “a” is actually a pointer to the memory location of the character array, not the contents of the array itself! That’s why I got the “incompatible types in assignment” error when I tried compiling the above code—I was trying to copy a string directly into a memory address!

The reason things are this way in C is for speed and simplicity. Sure, other languages automatically do the work of putting your five-character string into a variable and automatically allocate the necessary space in memory, but by doing that they spend a little more time behind the scenes—time and speed that may be precious to a systems-level programmer (who might be writing a program for, say, a tiny embedded device).

To copy a string into an array (i.e., assign a string to a variable), you can use the strcpy() function. This function does the work of taking each character in your string and putting it into the correct place in the given array:

#include        <stdio.h>
#include        <strings.h>

int main()
{
        char    a[10], b[10];

        strcpy(a, "hello");
        strcpy(b, "world!");

        printf("%s %s", a, b);

        return 0;
}


$ cc test.c
$ ./a.out
hello world!

C was written in a time when assembly language was the norm. The problem with assembly language was that it was very tied to the hardware you were working on. Porting your work to other hardware, even if the changes in the hardware were only minor, required an entire rewrite of your code! Operating systems were also written in assembly at the time so creating a single operating system that worked on many different architectures was nearly impossible (unless you had an unlimited amount of time and money to have programmers constantly rewriting the operating system for every new hardware architecture that was released).

So the C programming language was created as a language one level higher than assembly. It was designed to maintain all the power and flexibility of assembly, while making it very easy to port to multiple architectures. This was made possible by using a compiler. The compiler simply took the C code and converted it into the necessary machine language for a specific architecture. If you wanted to port all your C code to a new architecture, all you needed to do was write a new compiler—not rewrite all your programs!

C lets you do stupid things not because it’s stupid, but because flexibility and closeness to the physical hardware is necessary for writing operating systems. (As the programmer, it’s your job to make sure what you’re doing is possible with the hardware you’re working on.) Where as other high-level languages will automatically take your string and stick it in the correct place in memory, C does only what you tell it to do. This makes it extremely fast, which is very important when you’re writing an operating system.

The basic example of how a string cannot be assigned directly to the character variable because the variable is actually a pointer to a memory address, helped me realize why C is still used for systems-level programming and why it continues to be in use more than 35 years after its invention. I have flipped through many C books but never quite gotten this explanation of how C works. Understanding things at this level really helps me put the language in perspective.

To rsync a file or directory that contains spaces, you must escape both the remote shell and the local shell. I tried doing one or the other and it never worked. Now I know that I need to do both!

So lets say I’m trying to rsync a remote directory with my local machine and the remote directory contains a space (oh so unfortunately common with Windows files). Here’s what the command should look like:

rsync 'raam@example.com:/path/with\ spaces/' /local/path/

The single quotes are used to escape the space for my local shell and the backslash is used to escape the remote shell.

When one of my users reported problems deleting files he had uploaded using a PHP script, I quickly discovered all the files being uploaded were owned by the user running the web server: nobody. This meant only the root user could delete those files.

Apache suEXEC is commonly used to resolve this problem. It allows Apache to run as the user who owns the domain being accessed. This way, files created by PHP would be owned by the user owning the site instead of the default nobody user.

However, Apache suEXEC only works if you’re using CGI as the PHP handler. The PHP5 handler on my server was set to use CGI, but I have PHP4 configured as the default PHP version and it was configured to use DSO. When I tried changing PHP4 to use CGI as the handler, most of the domains on my server displayed this:

Warning: Unexpected character in input: ‘’ (ASCII=15) state=1 in /usr/local/cpanel/cgi-sys/php4 on line 772
Warning: Unexpected character in input: ‘ in /usr/local/cpanel/cgi-sys/php4 on line 772
Warning: Unexpected character in input: ‘ in /usr/local/cpanel/cgi-sys/php4 on line 772
Warning: Unexpected character in input: ‘ in /usr/local/cpanel/cgi-sys/php4 on line 772
Parse error: syntax error, unexpected T_STRING in /usr/local/cpanel/cgi-sys/php4 on line 772

OK, that looks like a problem with cPanel. I don’t have time to debug cPanel’s problems.

suPHP, like suEXEC, is used to run Apache as the user who owns the domain. I decided to try recompiling Apache and PHP with suPHP enabled to see if that would fix the problem.

File Ownership Hell

suPHP worked, except now the sites using PHP sessions were trying to access stored session data in /tmp/ that was owned by the user nobody! So I deleted all the session data and that allowed the PHP sites to create new session data with file ownership of the user owning the domain.

But then I tried accessing my WordPress admin page and started getting permission denied errors in /wp-content/cache/. Same problem: the cache files that had been created before I enabled suPHP were owned by the user nobody and now the user who owns my domain couldn’t access them. A quick chown -R raamdev:raamdev /wp-content/cache/ fixed that problem.

Yeah, I could simply chown -R [user]:[user] /home/[user] for each of the users on the server, but there’s something about running a recursive command on files I’ve never seen, and know nothing about, that makes me uncomfortable.

More suPHP Limitations

I was beginning to worry that this was going to be more difficult than simply enabling suPHP and I wondered how many other sites I’m hosting could have similar problems. I tried accessing one of the high priority sites I’m hosting and discovered it was broken and displaying an “Internal Server Error”.

After a little research, I discovered that you cannot use php_value directives in .htaccess files with suPHP. The .htaccess file included with (created by?) Joomla! contained this at the bottom:

#Fix Register Globals
php_flag register_globals off

I already knew register_globals was turned off in the global PHP configuration, so I simply commented out that line and the site started working again.

Conclusion

It was at this point that I concluded it was too risky to just blindly enable suPHP while hosting over 50 domains, many of which I am not at all familiar with what’s being used or hosted. I will need to take the time to carefully crawl through all the sites making sure their .htaccess files don’t contain anything that might disrupt suPHP and then confirm all the sites are still working properly.

Lesson learned: Setup suPHP before you’re hosting 50+ domains.

Dialog is a really useful utility for creating professional looking dialog boxes and menus within a shell script. I’m working on a boot-time script that allows the user to make system-level changes before the system has fully booted.

When testing my script from the command line, the dialog menu looked fine. However, whenever I set the script to start during boot (update-rc.d myscript.sh defaults, on Debian-based systems) here is what the menu looked like:

UGH! It was barely usable. At first, I thought this would be an endlessly difficult problem to solve given my limited in-depth knowledge of Linux (I’m getting there!), but then I realized the main difference between the script running during boot and the script running after I had logged in was that my environment variables had not been loaded.

From the command line, I ran the env command to display all my current environment variables:

debian_vm:~# env
TERM=xterm-color
SHELL=/bin/bash
SSH_CLIENT=172.16.168.1 61315 22
SSH_TTY=/dev/pts/0
USER=root
MAIL=/var/mail/root
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11
PWD=/root
LANG=en_US.UTF-8
PS1=\h:\w\$
SHLVL=1
HOME=/root
LANGUAGE=en_US:en_GB:en
LOGNAME=root
SSH_CONNECTION=172.16.168.1 61315 172.16.168.132 22
_=/usr/bin/env

The three variables that caught my eye were TERM, SHELL, and LANG. After a little trial and error, I discovered setting the LANG variable fixed the display issues with dialog! I added the following near the top of my script:

export LANG=en_US.UTF-8

Now when my script loads during boot, everything looks correct:

A few months ago I wrote a post about creating Amazon S3 HMAC Signatures without PEAR or PHP5. One of the things I was using that PHP script for was to feed the necessary information to a bash script hosted on a remote machine. The bash script was to upload a file via POST to Amazon S3 using the information provided.

Since CURL was already installed on the remote machine, I wanted to use that to do the actual uploading. I found very little help on the net regarding how to do this with CURL so here you go:

eris:~ raam$ curl \
-F "key=screenshots/current_screenshot.jpg" \
-F "acl=public-read" \
-F "AWSAccessKeyId=2EO6H8MX1X8YWEA0V432" \
-F "Policy=eyAiZXhwaXshdGlvbpI6ICIyMDA4LTErLTAxVDtyOjAwOjAwLjAsMFoiLAogICJjb25kaXRpb25zPjogWwoJeyJidWNrZXQiOiAiczNwaG90b3MubW9hcHAubmV0IiB9LAogICAgWyJzdGFydHMtd2l0aCIsICIka2V5IiwgIkxpdmVTaG90cy8iXSwKICAgIHsiYWNsIjogInB1YmxpYy1yZWFkIiB9LAoJWyJlcSIsICIkQ29udGVudC1UeXBlIiwgImltYWdlL2pwZWciXSwKICBdCn0K" \
-F "Signature=20uh08kU75ADHL49NyhYRgZW8BY=" \
-F "Content-Type=image/jpeg" \
-F "file=@current_screenshot.jpg" \
http://screenshots.ekarma.net

Keep in mind this assumes the current_screenshot.jpg file is in your current directory.

Here are my notes from The Last HOPE. I started taking notes late, so unfortunately I don’t have notes from all the talks I attended.

Ghetto IDS and Honeypots
* An Evening with Berferd
* Low interaction honeypots: Nepenthes, honeyd, Honeytrap
* Monitor both Honeyd and Nepenthes with Prelude IDS

Monitoring Snort
* SGUIL
* BASE
* SnortSnarf

Remember, tcpdump (a common packet sniffer) writes data in pcap format which ngrep, WireShark, or Snort can process.

Kevin Mitnick - Featured Speaker
* Flowroute + Asterisks can be used to unmask Caller ID (I tested Mitnick’s setup by calling his phone… my blocked number showed up!)

PenTest Labs Using LiveCDs by Thomas Wilhelm
* de-ice.net
* BackTrack, Slax

PenTesting from Firefox URLs:
* isecom.org/osstmm/
* owasp.org/index.php/Main_Page/
* csrc.nist.gov/publications/PubsSPs.html
* vulnerabilityassessment.co.uk/Penetration Test.html
* centralops.net
* nmap-online.com
* hackerwhacker.com (similar to GRC)

Remember, use TOR when doing active tests!

More useful URLs:
* gdataonline.com/seekhash.php
* passcracking.com
* hash.insidepro.com
* md5this.com
* gdataonline.com
* us.md5.crysm.net
* md5.rednoize.com
* milw0rm.com
* freerainbowtables.com
* netcraft.com

Pen Testing the Web with Firefox

Firefox Extensions:
* FireCat
* ExploitMe (XSS-Me, SQL Inject-Me, Access-Me)
* Tamper Data
* Passive Recon
* Add N Edit Cookies
* Firebug
* HackBar
* Web Developer
* xssed.com

Using Firefox as a Front-End: Proxies
* Tor Button
* Paros Proxy
* SPIKE Proxy
* Burp Proxy

Web Frontends
* Metasploit
* FastTrack
* Inprotect (web interface for Nessus and Nmap)
* BASE (web front-end for Snort)

Use Firefox profile manager to install different selections of extensions to help with memory concerns.

FEBE (Firefox Environment Backup Extension)
CLEO (Compact Library Extension Organizer)
OPIE (Import/Export extension preferences)

Places/Things to hack “safely”
* OWASP WebGoat Project
* PwnOS (VMWare image, requires forum login)
* Your own VMWare lab

Identification Card Security: Past, Present, Future

The Complete Amature - ID Making Operating Guide by Doug Farre

* Epson Stylus R800 photo printer
* Laminator
* Dye cutter
* Magnetic stripe encoder
* Custom rubber stamp (simonstamp.com)
* Black light
* Scanner
* Signature pad
* Photoshop
* brainstormidsupply.com

Minimal needed materials:
* Laser Teslin
* Laminates
* Pearl-Ex pigment powders
* Ultraviolet pigment powder
* Transparent base

(Get these from practicingperfection.7p.com. That site is down as of right now, so you need the guy’s email address to contact him.)

Documentation on ID security can be found at idsysgroup.com.

Books to Read
* 1491: New Revelations of the Americas Before Columbus
* Hackers: Heroes of the Computer Revolution
* The Art of Intrusion: The Real Stories Behind the Exploits of Hackers, Intruders & Deceivers

Random URLs:
* foodhacking.com
* hackerspaces.org
* telephreak.org

When the LogWatch report from yesterday (for web.akmai.net) arrived in my Inbox, it had over 20,000 failed SSH login attempts. Today I decided it was finally time to do something about all those attacks.

After looking around a bit, I found several different solutions. Some solutions utilized firewall rules and others monitored your /var/log/secure (or /var/log/auth.log) log files for multiple failed login attempts and then added those IPs/Hosts to the /etc/hosts.deny file.

I decided to go with the latter method and quickly found a nice tutorial for setting up DenyHosts (be sure to download the latest version (2.6 as of this writing) instead of the older version 2.0). Rather than reinvent the wheel, here is what the DenyHosts website says about itself:

What is DenyHosts?

DenyHosts is a Python script that analyzes the sshd server log messages to determine what hosts are attempting to hack into your system. It also determines what user accounts are being targeted. It keeps track of the frequency of attempts from each host.

Additionally, upon discovering a repeated attack host, the /etc/hosts.deny file is updated to prevent future break-in attempts from that host.

An email report can be sent to a system admin.

Since I was setting up DenyHosts on a RedHat-based machine (CentOS) and not a Debian-based machine, I needed to change this line:

update-rc.d denyhosts defaults

to this:

chkconfig denyhosts --add

Other than that, the installation steps were just as the tutorial described. I decided to enable the ADMIN_EMAIL option so that I would receive an email every time something was added to hosts.deny, but within minutes of starting DenyHosts I had a dozen attacks with a dozen emails on my BlackBerry. I had to disable ADMIN_EMAIL to stop the spamming!

To make sure DenyHosts was working properly I tried logging in with the wrong password three times. When I tried to connect again, here is what I received:

ssh root@akmai.net
ssh_exchange_identification: Connection closed by remote host

DenyHosts also has the ability to report to a central server the hosts that are trying to break in and you can also download a list of hosts that have been reported by others. I choose to opt out of doing this for now. The DenyHosts statistics page is pretty cool. Notice how the majority of the hosts come from China? Hmm.

UPDATE:
I quickly discovered that DenyHosts was adding my IP address to the hosts.deny file. When I watched /var/log/secure I discovered the problem:

Jun 13 20:18:46 web sshd[5959]: reverse mapping checking getaddrinfo for 75-147-49-211-newengland.hfc.comcastbusiness.net failed - POSSIBLE BREAKIN ATTEMPT!
Jun 13 20:18:46 web sshd[5959]: Accepted publickey for fooUser from ::ffff:75.147.49.211 port 57926 ssh2
Jun 13 20:18:48 web sshd[5994]: Did not receive identification string from ::ffff:75.147.49.211

I’m not entirely sure how to fix this, but for now I added my IP address to /usr/share/denyhosts/data/allowed-hosts (I had to create this file) which prevents DenyHosts from blocking my IP no matter what (see this FAQ for more info). Also, I had to restart DenyHosts (/etc/init.d/denyhosts restart) before the change to allowed-hosts took effect.

I’ve been doing some stuff at work using Amazon S3 to store files and during my testing I uploaded a ton of files that didn’t need to be there. Unfortunately, the command line tool I’m using, s3cmd, does not allow me to delete multiple files at once. There is no way to do a wild-card delete. This means I would need to get the full path to each object and delete them one by one:


./s3cmd del s3://s3.ekarma.net/img/1205794432gosD.jpg
Object s3://s3.ekarma.net/img/1205794432gosD.jpg deleted
./s3cmd del s3://s3.ekarma.net/img/1205794432g34fjd.jpg
Object s3://s3.ekarma.net/img/1205794432g34fjd.jpg deleted


Yea, there’s no way I’m doing that for over 200 objects. I mean come on, there are tools to automate this kind of stuff! So I created s3delmany.sh:

#!/bin/sh
# -------------------------
# s3delmany.sh
# Author: Raam Dev
#
# Accepts a list of S3 objects, strips everything
# except the column containing the objects,
# and runs the delete command on each object.
# -------------------------

# Redirect output to the screen
2>&1

# If not using s3cmd, change this to the delete command
DELCMD="./s3cmd del"

# If not using s3cmd, change $4 to match the column number
# that contains the full URL to the file.
# This basically strips the rest of the junk out so
# we end up with a list of S3 objects.
DLIST=`awk 'BEGIN { print "" } { print $4, "\t"} END { print ""}'`

# Now that we have a list of objects,
# we can delete each one by running the delete command.
for i in "$DLIST"; do $DELCMD $i
done

Download
s3delmany.zip

Installation
1. Extract s3delmany.zip (you can put it wherever, but I put it in the same directory as s3cmd).
2. Edit it with a text editor and make sure DELCMD is set correctly. If you’re not using s3cmd, change it to match the delete object command for that tool.
3. Make it executable: chmod 755 s3delmany.sh

Usage
If you’re using s3cmd and you placed s3delmany.sh in the /s3cmd/ directory, you should be able to use the script without modifying it. The script works by taking a list of objects and running the delete command on each one.

To pass s3delmany.sh a list of objects, you can run a command like this:

./s3cmd ls s3://s3.ekarma.net/img/ | ./s3delmany.sh

This will delete all objects under /img/. Make sure you know the output of your s3cmd ls command before you pass it to s3delmany.sh! There is no prompt asking if you’re sure you want to delete the list, so get it right the first time!

Hint: s3cmd doesn’t allow you do use wild-cards, but when you run the ls command, you can specify the beginning of an object name and it will only return objects starting with that. For example, s3cmd ls s3://s3.ekarma.net/img/DSC_, will return only those objects that begin with DSC_.

Alternate Usage
If you have a text file containing a list of S3 objects that you want to delete, you can simply change print $4 to print $1 and then do something like this:

cat list.txt | ./s3delmany.sh

By the way, print $4 simply tells s3delmany.sh that the S3 objects are in the 4th column of the data passed to it. The ./s3cmd ls command outputs a list and the object names are in the 4th column. The awk command expects the columns to be separated by tabs (\t).

If you have any questions or comments, please don’t hesitate to use the comment form below!