Raj Shekhar's Homestead

An analysis of the X- headers in my spam

Introduction

RFC 822 allows people to define and use additional header fields in their emails. These fields start with X- in their names. For example, X-Quarantine-ID or X-Spam-Score. Usually these headers are not too interesting. I wanted to see if the spammers use any custom fields in the spam messages that they send out.

Analysing spam

I took a spam message and checked its X- headers. The following are the X-headers that I think can be ignored

• X-MimeOLE
• X-Spam-Status (added by spamassassin)
• X-Account-Key, X-Mozilla-Status2 and X-Mozilla-Status are added by Thunderbird
• X-Mailer
• X-Original-To
• X-Quarantine-ID

Any headers outside of the above list promises to be interesting.

Summary: X-headers in my spam

I ran a script on my spam mbox file (i.e. mails that have been marked as spam by spamassassin) to see what I can dig up. The following X-headers look interesting, because they occur so few times. (caveat: I don't think I have enough spam)

X-DSNContext

This seems to be header set by MS Exchange

X-AntiVirus: Checked,

This seems to be header set be spammers to confuse the mail filter

X-ME-UUID

Looks like header set by some mail server

X-Source, X-Source-Args, X-Source-Dir, X-Spam

These are headers set by phplist

X-Failed-Recipients

This is a header generated by exim when it handles mailing lists. Looks like some spammer has a mailing list with all his targeted email ids in it.

Summary: X-headers in non-spam email

I ran my script on a mbox which had emails from one of the mailing lists I am on. Some interesting headers

X-AOL-IP

Self explanatory

X-BigFish

Ummm —- no clue

X-Disclaimer

Hehe

X-Ninja-AttachmentFiltering, X-Ninja-PIM

Seems like a rarely used mail client

Script used in analysing the headers

This is a Perl script that uses regular expression to find the interesting X-headers. You can get the script from here. The interesting parts are explained below.

The isinterestingheader function filters out non interesting headers.

sub isinterestingheader($)
  {
    my $header = shift;
    my @ignore_x_headers = qw/Original-To Spam-Status Mozilla-.*  Quarantine-ID Virus-Scanned
			     Spam-Score Spam-Level Spam-Flag Amavis-.* Mailer Priority Account-Key
			     Priority /;
    my $regex_ignore_x_headers = join( "|", @ignore_x_headers );

    unless ( $header =~ /X-($regex_ignore_x_headers)/i) {
      #print "[$header]  is interesting and matched $1\n";
      return 1;
    }
    return 0;

  }

If you want to ignore/unignore any particular X-headers, you can add/remove it to the @ignore_x_headers variable.

The loop below is the main workhorse. It iterates over the mbox file and if it finds a X-header, it extracts that, checks if the header is interesting and if the header is interesting, stuffs it into the %xheaders hash.

while (<INFILE>) {
    my $mline = $_;

    if ( $mline =~ /(^X-.*).*:\s(.+)/ ) {
      if (isinterestingheader($1)){
	countthisheader($1,$2);

      }
    }
  }

Raw data

Spam

Result of running the script on my spam folder (2704 mails). Another note - UIDL is Unique IDentification Listing, hence the number X-UIDL in the results below gives a count of the number of emails in the mbox file.

I ran the script on the mbox file that had been marked as spam by Thunderbird (979 mail). The results are

Non Spam