|
Junk email tests
The current configuration uses a series of tests to gauge whether
email has come from a known spam source or whether email headers
have been modified or are incomplete - sure signs that mail is
likely to be spam.
Weighting
We test all email against 14 specific tests each test is assigned
a weight depending its perceived risk. When an email fails a test
it's weight is increased by the weighting of the test it failed, an
email that passes all tests has a weighting of 0.
For instance an email that fails the MAILFROM test it receives a
weighting of 12.
If an email fails one or more tests a 'X-RBL-Warning' header is
added to the email header along with the weight, this is not visible
in the email itself but can be viewed by right clicking the email
and selecting Options (Outlook 2000).
The header 'X-RBL-Warning' can take one of four values based upon
the emails weighting:
| Weight |
Headers added |
Email Subject |
| 0-4 |
X-RBL-Warning: SPAM-NONE: Total weight between 0 and 4.
X-Note: SPAM TEST failed:(SPAM-NONE) |
none |
| 5-9 |
X-RBL-Warning: SPAM-VLOW: Total weight between 5 and 9.
X-Note: SPAM TEST failed:(SPAM-VLOW) |
none |
| 10-14 |
X-RBL-Warning: SPAM-LOW: Total weight between 10 and 14.
X-Note: SPAM tests failed:(SPAM-LOW) |
none |
| 15-19 |
X-Note: SPAM tests failed:(SPAM-MID) |
SPAM-MID |
| 20-29 |
X-Note: SPAM tests failed:(SPAM-HIGH) |
SPAM-HIGH |
| 30+ |
X-Note: SPAM tests failed:(SPAM-VHIGH) |
SPAM-VHIGH |
Example headers (incomplete) from an email that has been filtered
X-RBL-Warning: DSN: Not supporting null originator (DSN)
X-RBL-Warning: SPAM-VLOW: Total weight between 5 and 9.
X-Declude-Sender: dbd@btconnect.com [193.113.209.26]
X-Note: This E-mail was scanned by Declude JunkMail for evidence of spam.
X-Note: SPAM tests failed:[DSN, SPAM-VLOW]
X-Spam-Weight: 0.
|
The last line may be
of interest, X-Spam-Weight: 0., your could set you own filter on
weight alone. We automatically hold on a weight of 30 but if you
don't want this restriction you can have it lifted, simply send an
email to our Support team stating you'd like the HOLD test for your
domain removed.
Action options
|
Test |
Action Example
(see below for examples) |
| SPAM-NONE |
WARN |
| SPAM-VLOW |
WARN |
| SPAM-LOW |
WARN |
| SPAM-MID |
SUBJECT SPAM-MID |
| SPAM-HIGH |
SUBJECT SPAM-HIGH |
| SPAM-VHIGH |
SUBJECT SPAM-VHIGH |
Actions
IGNORE does nothing
WARN will add a warning to the headers of the message
FOOTER will add text to the end of the E-mail
HEADER will add text to the beginning of the E-mail
SUBJECT will add text to the beginning of the subject
ALERT will send a "bounce" message as well as deliver the E-mail
HOLD will hold the message (nearly always deleted by sys admin)
BOUNCE will send a standard "bounce" message (and not deliver the
E-mail)
DELETE will delete the E-mail
How weights are applied:
|
Test |
Type |
Weight applied |
| ORBZIN |
ip4r |
5 |
| ORBZOUT |
ip4r |
5 |
| ORDB |
ip4r |
5 |
| OSDUL |
ip4r |
5 |
| OSFORM |
ip4r |
5 |
| OSLIST |
ip4r |
5 |
| OSRELAY |
ip4r |
5 |
| OSSMART |
ip4r |
5 |
| OSSOFT |
ip4r |
5 |
| OSSRC |
ip4r |
5 |
| SPAMCOP |
ip4r |
10 |
| MONKEYPROXIES |
ip4r |
6 |
| MONKEYFORMMAI |
ip4r |
3 |
| DSN |
rhsbl |
5 |
| NOABUSE |
rhsbl |
5 |
| NOPOSTMASTER |
rhsbl |
5 |
| BADHEADERS |
see below |
8 |
| MAILFROM |
see below |
12 |
| PERCENT |
see below |
10 |
| REVDNS |
see below |
5 |
| ROUTING |
see below |
5 |
| SPAMHEADERS |
see below |
5 |
| HEUR10 |
see below |
8 |
| BASE64 |
see below |
18 |
| SNIFFER |
see below |
15 |
It is quite common for valid email to fail a number of tests
either due to the email software not fully complying with RFC's or
because your domain or your ISP's domain doesn't have RFC compliant
DNS records and mail accounts.
For instance not having a mail address
abuse@domain for instance is a
violation of the RFC's and would fail the NOABUSE test and be given
a weighting of 5. If you see these failures in your own email it's
worth having a word with your ISP.
Note: RFC - Request For Comments, a series of documents
detailing technical and organisational notes about the Internet (orginally
the ARPANET), beginning in 1969.
X Headers
The full range of X headers added to email that can be filtered
using email clients such as Outlook 2000 are:
|
Test Name |
Description |
| MAILFROM |
This test checks the SMTP envelope "Mail From:" address
(which should be the sender of the E-mail) and makes sure that
the domain name it is coming from is valid. |
| BADHEADERS |
This test checks the E-mail for illegal headers that are
common in spam, but not common in legitimate E-mail. This test
can catch about 50% of all spam, with the only false positives
being mail that comes from broken mail clients. |
| SPAMHEADERS |
This test checks the E-mail for headers that are common in
spam, but not common in legitimate E-mail. This test is very
similar to the BADHEADERS test, except the problems this test
looks for are not RFC violations, so there's a chance you could
catch a small amount of legitimate E-mail. |
| ROUTING |
This test will analyze the route that an E-mail takes, and
look for highly inefficient routing that is very common in spam.
For example, an E-mail might get caught if it is sent from a
dialup in the U.K. to another account in the U.K., but is routed
through a server in China, but not if it goes from a mail server
in China directly to a U.K. mail server. Not a reliable test
outside the U.S. |
| REVDNS |
This test will check to see if a mail server has a reverse
DNS entry. If not, it will fail this test. All Internet hosts
are required to have a reverse DNS entry, although most do not.
Most mail servers do have the required reverse DNS entry, but
there are huge numbers that do not, so it is likely that this
test will catch a lot of legitimate mail. A warning in the
headers might be appropriate for this test. |
| PERCENT |
This test will catch all mail with "To:" addresses that
contain a percent sign. The percent sign indicates an outdated
routing method that can be used by spammers to bypass closed
relays. |
| BASE64 |
This test will catch E-mail that uses MIME "base64" encoding
for text or HTML segments. Using base64 encoding in these
segments is becoming common in spam, as it allows spammers to
bypass most filtering systems. However, there is no advantage
for legitimate mail to be sent this way (worse, it ends up
causing the size of the E-mail to be greater). Very few
legitimate E-mails will be caught by this test |
| SNIFFER |
Sniffer is a software utility and a service which provides
advanced message content filtering. The utility uses advanced
pattern recognition technologies to simultaneously apply
thousands of heuristic algorithms during a single pass through
an email message. A weigh of 15 is applied if an email fails the
SNIFFER test. This test alone can detect 92% of spam. |
(courtesy
www.declude.com)
Please note: These weights are likely to change as we
develop this system, to keep abreast of developments please revisit
this page regularly.
Technology We used
Declude software to provide the
technology behind our spam tests.
Declude is a recognised world
authority in both spam and virus email detection and in our opinion
is second to none.
MessageSniffer provides our message filtering test.
Last updated:
Tuesday July 29, 2003 |
