The Failure of Spam Filtering
Spam filtering and blocking isn't working, in fact spam has increased in the last two years, flooding the global network. Email and Internet users are demanding solutions but the technology market is slow to respond to consumer need.
1. The Economic Idiocy of Spam Filtering
2. Anti-Spam Software "Doesn't Work"
3. Anti-Spam Industry Recommendations Contradict Themselves, Offer no Hope
4. PEW Internet Research
5. 8 Reasons Why Content Blocking Does Not Work
6. The Pushdown Network
7. The Infinite Monkey Theorem
8. Spam that isn't email
Not only does filtering not work, but it makes no money sense. If we accept the overwhelming evidence that 90% or more of Internet traffic is junk, then the criminals have clearly hijacked the global network. What is the Internet? It is a collection private networks, commercial cable and public phone systems. Who pays for the maintenance of this network? We all do. Through taxes, access fees and overhead passed to the consumer. So the consumer is more or less supporting the spam network. How much does that end up being? In the United States it could be as high as $1.5 Billion per month or $18 Billion per year. This figure does not include the amount of money spent on filtering, or the lost work hours, or money spent on chasing e-crooks, only the estimated cost of transmitting the spam.
Based on the average household paying $30 per month for access, even you have a virus scan and filtering software and get no spam in your inbox, you are still paying $27 per month to guarantee that it gets delivered just short of your mailbox. By the way, since the spammers are hijacking machines with malware, their costs are zero.
The estimate is based on 55,544,208 households with net access (an outdated 2000 census) with only 10% of paid fees or taxes going to support traffic that is wanted. The estimate is possibly lower than the true cost (which is difficult to truly quantify), and this is only the United States. The global cost is probably much higher.
Most people are not happy with their anti-spam products, according to a new survey.
From McAfee and Symantec to Apple and Microsoft, most anti-spam vendors are failing to fully satisfy customers, according to the survey by Brockmann & Company.
...
customers rarely are fully satisfied by anti-spam filters packaged with email clients, hosted email or commercial anti-virus software. Too often, the products let spam through and mistakenly delete email that's not spam.
(techworld.com)
Anti-spam vendors such as McAfee, Symantec and Microsoft, are failing to satisfy their customers in fighting annoying spam. "Email is consistently recognized as the most important communications service affecting job performance in virtually all industries and all job roles. Yet spam continues to detract from user productivity by providing too many inappropriate, anonymous, bulk and irrelevant messages.
(sda-india.com)
Brockmann & Co. Study Results (brockmann.com)
In addition to the Brockmann & Co. Study Results, we have a more
detailed commentary from McAfee to analyze. The article
Are you back from vacation? Spam awaits, McAfee offers simple tips to help e-mail users fight back (spokanejournal.com)
is full of contradictions and bad advice. For starters, the title implies that these are "simple tips", but there
is nothing simple about completely reconfiguring your own filtering software after analyzing all the junk email you receive.
In the first paragraph we read:
|
McAfee Inc., the Santa Clara, Calif.-based security software vendor, has released tips to help e-mail users unsubscribe and block unwanted e-mail.
|
But the article later states:
|
do not unsubscribe from the list. By unsubscribing, you show the spammer that your e-mail address is active.
|
Why is this a "release" from McAfee? There is nothing new here. So we can take unsubscribing off the table, which leaves us with blocking.
|
“While today’s spam filters have improved to catch a larger percentage of junk mail, the threat of spam never really goes away,” says Jeff Green, senior vice president of McAfee Avert Labs.
|
It seems here he is admitting the failure of the block and filter approach. Don't unsubscribe and blocking has failed, what is the point of this article now?
These two items:
|
Don’t publish your e-mail address on any Web site or discussion forum.
|
|
If your mailbox starts receiving an abundance of spam, you can delete the mailbox
|
have been pointed out by Knujon many times as surrender, actions that only inconvenience the consumer and do not address the
core issues. The final two recommendations seem to contradict each other:
|
Use anti-spam software, which blocks 97 percent to 99 percent of spam.
|
|
Keep a collection of mail you want to stop receiving and determine which e-mail addresses or phrases in the messages don’t change. Use this information to keep further unwanted e-mail out of your inbox by creating filters.
|
What? Which am I supposed to do, buy your software(which he has admitted doesn't solve the problem) or become so involved in the filtering process that it makes me wonder why I bought filtering software?
Tinkering around with all these packages is great for programmers, but useless to the consumer at large. The consumer who falls for scams, buys junk, and responds to spam.
At only one point does he mention reporting it to your ISP, which is a problem in and of itself. As Knujon has pointed
out many times ISP response to spam is inconsistent, unprofessional, and ineffective. Some ISPs are helping the spammers.
Nowhere does McAfee mention that obvious criminal spam or stock fraud should be reported to law enforcement.
At the Spam Summit 2007 held by the FTC in Washington, D.C.
The results of a spam survey conducted by PEW/INTERNET(Pew Research)
were presented by Susannah Fox. We found the results of this
survey troubling since they suggest the public's acceptance of spam is growing. People are just assuming
that spam is a part of modern life and nothing can be done to stop it. As many of you are aware, at KnujOn, we think this
is nonsense and we have been working to deal with the problem in creative and unique ways. It is important to review how the Internet industry has dealt with this
problem from the beginning.
- When spam started to become a problem for email/Internet users it was generally assumed
that the user/consumer had done something to bring it on themselves: they purchased pornography,
signed-up for questionable websites, etc(i.e., it's the user's own fault).
- As spam began to reach people who had never purchased pornography the blame shifted to
posting and sharing of email addresses. Users were told not to public post their email addresses
and be careful who they share them with. Also, user mailboxes became infected. Users with unprotected
email programs turned into relays for viruses and address harvesting. Again, the consumer/user is blamed.
- Once it was realized that spammers get addresses from a variety of methods, including scripts that generate random or
sequential strings, the consumer was told to ignore or delete the spam they receive. Many concerned citizens
tried to report phishing attempts to their banks, but the banks told them to delete and ignore. Once again,
the burden is on the consumer to deal with it.
- The problem grew and a new industry of email filtering and blocking software emerged. However,
the responsibility is still on the consumer to purchase, maintain, update and upgrade the filtering software.
While the algorithms behind these filtering programs are complex, the scheme itself is litter more than
an enhanced tool for ignoring and deleting.
- A year after the widespread deployment of filtering software, spam is still a growth industry.
Armies of botnets(zombie PCs) are collections of computers on the Internet that power spam
delivery. These zombie PCs are private computers infected with malware and left connected to the
Internet usually without the owner knowing. The plague of botnets is again viewed as an end user/consumer
problem since it is the inept public downloading viruses and leaving their connections open that drives
spam.
- Now, the mantra being delivered by these survey results and some other recent media is:
"There's nothing we can do, accept it, suck it up."
So, after all these years, hopefully the message is clear: It's your fault and there is nothing
we can do about it.
- Does not actually reduce the flow of junk mail
Every study, statistic, news article and expert in the last 3 years has clearly stated that
the junk email problem is getting worse(and much worse) since the common institution
of filters and content blockers at the ISP, network, and user levels.
- Junk mail still gets through
Every week a new, clever piece of spam bypasses filters and hits users inboxes.
- Good mail gets blocked
The other day an email sent to me by a friend was blocked and marked as pornographic.
The person who sent the email was somewhat conservative so I was a little confused.
After getting the email released I realized why it was flagged as porn. The subject line
read “My phone numbers” and the email message contained the phrase “I’ll be having fun with
the girls.” The email was from someone who was going on vacation with her infant daughters,
there was nothing illicit about the situation. Yet, taken out of context “phone numbers” and
“fun with the girls” can give one the wrong impression like a conversation heard on Three’s
Company.
The following are statements made by on-line companies:
|
IMPORTANT NOTICE: Some Internet Service Providers may block replies, assuming they are unwanted messages.
To ensure that you receive a response to your inquiry, we recommend that you add carecenter@cits.canon.com to
your address book. This will also allow you to receive valuable information from Canon such as product updates
and special information about Canon products, supplies and accessories.
|
|
Internet Service Providers (ISP) have tightened their definitions of SPAM. As a result, your ISP might
categorize an email confirmation from this site as potential spam and filter it into a "Bulk" folder or a
predetermined "SPAM" folder you define. If you place an order and do not receive your email confirmation in
your Inbox, please check in these areas before contacting customer support.
|
They are basically saying "we cannot guarantee that our email will be delivered, you have to make an extra effort to dig
it out." Where does it end? Read about The Infinite Monkey Theorem below.
- Legitimate Marketing and Corporate Communication Treated as Spam
Lawful, legitimate companies have a diminished ability to reach potential and current customers. Since it is assumed
that all marketing email is illegal junk mail, it is all ignored and deleted. This includes newsletters
and catalogues that customers have subscribed to.
- Filtering does not stop the crimes behind the email
This isn't just about the email. There is a world of fraud behind these emails that needs to be addressed
aggressively.
- Anti-Spam Companies as Censors
“Viagra” and “Valium” are blocked by most filters. What if I work at a pharmacy and I want to
send email about these products? What if I want emails with adult content? What if I send email
that contains blocked content but I’m not actually selling anything? There has been quite a bit
of debate about censorship in the press, television, radio, music, film, and video games but very
little concern about the power given to companies that filter email. Privacy alarms sound over
credit card databases and telephone records but we thoughtlessly hand over our email to be filtered.
I am not implying that email filtering companies are spying on us or that we should all together
stop using filters. However, we should examine our use and dependence on them. ISPs have
in fact been sued over this issue. See:
Verizon faces lawsuit over email blocking (theregister.co.uk).
- Reduces the Value of Email as a Communication Tool
No links, no HTML, no pictures, no spread sheets, no compressed files, no documents. All hold potential spam and viruses therefore
none can be trusted. Sad as email is a perfect universal communication tool.
- Creates an Underground Network for Scam Artists
While most of us are blocking and deleting junk email, there are those who are exposed to on-line predators and no one is
watching. Read about the Pushdown Network below.
Organizations and Personal Email users are blocking/filtering millions of junk emails every day.
This is to the advantage of spammers as it allows them to target the most vulnerable users who do not have filtering software or technical savvy.
Beneath the protected networks is a wide-open "pushdown" network full of potential victims waiting to be scammed. It's called "pushdown" because
we have created it by pushing down the junk by blocking, filtering, and deleting it.
This is another reason why content blocking alone will
not solve the problem. While you and I may be protected, those without protection are allowing their PCs
to turn into zombie PCs and bringing infected files onto office networks. These people may end up being victims
of fraud or identity theft and this effects all of us.
The word “Viagra” is typically blocked by email filters. As is \/iagra, v|agra, vi4gra,
via6ra, viagr@, etc… and you get the picture. Each time the spammers change the spelling
of a keyword, the filter programmers add that spelling to their libraries which increases
the probability that innocuous content may be flagged as junk.
“The Infinite Monkey Theorem”
basically states that a monkey could reproduce all the great works in our libraries by punching random
keys over a very long period of time. Because of the constant changes made by spammers to bypass filters
we may end up with just such a reporduction within a filtering library. Sounds extreme but spammers are now
adding random word patterns and excerpts from on-line news stories to their emails. You may have seen junk mail like this:
|
cowhide it's predispose someirresolvable butpeste onwe've inlarry becancelling orventricle andperky butmaledict oncolorado beannulling thechortle anddrool orparkinson notpillow ,adulthood butliven butbaltimorean maybenefice seekidney bevent inleviticus andeerie orbeast incoequal adescendent !alternate acorinth someroll butargot onforborne somezion it'snascent it'sendothelial ,ribonucleic mayegypt andtriceratops mayattic andcheese notsimilitude
|
Eventually every word in the dictionary will be suspect content. The only way to get email through is similar to the joke of someone having to strip naked in order to pass through a metal detector.
It should not be this way, we don't have to play their game.
Many people believe that spam can be stopped by "fixing email," but we like to point them to various examples of spam that has nothing to
do with email:
Search Stacking
Junk Faxes
Forum, blog and wiki spam
Social Engineering Site Abuse
Snail mail scams endure
Telephone harassment endures despite do-not-call
|
