Mac Lab Report

Fighting Spam with Bayesian Filtering

- 2003.02.27

My recent article on spam filtering led reader Lee Williams to send me this message.

A slight variant on your suggestion is based in AI, and it's called bayesian classification. Rather than having the user explain to the machine every keyword it wants to flag, the bayesian classifier simply assigns probabilities to each (word, phrase) based on what class you put the email in. Then, when it sees another email, it is able to use these probabilities to estimate the class it belongs in. It's actually a lot simpler than having the enduser manually create these filters.

Paul Graham discusses this in his "plan for spam".

The key element in both solutions is that they are completely customizable. The existence of individual filters would not necessarily eliminate 100% of spam to each person, but in the aggregate it would mean that each email reaches considerably fewer people, thereby hopefully making it at least somewhat less cost-effective.

That article is good reading. I think it would be something I would definitely try. An email client with such filtering would be a great hit, in my opinion. [Editor's note: Apple's Mail client in OS X 10.2 includes a Bayesian filter. I've been using it for over a month and am still training it. dk]

If we can write software that recognizes their messages, there is no way they can get around that.

- Paul Graham

I wonder if a combination of the traditional manual filtering and this would be even better. The manual filtering would allow you to set a rule for specific phrase or person (say your ex for example) which gives a 100% probability of being filtered.

One interesting point related to your statement about the email reaching fewer people - which he doesn't make either - is that if a user actually wanted to receive these emails - after all, someone must be responding to them - they'd like this filter, too, because they'd sort the spam they like into the "keep" folder.

Contrary to Graham's hypothesis, spam need not end. It would become highly targeted at those people most likely to respond to it. The rest of us would ignore it. If everyone had such filters, the things you're most interested in would be the only things that would get through.

Everyone would like this type of filtering. Haters of spam, responders of spam, and senders of spam.

Now for an even spookier thought - apply the same technology to a TiVo with voice recognition and filter your TV commercials the same way.

Jeff Adkins is a science teacher who isn't afraid to state his preferences in computing platforms. In his classroom he has everything from a beige All-in-One to a a G4 XServe, and they all work together nicely. He calls himself the "poster child for technology integration" in the classroom. He was the 2006 Outstanding Educator of the Year for the California Computer Using Educators (CUE) organization. He also maintains a site for astronomy teachers at www.AstronomyTeacher.com.

Recent Mac Lab Reports

Microsoft Word 2004 vs. iWork Pages 1.0 for writing a book, 2008.01.10. Microsoft Word is great for technical writing, powerful yet slow, while Pages lets you concentrate on just writing, making it great for novels.
iWeb a great tool for quickly creating an attractive website, 2007.09.11. Apple's iWeb software isn't just easy to use, it also integrates nicely with .mac and other programs in the iLife bundle.
Use your Bluetooth phone to control your Mac? Maybe, 2007.02.27. Salling Clicker software turns many Bluetooth phones into remote controls for Bluetooth-equipped Macs.
More in the Mac Lab Report index.

Links for the Day

Mac of the Day: Macintosh Portable, introduced 1989.09.20. The nearly 16 lb. behemoth was innovative but not a smashing success.
February 13 in LEM history: 01: Layoffs may hurt Mac market - 02: Unix for the Mac - Rage against the Macintosh - 03: Options to move data from PCs to Macs - 04: Low cost RAM for older 'Books - 06: Apple, IBM, and Intel - 07: Picking the right cheap computer, new or used - 08: I needed to find an older Mac
Support Low End Mac

Recent Deals

About LEM Support Usage Privacy Contact

Mac Lab Report copyright © 2000-08 by Jeff Adkins. Low End Mac is an independent publication and has not been authorized, sponsored, or otherwise approved by Apple Inc. Entire Low End Mac website copyright ©1997-2012 by Cobweb Publishing, Inc. unless otherwise noted. All rights reserved. Apple, the Apple logo, Macintosh, iPad iPhone, iMac, iPod, MacBook, MagSafe, Mac Pro, Apple TV, and AirPort are registered trademarks of Apple Inc. Additional company and product names may be trademarks or registered trademarks and are hereby acknowledged.
Advice is presented in good faith, but what works for one may not work for all. Please report errors to .
LINKS: We allow and encourage links to any public page as long as the linked page does not appear within a frame that prevents bookmarking it.
Email may be published at our discretion unless marked "not for publication"; email addresses will not be published without permission, and we will encrypt them in hopes of avoiding spammers. Letters may be edited for length, context, and to match house style.
PRIVACY: We don't collect personal information unless you explicitly provide it. For more details, see our Terms of Use.

Custom Search

Follow Low End Mac on Twitter
Join Low End Mac on Facebook

Low End Mac Reader Specials

TypeStyler 11 is now in the Mac App Store!! -- Special Introductory Price of $59.95!! -- To Buy From The Mac App Store Click Here Now!! Or buy direct from Strider Software.

Don't install Parallels to play poker online! Poker Mac will show you how to download and install a native Mac poker and Mac Casino applications in minutes.

Affiliates

Amazon.com
The Apple Store
The iTunes Store
PC Connection Express
GainSaver
Parallels Desktop for Mac
eBay

Mac Lab Report