Are privacy add-ons effective? Surprising results from our testing

There’s no shortage of browser add-ons for consumers who want to block tracking by data and marketing companies. However, based on our testing, there is a wide variation in the actual effectiveness of these tools.

Methodology

We separately tested four different Firefox add-ons, AdBlock Plus, Better Advertising’s Ghostery, Abine and our own TrackerBlock, by running them on a clean test machine at full blocking. In each case, after visiting all webpages linked from the top page of Google News, we looked at the browser cookie file to see which tracking companies from our Index had been able to write unique cookies on the machine. This sequence was also repeated without any add-on enabled.

We estimated effectiveness based on the number of unique tracking-company cookies allowed by each tool, relative to the number of such cookies present with no add-on installed. We assumed that any unique cookie could be used for tracking and should be counted, because companies do not specify which cookies are used for tracking purposes. We did not test versions of these add-ons for Internet Explorer, Chrome or other browsers.

Results

Here are the results:

Observations

• No add-on provided perfect blocking.
• Many people view AdBlock Plus as not only a streamlining tool, but also a privacy tool. It’s not clear that it delivers fully on that expectation.
• Ghostery’s approach of disabling “web bugs” versus cookie interactions appears to provide incomplete coverage. The companies slipping through included widely installed networks like DoubleClick, Right Media and Audience Science, perhaps compounding the privacy exposure.
• Abine’s combination of opt-out cookie retention and selective domain blocking appears to provide the least effective results when it comes to cookie blocking.

More detail on methodology and the raw cookie output is available here. Due to the dynamic nature of ad delivery, we expect results to vary from test to test and across a different set of pages and sites. However, in repeat testing, we observed consistency as to those tracking companies able to avoid blocking by each add-on.

Ghostery’s team reported results of their own run of this test, indicating a maximum of 19 cookies making it through, out of 232 in their own database. This may reflect a difference in the classification of tracking domains, among other factors.

We invite comment on this methodology and are happy to assist anyone who wants to repeat the test on their own machine. Please send any suggestions for other ways to measure effectiveness (we’re also looking at cookie reading, not just writing).

TrackerBlock: Simple, Effective, Transparent

This week we released TrackerBlock for Firefox, which is the easiest and most effective tool to control online tracking. For the first time, users have complete visibility and control over the tracking company universe, including detailed policy and oversight information hundreds of companies.

TrackerBlock also is the only privacy tool that supports the emerging self-regulatory framework for behavioral advertising. Users can choose between blocking all tracking companies or only those not committed to industry best practices and compliance reviews.

Moving Beyond Opt-Outs for Add-ons

TrackerBlock represents a big shift in approach from our first Firefox addon, which served to make opt-out cookies permanent in your browser. With TrackerBlock, users don’t have to rely on ad firms to respect their cookie preferences, or worry about keeping opt-out cookies in place. Instead, the add-on prevents companies from reading cookies by stripping cookie information out of headers that are sent by the browser to the ad server. It also blocks companies from writing new cookies. For all blocked companies, TrackerBlock removes Flash cookies from those domains at the end of each browsing session. (Once Firefox 4.0 is out, we will also provide removal of other local stored objects, such as html5.)

I couldn’t escape this conclusion: If you’re going to offer an add-on to control tracking, the user deserves the best possible assurance that their preferences will be honored. Opt-out cookies shift a burden of trust and verification to the consumer. Unfortunately, many ad companies continue to retain tracking ID’s on opted-out computers, because their tracking cookies are stored separately from their opt-out cookies (see prior post). The computer’s unique identifier continues to be transmitted with every ad.

Simple, Effective and Transparent

Click to Enlarge

Two important things about the TrackerBlock interface:

1. Consumers can customize as much as they want, but they also can have a few simple choices that are immediately effective.

2. As part of the interface, consumers for the first time can see a comprehensive set of tracking company information for hundreds of companies. For each company, we’re providing a summary of their policies in four different key categories (Anonymity, Sharing, Boundaries and Deletion), with indicators of those policies that, in our view, deserve more attention.

We did our best to avoid any compromise between completeness of information and ease of use. Now it’s up to users to tell us whether we’ve succeeded.

What’s Next?

We’re at work on improvements to the PrivacyChoice Bookmark (still a good idea for users on other browsers or who won’t use an add-on), and we’re testing a form of TrackerBlock for Internet Explorer, so stay tuned.