2014 07 22: µBlock and others: Blocking ads, trackers, malwares
Hard data, not hype.
Latest benchmark: 22 July 2014 (raw data spreadsheet).
This benchmark is to measure privacy exposure, by counting the number of distinct 3rd-party domains which have been hit by net requests during the benchmark. The lower the number of distinct 3rd-party domains hit, the better.
Some benchmarks measure the amount of requests blocked, which I think is of no interest as a useful measurement of privacy exposure. The number of requests blocked is no guarantee of less distinct 3rd-party domains being hit (and leaving a trace in the servers' logs).
Measuring directly the number of distinct 3rd-party domains which were hit is a much better and relevant measurement for comparison of privacy protection efficiency in my opinion.
Caveat: "3rd-party" is defined as a domain which doesn't match the domain of the web page. For sure many
domains reported as "3rd-party" actually belong to the same entity which owns the page domain (for example, yimg.com is owned by yahoo.com). There is no way for the benchmark code to know this, unless using a comprehensive database of who owns which domain -- that is beyond my means. Still, the benchmark is useful if comparing blockers among themselves, or against when no blocker is used.
Results -- figures are "3rd party / all". Ordered from least 3rd-party hits to most 3rd-party hits. Privacy-wise, lower numbers are better.
- Distinct 1st-party/3rd-party pairs: 245
- Scripts: 569 / 852
- Outbound cookies: 1 / 112
- Net requests: 2,458 / 5,020
- Distinct 1st-party/3rd-party pairs: 255
- Scripts: 563 / 839
- Outbound cookies: 1 / 120
- Net requests: 2,415 / 4,963
- Distinct 1st-party/3rd-party pairs: 282
- Scripts: 589 / 894
- Outbound cookies: 1 / 135
- Net requests: 2,605 / 5,301
- Distinct 1st-party/3rd-party pairs: 283
- Scripts: 637 / 930
- Outbound cookies: 1 / 136
- Net requests: 2,600 / 5,251
- Distinct 1st-party/3rd-party pairs: 352
- Scripts: 716 / 989
- Outbound cookies: 1 / 174
- Net requests: 2,704 / 5,276
- Distinct 1st-party/3rd-party pairs: 604
- Scripts: 853 / 1181
- Outbound cookies: 1 / 182
- Net requests: 3,190 / 5,990
- Distinct 1st-party/3rd-party pairs: 1160
- Scripts: 1471 / 1799
- Outbound cookies: 1 / 216
- Net requests: 5,317 / 8,207
The figures show the number of requests allowed, thus lower numbers are better. The point is to count the number of distinct 3rd-party/1st-party pairs after running the reference benchmark (three repeats in the current instance).
The less distinct 3rd-party/1st-party pairs, the better.
Adguard: it sends GET requests in the form https://sb.adtidy.org/safebrowsing-lookup-domain.html?domain={page hostname} for the first time a URL is visited. This may be related to its "Phishing and malware protection" setting. Just a guess.
Privacy Badger: warning from the browser: "This extension is slowing down Chromium. You should disable it to restore Chromium's performance."
Ultimately, if you really want to increase significantly control over your privacy, HTTP Switchboard is the way to go. If web page breakage annoys you, just start using HTTP Switchboard in allow-all/block-exceptionally mode, and blacklist your way up from this starting point. Unlike µBlock and others here, HTTP Switchboard does not have unseen exception filters which often defeat good blocking filters. For example, this is the way to foil many fingerprinting tricks, canvas fingerprinting included, without preventing javascript execution.
All blockers were configured in such a way as to compare apples-vs-apples:
- µBlock: out-of-the-box settings -- no change.
- Adblock Plus: out-of-the-box settings + "EasyPrivacy", "Malware Domains" checked. "Acceptable ads" unchecked. "Update now" clicked.
- Ghostery: out-of-the-box settings + "Advertising", "Analytics", "Beacons", "Privacy" checked. "Widgets" not checked. "GhostRank" not checked. "Update now" clicked (and ensured whatever new filters were used).
- Adguard: out-of-the-box settings + "Spyware and tracking", "Phishing and malware protection" checked. "Social media" not checked. "Acceptable ads" unchecked. "Check for filter updates" clicked.
- Disconnect: out-of-the-box settings -- no change.
- Privacy Badger: out-of-the-box settings -- no change. The extension was "primed" by visiting all the URLs in the benchmark three times before running the real benchmark.
Browser settings (if you mind your privacy, there is no way around these settings):
- "Click to play" enabled.
- "Block third party cookies and site data" enabled.
Sessbench was used to run the benchmarks, and each extension was tested as the only extension active in the browser.
The official Public Suffix List is used to determine the domain of a URL.
Note regarding the methodology: It has been said that I was unfair toward ABP because I didn't use Peter Lowe’s Ad server list for ABP while I did for µBlock. It is true that I could have imported the list into ABP, which most certainly account for the difference between ABP and µBlock. My answer to this is available at Wilders Security Forum.