µBlock and others: Blocking ads, trackers, malwares
Hard data, not hype.
Edit 2014-10-02: I will be re-doing the benchmark once the stable release of µBlock v0.6.6.0 is out. Settings for Adblock Plus and µBlock will be brought more inline with those of Ghostery and Disconnect, which block Facebook completely. For ABP and µBlock, it's just a question of enabling "Fanboy’s Anti-Facebook" list.
Latest benchmark: 30 September 2014 (raw data spreadsheet). (Previous)
This benchmark is to measure privacy exposure, by counting the number of distinct 3rd-party domains which have been hit by net requests during the benchmark. The lower the number of distinct 3rd-party domains hit, the better.
Some benchmarks measure the amount of requests blocked, which I think is of no interest as a useful measurement of privacy exposure. The number of requests blocked is no guarantee of less distinct 3rd-party domains being hit (and leaving a trace in the servers' logs).
Measuring directly the number of distinct 3rd-party domains which were hit is a much better and relevant measurement for comparison of privacy protection efficiency in my opinion.
Caveat: "3rd-party" is defined as a domain which doesn't match the domain of the web page. For sure many
domains reported as "3rd-party" actually belong to the same entity which owns the page domain (for example, yimg.com is owned by yahoo.com). There is no way for the benchmark code to know this, unless using a comprehensive database of who owns which domain -- that is beyond my means. Still, the benchmark is useful if comparing blockers among themselves, or against when no blocker is used.
Results -- figures are "3rd party / all". Ordered from least 3rd-party hits to most 3rd-party hits. Privacy-wise, lower numbers are better.
- Distinct 1st-party/3rd-party pairs: 197
- Scripts: 490 / 796
- Outbound cookies: 0 / 135
- Net requests: 2,548 / 5,304
µBlock (gorhill) 0.6.6.0
- Distinct 1st-party/3rd-party pairs: 205
- Scripts: 507 / 793
- Outbound cookies: 0 / 112
- Net requests: 2,467 / 5,057
- Distinct 1st-party/3rd-party pairs: 285
- Scripts: 681 / 1011
- Outbound cookies: 0 / 131
- Net requests: 2,871 / 5,558
- Distinct 1st-party/3rd-party pairs: 369
- Scripts: 774 / 1106
- Outbound cookies: 0 / 139
- Net requests: 2,966 / 5,671
- Distinct 1st-party/3rd-party pairs: 400
- Scripts: 922 / 1258
- Outbound cookies: 0 / 202
- Net requests: 3,266 / 6,141
- Distinct 1st-party/3rd-party pairs: 1578
- Scripts: 2659 / 3156
- Outbound cookies: 0 / 250
- Net requests: 8,225 / 11,718
The figures show the number of requests allowed, thus lower numbers are better. The point is to count the number of distinct 3rd-party/1st-party pairs after running the reference benchmark (three repeats in the current instance).
The less distinct 3rd-party/1st-party pairs, the better.
I included µBlock with my own selection of lists, to remind users that they can easily turn µBlock into a much more aggressive privacy exposure-reducing blocker.
This shows the differences in what was not blocked. If something appears on side A but not on side B, this mean side B blocked something not blocked by side A, and vice versa.
- µBlock/Ghostery: https://www.diffchecker.com/pz6rv6lq
- µBlock/ABP: https://www.diffchecker.com/r7v1cq6x
- µBlock/Disconnect: https://www.diffchecker.com/djp4xlg1
- µBlock/µBlock (gorhill): https://www.diffchecker.com/861355g2
All blockers were configured in such a way as to compare apples-vs-apples:
- Ghostery: Select all trackers. "GhostRank" not checked. "Update now" clicked (and ensured whatever new filters were used).
- µBlock: out-of-the-box settings + local mirroring enabled and primed (through "Experimental features").
- µBlock (gorhill): see "Filter lists: gorhill" + local mirroring enabled and primed (through "Experimental features").
- Adblock Plus: "EasyList" + "EasyPrivacy", "Fanboy's Social Block List", "Malware Domains" checked. "Acceptable ads" unchecked. "Update now" clicked.
- Disconnect: out-of-the-box settings -- no change.
Browser settings (if you mind your privacy, there is no way around these settings):
- "Click to play" enabled.
- "Block third party cookies and site data" enabled.
Sessbench was used to run the benchmarks, and each extension was tested as the only extension active in the browser.
The official Public Suffix List is used to determine the domain of a URL.