feat(p2p): Add new whitelist policy by msbrogli · Pull Request #1544 · HathorNetwork/hathor-core

msbrogli · 2026-01-08T02:40:08Z

Co-Author: LFRezende lfsrsprofessional@gmail.com

Motivation

The previous peer whitelist implementation was tightly coupled across HathorManager, ConnectionsManager, and HathorSettings, mixing concerns and making it difficult to change the whitelist source (URL, local file) or policy at runtime. The whitelist was controlled by a global ENABLE_PEER_WHITELIST setting with special-case logic for sync-v1 vs. sync-v2, and the peer ID list lived on HathorManager. This refactor extracts the whitelist into a self-contained, pluggable module (hathor.p2p.whitelist) that supports multiple backends (URL-based, file-based), configurable policies (allow-all vs. only-whitelisted-peers), runtime switching via sysctl, and a grace period for bootstrap peers before the first successful fetch completes. It also removes the now-unused SyncVersion.V1_1 variant and the ENABLE_PEER_WHITELIST setting.

Acceptance Criteria

The whitelist is modeled as an abstract PeersWhitelist base class with two concrete implementations: URLPeersWhitelist (fetches from a remote URL) and FilePeersWhitelist (reads from a local file), both supporting periodic refresh with exponential backoff on failure
Whitelist files support an optional policy: directive that controls connection behavior: allow-all permits any peer, only-whitelisted-peers (default) restricts connections to listed peer IDs only
A factory function (create_peers_whitelist) and CLI argument --x-p2p-whitelist allow selecting the whitelist source at startup: default/hathorlabs (uses settings URL), none/disabled (no whitelist), a file path, or a custom URL
Before the first successful whitelist fetch, a grace period allows only bootstrap-discovered peers to connect, preventing connections to arbitrary peers while the whitelist is still loading
The ENABLE_PEER_WHITELIST setting and SyncVersion.V1_1 are removed; whitelist capability is now always required in the HELLO handshake, and the whitelist is enabled/disabled purely by whether a PeersWhitelist instance is provided to ConnectionsManager
Whitelist state is owned by ConnectionsManager.peers_whitelist instead of HathorManager.peers_whitelist, and peer_id.py's _is_peer_allowed simply delegates to the whitelist object
The sysctl interface exposes whitelist (get/set the active whitelist source, or toggle on/off to suspend/resume) and whitelist.status (returns structured state, policy, peer count, and source)
When a new whitelist is set at runtime, peers not in the updated list are immediately disconnected

Checklist

If you are requesting a merge into master, confirm this code is production-ready and can be included in future releases as soon as it gets merged

msbrogli · 2026-01-08T02:41:32Z

Replaces: #1269

github-actions · 2026-01-08T02:47:29Z

Bencher Report

Branch	feat/new-whitelist-policy
Testbed	ubuntu-22.04

Click to view all benchmark results

Benchmark	Latency	Benchmark Result minutes (m) (Result Δ%)	Lower Boundary minutes (m) (Limit %)	Upper Boundary minutes (m) (Limit %)
sync-v2 (up to 20000 blocks)	📈 view plot 🚷 view threshold	1.71 m (-0.10%) Baseline: 1.71 m	1.54 m (90.09%)	2.06 m (83.25%)

🐰 View full continuous benchmarking report in Bencher

hathor_cli/run_node.py

hathor/p2p/whitelist/parsing.py

msbrogli · 2026-02-05T19:14:02Z

hathor/p2p/manager.py

+        """Suspend the active whitelist (sysctl 'off')."""
+        if not self.peers_whitelist:
+            return
+        self.peers_whitelist.stop()


Should suspended whitelists keep updating?

No, but require a refresh when reactivating.

codecov · 2026-02-05T19:43:23Z

Codecov Report

❌ Patch coverage is 73.93484% with 104 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.67%. Comparing base (82ac020) to head (30a7cac).

Files with missing lines	Patch %	Lines
hathor/sysctl/p2p/manager.py	36.53%	33 Missing ⚠️
hathor/p2p/whitelist/url_whitelist.py	62.31%	25 Missing and 1 partial ⚠️
hathor/p2p/whitelist/file_whitelist.py	75.00%	10 Missing and 1 partial ⚠️
hathor/p2p/whitelist/peers_whitelist.py	88.04%	7 Missing and 4 partials ⚠️
hathor/p2p/whitelist/parsing.py	84.09%	4 Missing and 3 partials ⚠️
hathor/builder/builder.py	64.28%	4 Missing and 1 partial ⚠️
hathor/p2p/manager.py	90.24%	2 Missing and 2 partials ⚠️
hathor/p2p/whitelist/factory.py	85.18%	3 Missing and 1 partial ⚠️
hathor/p2p/states/hello.py	0.00%	1 Missing and 1 partial ⚠️
hathor/p2p/sync_version.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1544      +/-   ##
==========================================
+ Coverage   85.65%   85.67%   +0.02%     
==========================================
  Files         439      445       +6     
  Lines       33515    33817     +302     
  Branches     5264     5302      +38     
==========================================
+ Hits        28707    28974     +267     
- Misses       3797     3831      +34     
- Partials     1011     1012       +1

Flag	Coverage Δ
test-lib	`85.67% <73.93%> (+0.02%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

LFRezende

Review - Luis Felipe/LFRezende

Core:

Possible oversight which does not add bootstrap peers at connect_with_bootstrap_registration, lines 268-275
Discussion and further recommendation to delete --x-p2p-whitelist-only flag.
Discussion on expanding mechanism of suspended_peers.

The rest are smaller refactor suggestions.

LFRezende · 2026-02-06T13:39:05Z

hathor/p2p/whitelist/factory.py

+        PeersWhitelist instance or None if disabled
+    """
+    peers_whitelist: PeersWhitelist | None = None
+    spec_lower = whitelist_spec.lower()


Perhaps a spec = whitelist_spec.lower().strip() to eliminate any accidental whitespaces and forcing it into default by accident.

Done! Thanks!

LFRezende · 2026-02-06T14:06:14Z

hathor/p2p/whitelist/url_whitelist.py

+
+        if self._url is None:
+            return
+        if self._url.lower() == 'none':


Same as before - perhaps .lower().strip() would add a further caution to detail.

Done! Thanks!

LFRezende · 2026-02-06T14:16:08Z

hathor/p2p/manager.py

+            def connect_with_bootstrap_registration(entrypoint: PeerEndpoint) -> None:
+                if self.peers_whitelist and entrypoint.peer_id is not None:
+                    self.peers_whitelist.add_bootstrap_peer(entrypoint.peer_id)
+                self.connect_to_endpoint(entrypoint)


At the beginning of the bootstrap, when peers_whitelist = set(), each entrypoint provided for bootstrapping peers will connect, but it will not add the peer_id of that entrypoint to peers_whitelist.

peers_whitelist AND entrypoint.peer_id is not None --> At the beginning, peers_whitelist == None, hence bootstrap will not add peerId to the set, hence none will be added, just have its entrypoint connected via connect_to_endpoint.

Perhaps we could do something like:

def connect_with_bootstrap_registration(entrypoint: PeerEndpoint) -> None: if not self.peers_whitelist: return # Or raise exception if entrypoint.peer_id is not None: self.peers_whitelist.add_bootstrap_peer(entrypoint.peer_id) self.connect_to_endpoint(entrypoint)

It ensures that, if no whitelist, there will be no connection, since there are no peers to bootstrap to.
And if there is, check whether they yield an entrypoint peer id. If so, then, and only then shall you add them to the bootstrap peer.

If you want only to connect to entrypoints with a peer_id, then connect to them.

If not, just put self.connect_to_endpoint(entrypoint) out of the if clause.

Makes sense?

I fixed it using another approach: All bootstrap nodes skip the whitelist verification. Can you review it again, please?

Just did - seems fair: bootstrap peer ids are ones we are completely knowable about, so it makes sense to skip whitelist verification. Additionally, the flag "has_successful_fetch" was ingenious as it blocks unintended peers from connecting before bootstrap peers have connected.

LFRezende · 2026-02-07T16:46:18Z

hathor/p2p/manager.py

+            peer_id = conn.get_peer_id()
+            if peer_id is None:
+                continue
+            if not self.peers_whitelist.is_peer_whitelisted(peer_id):


Small suggestion: perhaps whitelist = self.peers_whitelist would make the function less verbose.

--> if not whitelist;
--> if not whitelist.is_peer_whitelisted(peer_id)

Done! Thanks!

LFRezende · 2026-02-07T16:59:38Z

hathor/sysctl/p2p/manager.py

+        if option == 'on':
+            if self._suspended_whitelist is None:
+                return
+            self.connections.set_peers_whitelist(self._suspended_whitelist)
+            self._suspended_whitelist = None
+            return
+        if option == 'off':
+            if self.connections.peers_whitelist is None:
+                return
+            self._suspended_whitelist = self.connections.peers_whitelist
+            self.connections.set_peers_whitelist(None)
+            return


So the mechanism proposed is to put the whitelist peers in suspension and turn the whitelist None (Off) and then take these peers from the "suspended set" to whitelist if ON? Interesting.

Perhaps we could expand on this idea and suspend peers in general (not just whitelist) in case we detect misbehavior from other peers - like a "blacklist" or "quarantine" of sorts.

I have a PR open for this blacklist, if this is what you were thinking about: #1554

It's a manual blacklist, we need to add peers there on demand.

My PR actually only adds support in sysctl to a blacklist logic we had already implemented in hathor-core.

Blacklisting can help when a specific full node is misbehaving. However, it’s not very effective against targeted attacks. Generating new peer IDs is extremely cheap, so an attacker can simply create a new identity and reconnect to your node almost immediately.

LFRezende · 2026-02-07T17:15:10Z

hathor/sysctl/p2p/manager.py

+    def set_whitelist(self, new_whitelist: str) -> None:
+        """Set the whitelist-only mode. If 'on' or 'off', simply changes the
+        following status of current whitelist. If an URL or Filepath, changes
+        the whitelist object, following it by default.
+        It does not support eliminating the whitelist (passing None)."""
+
+        option: str = new_whitelist.lower().strip()
+        if option == 'on':
+            if self._suspended_whitelist is None:
+                return
+            self.connections.set_peers_whitelist(self._suspended_whitelist)
+            self._suspended_whitelist = None
+            return
+        if option == 'off':
+            if self.connections.peers_whitelist is None:
+                return
+            self._suspended_whitelist = self.connections.peers_whitelist
+            self.connections.set_peers_whitelist(None)


Small nitpick: perhaps connections = self.connections would reduce verbose, as it is a term used frequently.

Done! Thanks!

hathor_cli/run_node.py

LFRezende · 2026-02-07T17:50:04Z

hathor_tests/others/test_cli_builder.py

+    def test_whitelist_cli_args(self):
+        """Test --x-p2p-whitelist and --x-p2p-whitelist-only CLI arguments."""
+        # Test with whitelist URL
+        manager = self._build(['--temp-data', '--x-p2p-whitelist', 'https://example.com/whitelist'])
+        self.assertIsNotNone(manager.connections.peers_whitelist)
+        self.assertIsInstance(manager.connections.peers_whitelist, URLPeersWhitelist)
+
+        # Test with whitelist-only flag (now a no-op, whitelist always enforces)
+        manager2 = self._build([
+            '--temp-data', '--x-p2p-whitelist', 'https://example.com/whitelist',
+            '--x-p2p-whitelist-only',
+        ])
+        self.assertIsNotNone(manager2.connections.peers_whitelist)
+
+        # Test with disabled whitelist
+        manager3 = self._build(['--temp-data', '--x-p2p-whitelist', 'none'])
+        self.assertIsNone(manager3.connections.peers_whitelist)


Since we're removing --x-p2p-whitelist-only flag from cli, we may delete them here.

Done! Thanks!

hathor/p2p/whitelist/peers_whitelist.py

LFRezende

Last changes on bootstrap peers and their derived tests seem reasonable.

Ran linter and tests locally:

All linters work fine
Two tests failed, and the last one seems to be connected to bootstrap (already communicated msbrogli).

It seems to be flaking - I'll approve.

glevco · 2026-02-25T02:49:58Z

hathor_cli/run_node.py

        parser.add_argument('--x-disable-ipv4', action='store_true',
                            help='Disables connecting to IPv4 peers')
+
+        parser.add_argument("--x-p2p-whitelist", help="Add whitelist to follow from since boot.")


glevco · 2026-02-25T02:53:18Z

hathor/p2p/whitelist/factory.py

+# Whitelist specification constants
+WHITELIST_SPEC_DEFAULT = 'default'
+WHITELIST_SPEC_HATHORLABS = 'hathorlabs'
+WHITELIST_SPEC_NONE = 'none'
+WHITELIST_SPEC_DISABLED = 'disabled'


Create a StrEnum?

glevco · 2026-02-25T02:56:00Z

hathor/p2p/whitelist/factory.py

+    if spec_lower in (WHITELIST_SPEC_DEFAULT, WHITELIST_SPEC_HATHORLABS):
+        peers_whitelist = URLPeersWhitelist(reactor, str(settings.WHITELIST_URL), True)
+    elif spec_lower in (WHITELIST_SPEC_NONE, WHITELIST_SPEC_DISABLED):
+        peers_whitelist = None
+    elif _looks_like_url(whitelist_spec):
+        peers_whitelist = URLPeersWhitelist(reactor, whitelist_spec, True)


Why are you passing mainnet=True in the constructors? Also it would be better if those args were kw only.

glevco · 2026-02-25T03:00:10Z

hathor/p2p/whitelist/parsing.py

+    parse_whitelist('''hathor-whitelist
+# node1
+ 2ffdfbbfd6d869a0742cff2b054af1cf364ae4298660c0e42fa8b00a66a30367
+
+2ffdfbbfd6d869a0742cff2b054af1cf364ae4298660c0e42fa8b00a66a30367
+
+# node3
+G2ffdfbbfd6d869a0742cff2b054af1cf364ae4298660c0e42fa8b00a66a30367
+2ffdfbbfd6d869a0742cff2b054af1cf364ae4298660c0e42fa8b00a66a30367
+''')
+    {'2ffdfbbfd6d869a0742cff2b054af1cf364ae4298660c0e42fa8b00a66a30367'}


This comment is wrong, use doctests instead.

glevco · 2026-02-25T03:05:47Z

hathor/p2p/whitelist/parsing.py

+
+    """
+    lines = parse_file(text, header=header)
+    return {PeerId(line.split()[0]) for line in lines}


Why this split?

glevco · 2026-02-25T03:26:56Z

hathor/p2p/whitelist/peers_whitelist.py

+
+        peers_to_remove = current_whitelist - new_whitelist
+        for peer_id in peers_to_remove:
+            if self._on_remove_callback:


Should assert instead

glevco · 2026-02-25T03:28:55Z

hathor/p2p/whitelist/file_whitelist.py

+    def refresh(self) -> Deferred[None]:
+        return self._unsafe_update()


It looks like this is only used in a test, it could be removed.

glevco · 2026-02-25T03:35:00Z

hathor/p2p/whitelist/url_whitelist.py

+        if mainnet:
+            if result.scheme != 'https':
+                raise ValueError(f'invalid scheme: {self._url}')


Should this come from a setting in HathorSettings instead of hardcoding it for mainnet?

glevco · 2026-02-25T03:35:21Z

hathor/p2p/whitelist/url_whitelist.py

+            if not result.netloc:
+                raise ValueError(f'invalid url: {self._url}')


This should be checked even for non-mainnet, right?

glevco · 2026-02-25T03:36:34Z

hathor/p2p/whitelist/url_whitelist.py

+            d: Deferred[None] = Deferred()
+            d.callback(None)
+            return d


Use succeed

Co-authored-by: LFRezende <lfsrsprofessional@gmail.com>

msbrogli requested a review from jansegre as a code owner January 8, 2026 02:40

msbrogli self-assigned this Jan 8, 2026

msbrogli added this to Hathor Network Jan 8, 2026

github-project-automation bot moved this to Todo in Hathor Network Jan 8, 2026

msbrogli moved this from Todo to In Progress (WIP) in Hathor Network Jan 8, 2026

msbrogli force-pushed the feat/new-whitelist-policy branch from a414e74 to 812f883 Compare January 20, 2026 21:19

msbrogli force-pushed the feat/new-whitelist-policy branch 5 times, most recently from df4d9d6 to e5c9d1b Compare February 3, 2026 05:34

msbrogli commented Feb 5, 2026

View reviewed changes

hathor_cli/run_node.py Outdated Show resolved Hide resolved

msbrogli force-pushed the feat/new-whitelist-policy branch 4 times, most recently from c35a8f9 to 743a986 Compare February 5, 2026 18:21

msbrogli commented Feb 5, 2026

View reviewed changes

hathor/p2p/whitelist/parsing.py Outdated Show resolved Hide resolved

msbrogli commented Feb 5, 2026

View reviewed changes

hathor/p2p/whitelist/parsing.py Outdated Show resolved Hide resolved

msbrogli force-pushed the feat/new-whitelist-policy branch from 743a986 to 4519d69 Compare February 5, 2026 19:11

msbrogli commented Feb 5, 2026

View reviewed changes

msbrogli force-pushed the feat/new-whitelist-policy branch from 4519d69 to d944ce0 Compare February 5, 2026 19:17

msbrogli force-pushed the feat/new-whitelist-policy branch 2 times, most recently from a60aa15 to 5e76bfa Compare February 5, 2026 20:19

msbrogli moved this from In Progress (WIP) to In Progress (Done) in Hathor Network Feb 5, 2026

LFRezende reviewed Feb 7, 2026

View reviewed changes

msbrogli moved this from In Progress (Done) to In Review (WIP) in Hathor Network Feb 10, 2026

msbrogli force-pushed the master branch from e901df3 to eb416fa Compare February 12, 2026 23:10

luislhl reviewed Feb 18, 2026

View reviewed changes

hathor/p2p/whitelist/peers_whitelist.py Outdated Show resolved Hide resolved

luislhl reviewed Feb 18, 2026

View reviewed changes

hathor/p2p/whitelist/peers_whitelist.py Show resolved Hide resolved

msbrogli moved this from In Review (WIP) to In Review (Done) in Hathor Network Feb 19, 2026

msbrogli force-pushed the feat/new-whitelist-policy branch 3 times, most recently from 5b8b5fd to e623233 Compare February 20, 2026 18:43

msbrogli moved this from In Review (Done) to In Progress (Done) in Hathor Network Feb 20, 2026

luislhl previously approved these changes Feb 20, 2026

View reviewed changes

jansegre previously approved these changes Feb 20, 2026

View reviewed changes

github-project-automation bot moved this from In Progress (Done) to In Review (WIP) in Hathor Network Feb 20, 2026

msbrogli dismissed stale reviews from jansegre and luislhl via bdd05ab February 20, 2026 21:12

luislhl previously approved these changes Feb 20, 2026

View reviewed changes

msbrogli dismissed luislhl’s stale review via 70810e9 February 21, 2026 02:07

msbrogli force-pushed the feat/new-whitelist-policy branch 2 times, most recently from b361dc6 to 0fdcd2e Compare February 23, 2026 18:15

jansegre mentioned this pull request Feb 23, 2026

feat: add peer-ids to blacklist using sysctl #1554

Open

3 tasks

LFRezende previously approved these changes Feb 24, 2026

View reviewed changes

jansegre previously approved these changes Feb 24, 2026

View reviewed changes

jansegre moved this from In Review (WIP) to In Review (Done) in Hathor Network Feb 24, 2026

glevco requested changes Feb 25, 2026

View reviewed changes

github-project-automation bot moved this from In Review (Done) to In Review (WIP) in Hathor Network Feb 25, 2026

glevco moved this from In Review (WIP) to In Review (Done) in Hathor Network Feb 25, 2026

msbrogli and others added 4 commits February 26, 2026 10:59

feat(p2P): Add new whitelist policy

230b8fa

Co-authored-by: LFRezende <lfsrsprofessional@gmail.com>

review changes

257ae7f

review changes (2)

9c02dae

review changes (3)

30a7cac

msbrogli dismissed stale reviews from jansegre and LFRezende via 30a7cac February 26, 2026 17:07

msbrogli force-pushed the feat/new-whitelist-policy branch from 0fdcd2e to 30a7cac Compare February 26, 2026 17:07

		def refresh(self) -> Deferred[None]:
		return self._unsafe_update()

		if not result.netloc:
		raise ValueError(f'invalid url: {self._url}')

Conversation

msbrogli commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Acceptance Criteria

Checklist

Uh oh!

msbrogli commented Jan 8, 2026

Uh oh!

github-actions bot commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

msbrogli Feb 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

LFRezende left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LFRezende Feb 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

LFRezende left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

msbrogli commented Jan 8, 2026 •

edited

Loading

github-actions bot commented Jan 8, 2026 •

edited

Loading

msbrogli Feb 20, 2026 •

edited

Loading

codecov bot commented Feb 5, 2026 •

edited

Loading

LFRezende Feb 20, 2026 •

edited

Loading