T6948: Keep DHCP server leases in sync with hostd records #4237

indrajitr · 2024-12-13T11:28:46Z

Change Summary

Keep DHCP server leases in sync with vyos-hostd records via helper script invoked with ExecStartPost directive in kea-dhcp4-server.service.

The helper script updates VyOS hostd records from DHCP server leases. This ensures that hostd records with the DHCP server leases are kept in sync with VyOS hostd records right after DHCP server is started.

This is similar to the capability exposed via kea hook libdhcp_run_script.so which is invoked internally by kea when a single lease changes state.

Since the kea hook is currently implemented for DHCPv4 only, this helper script is implemented for DHCPv4 only as well.

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Code style update (formatting, renaming)
Refactoring (no functional changes)
Migration from an old Vyatta component to vyos-1x, please link to related PR inside obsoleted component
Other (please describe):

Related Task(s)

https://vyos.dev/T6948

Related PR(s)

#4307

Component(s) name

dhcp server

Proposed changes

How to test

Enable DHCP server and have active server leases
Ensure that relevant host entries exist in /etc/hosts and/or /run/vyos-hostsd/vyos-hostsd.state
Restart VyOS
Verify DHCP server leases are still active (show dhcp server leases)
Verify that the leases have NOT been applied to /etc/hosts
Restart dhcp server (restart dhcp server)
Verify that the leases have been applied to /etc/hosts

Smoketest result

vyos@test-1216:~$ /usr/libexec/vyos/tests/smoke/cli/test_service_dhcp-server.py
test_dhcp_exclude_in_range (__main__.TestServiceDHCPServer.test_dhcp_exclude_in_range) ... ok
test_dhcp_exclude_not_in_range (__main__.TestServiceDHCPServer.test_dhcp_exclude_not_in_range) ... ok
test_dhcp_high_availability (__main__.TestServiceDHCPServer.test_dhcp_high_availability) ... ok
test_dhcp_high_availability_standby (__main__.TestServiceDHCPServer.test_dhcp_high_availability_standby) ... ok
test_dhcp_hostsd_lease_sync (__main__.TestServiceDHCPServer.test_dhcp_hostsd_lease_sync) ... ok
test_dhcp_multiple_pools (__main__.TestServiceDHCPServer.test_dhcp_multiple_pools) ... ok
test_dhcp_on_interface_with_vrf (__main__.TestServiceDHCPServer.test_dhcp_on_interface_with_vrf) ... ok
test_dhcp_relay_server (__main__.TestServiceDHCPServer.test_dhcp_relay_server) ... ok
test_dhcp_single_pool_options (__main__.TestServiceDHCPServer.test_dhcp_single_pool_options) ... ok
test_dhcp_single_pool_options_scoped (__main__.TestServiceDHCPServer.test_dhcp_single_pool_options_scoped) ... ok
test_dhcp_single_pool_range (__main__.TestServiceDHCPServer.test_dhcp_single_pool_range) ... ok
test_dhcp_single_pool_static_mapping (__main__.TestServiceDHCPServer.test_dhcp_single_pool_static_mapping) ... ok

Checklist:

I have read the CONTRIBUTING document
I have linked this PR to one or more Phabricator Task(s)
I have run the components SMOKETESTS if applicable
My commit headlines contain a valid Task id
My change requires a change to the documentation
I have updated the documentation accordingly

github-actions · 2024-12-13T11:29:03Z

👍
No issues in PR Title / Commit Title

indrajitr · 2024-12-13T18:01:44Z

Keeping it as draft till I have a working test environment.

indrajitr · 2024-12-26T18:17:17Z

Update: Ready for review.
This has been working fine for me in the local environment for the last few days.

sarthurdev

LGTM. Thanks for adding the smoketest.

indrajitr · 2025-01-09T22:41:31Z

src/etc/systemd/system/kea-dhcp4-server.service.d/override.conf

@@ -5,3 +5,5 @@ After=vyos-router.service
 [Service]
 ExecStart=
 ExecStart=/usr/sbin/kea-dhcp4 -c /run/kea/kea-dhcp4.conf
+ExecStartPost=!/usr/bin/python3 /usr/libexec/vyos/op_mode/dhcp.py update_dhcp_server_lease_to_hostd_state --family inet


Thanks for the review @sarthurdev! I need some feedback on the convention of invoking this one.
So far I haven't seen any precedence of invoking op_mode/*.py directly in the codebase. Is this okay, or do you want me to revisit this?

it's not the ideal solution, as I don't see a use case where it'd be needed in op-mode, nor is it defined in XML.

Probably best done in a script in src/system like the Kea hook scripts.

Yes, that's why I asked :)

As you can see in the implementation of update_dhcp_server_lease_to_hostd_state, the method crucially depends on _get_dhcp_pools and _get_raw_server_leases which are implemented in op_mode/dhcp.py which is why I added the method in update_dhcp_server_lease_to_hostd_state. Could they be refactored to kea lib (python/vyos/kea.py) perhaps so that they can be reused?

Or what if we actually add an op-mode command - something like update hostd state and define it in XML as well?

NB: I realize that we are getting into design discussion here. We can discuss the options on Slack if necessary.

Ideally there shouldn't be a scenario where the user needs to manually re-sync hostd state.

I agree that the utility functions could move to the vyos.kea module and be imported in op-mode and a new src/system script. Though for the module function, it should probably be just the logic that needs to be re-used (no sorting, filtering etc. Just returning the lease data as-is).

@sarthurdev, raised #4307 to refactor utility functions to vyos.kea. Would appreciate feedback on that.

Now that #4307 has been applied, I have made the necessary changes to better comply with the recommended convention.

github-actions · 2025-01-20T20:09:05Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2025-01-21T04:27:52Z

Conflicts have been resolved. A maintainer will review the pull request shortly.

Add helpers: - `kea_add_lease` to add a lease to the running kea server - `kea_get_domain_from_subnet_id` to get the domain name from subnet id Also, enrich leases with domain name from subnet id

Keep DHCP server leases in sync with vyos-hostd records via helper script invoked with `ExecStartPost` directive in kea-dhcp4-server.service. The helper script updates VyOS hostd records from DHCP server leases. This ensures that hostd records with the DHCP server leases are kept in sync with VyOS hostd records right after DHCP server is started. This is similar to the capability exposed via kea hook `libdhcp_run_script.so` which is invoked internally by kea when a single lease changes state. Since the kea hook is currently implemented for DHCPv4 only, this helper script is implemented for DHCPv4 only as well.

The helper script updates VyOS hostd records from DHCP server leases. This ensures that hostd records with the DHCP server leases are kept in sync with VyOS hostd records right after the DHCP server is started. Note that `Restart` directive needs to be updated to `on-failure` so that the service is restarted in case of failure/timeout in interaction with hostd service.

The test `test_dhcp_hostsd_lease_sync` validates DHCP server leases in sync with hostd records. Since the DHCP server running in smoketest environment might not have actual clients to lease IP addresses, we simulate the lease creation and deletion using kea helper functions (`kea_add_lease` and `kea_delete_lease`).

The formatter methods are mostly `family` agnostic now.

github-actions · 2025-01-25T09:58:20Z

CI integration 👍 passed!

Details

CI logs

CLI Smoketests (no interfaces) 👍 passed
CLI Smoketests (interfaces only) 👍 passed
Config tests 👍 passed
RAID1 tests 👍 passed
TPM tests 👍 passed

indrajitr requested review from dmbaturin, sarthurdev, jestabro, sever-sever, c-po, fett0, nicolas-fort and zdc as code owners December 13, 2024 11:28

github-actions bot assigned indrajitr Dec 13, 2024

github-actions bot added the current label Dec 13, 2024

indrajitr marked this pull request as draft December 13, 2024 18:01

indrajitr force-pushed the hostd-update branch 2 times, most recently from 5e444ca to 748310c Compare December 19, 2024 00:24

indrajitr marked this pull request as ready for review December 19, 2024 00:29

indrajitr force-pushed the hostd-update branch from 748310c to d1a8438 Compare December 19, 2024 00:31

indrajitr force-pushed the hostd-update branch from d1a8438 to e8d7cc3 Compare January 9, 2025 20:33

sarthurdev approved these changes Jan 9, 2025

View reviewed changes

indrajitr force-pushed the hostd-update branch from e8d7cc3 to 5d094e9 Compare January 9, 2025 22:24

indrajitr commented Jan 9, 2025

View reviewed changes

github-actions bot added the conflicts label Jan 20, 2025

indrajitr force-pushed the hostd-update branch from 5d094e9 to 462f009 Compare January 21, 2025 04:27

github-actions bot removed the conflicts label Jan 21, 2025

indrajitr force-pushed the hostd-update branch 2 times, most recently from 7c2f986 to 2aed4d7 Compare January 21, 2025 04:40

indrajitr requested a review from sarthurdev January 21, 2025 04:43

sever-sever approved these changes Jan 21, 2025

View reviewed changes

indrajitr force-pushed the hostd-update branch 2 times, most recently from 3de32ce to 6333e4f Compare January 22, 2025 05:38

indrajitr added 6 commits January 25, 2025 00:41

dhcp: T6948: kea: Add kea helpers and enrich leases with domain name

61907b2

Add helpers: - `kea_add_lease` to add a lease to the running kea server - `kea_get_domain_from_subnet_id` to get the domain name from subnet id Also, enrich leases with domain name from subnet id

ruff: T6583: Reformat to comply with code style

33ac5b3

dhcp: T7052: Clean up unnecessary params in formatter methods

6c14589

The formatter methods are mostly `family` agnostic now.

indrajitr force-pushed the hostd-update branch from 6333e4f to 6c14589 Compare January 25, 2025 08:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

T6948: Keep DHCP server leases in sync with hostd records #4237

T6948: Keep DHCP server leases in sync with hostd records #4237

indrajitr commented Dec 13, 2024 •

edited

Loading

github-actions bot commented Dec 13, 2024 •

edited

Loading

indrajitr commented Dec 13, 2024

indrajitr commented Dec 26, 2024

sarthurdev left a comment

indrajitr Jan 9, 2025

sarthurdev Jan 10, 2025

indrajitr Jan 10, 2025

sarthurdev Jan 12, 2025

indrajitr Jan 16, 2025

indrajitr Jan 21, 2025

github-actions bot commented Jan 20, 2025

github-actions bot commented Jan 21, 2025

github-actions bot commented Jan 25, 2025

T6948: Keep DHCP server leases in sync with hostd records #4237

Are you sure you want to change the base?

T6948: Keep DHCP server leases in sync with hostd records #4237

Conversation

indrajitr commented Dec 13, 2024 • edited Loading

Change Summary

Types of changes

Related Task(s)

Related PR(s)

Component(s) name

Proposed changes

How to test

Smoketest result

Checklist:

github-actions bot commented Dec 13, 2024 • edited Loading

indrajitr commented Dec 13, 2024

indrajitr commented Dec 26, 2024

sarthurdev left a comment

Choose a reason for hiding this comment

indrajitr Jan 9, 2025

Choose a reason for hiding this comment

sarthurdev Jan 10, 2025

Choose a reason for hiding this comment

indrajitr Jan 10, 2025

Choose a reason for hiding this comment

sarthurdev Jan 12, 2025

Choose a reason for hiding this comment

indrajitr Jan 16, 2025

Choose a reason for hiding this comment

indrajitr Jan 21, 2025

Choose a reason for hiding this comment

github-actions bot commented Jan 20, 2025

github-actions bot commented Jan 21, 2025

github-actions bot commented Jan 25, 2025

Details

indrajitr commented Dec 13, 2024 •

edited

Loading

github-actions bot commented Dec 13, 2024 •

edited

Loading