Skip to content

Conversation

leoparente
Copy link
Contributor

This pull request introduces significant improvements to the policy runner's backend integration and job scheduling, focusing on supporting backend caching, forwarding additional arguments, and enhancing test coverage. It also includes minor workflow adjustments for linting jobs.

Backend extensibility and caching:

  • The Backend.run method now accepts arbitrary keyword arguments (**kwargs), allowing the policy runner to pass additional parameters such as cache objects and scheduling controls. ([worker/worker/backend.pyL28-R36](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-3b735cc0006b8dac829eecfb9a8832393219518c12aaed7535e4410a5c36ab10L28-R36))
  • The policy runner (PolicyRunner) now supports a backend cache mechanism: it attempts to create a cache using a backend's create_cache method if available, or falls back to cache config or an empty dict. The cache is injected into backend runs and reset on runner stop. ([[1]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-77ccd0c70b117c80a734aad5873b3fce2bfca6d5d1486c8bcf12f6e19cdd57e2R37-R39), [[2]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-77ccd0c70b117c80a734aad5873b3fce2bfca6d5d1486c8bcf12f6e19cdd57e2R58-R61), [[3]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-77ccd0c70b117c80a734aad5873b3fce2bfca6d5d1486c8bcf12f6e19cdd57e2R83-R92), [[4]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-77ccd0c70b117c80a734aad5873b3fce2bfca6d5d1486c8bcf12f6e19cdd57e2R254-R264), [[5]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-77ccd0c70b117c80a734aad5873b3fce2bfca6d5d1486c8bcf12f6e19cdd57e2R200-R223))
  • The runner also forwards schedule_now and any other relevant configuration to backend jobs, making the job execution context more flexible. ([[1]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-77ccd0c70b117c80a734aad5873b3fce2bfca6d5d1486c8bcf12f6e19cdd57e2R83-R92), [[2]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-77ccd0c70b117c80a734aad5873b3fce2bfca6d5d1486c8bcf12f6e19cdd57e2L94-R110))

Job scheduling and execution improvements:

  • The runner maintains internal job args and kwargs, uses them for scheduled and immediate jobs, and provides a schedule_now method to trigger immediate backend runs with merged kwargs. ([[1]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-77ccd0c70b117c80a734aad5873b3fce2bfca6d5d1486c8bcf12f6e19cdd57e2R37-R39), [[2]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-77ccd0c70b117c80a734aad5873b3fce2bfca6d5d1486c8bcf12f6e19cdd57e2R83-R92), [[3]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-77ccd0c70b117c80a734aad5873b3fce2bfca6d5d1486c8bcf12f6e19cdd57e2L94-R110), [[4]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-77ccd0c70b117c80a734aad5873b3fce2bfca6d5d1486c8bcf12f6e19cdd57e2R200-R223))
  • The run method ensures the cache is always passed to the backend, either from the runner or from explicit kwargs. ([[1]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-77ccd0c70b117c80a734aad5873b3fce2bfca6d5d1486c8bcf12f6e19cdd57e2L104-R124), [[2]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-77ccd0c70b117c80a734aad5873b3fce2bfca6d5d1486c8bcf12f6e19cdd57e2R141-R145))

Testing enhancements:

  • Extensive new and updated tests verify that caches and extra kwargs are forwarded correctly, that backend cache factories are used, and that immediate scheduling works as intended. Tests for error handling and metrics are also updated to reflect the new cache logic. ([[1]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-f6ad41627bb040e8e4b7e13d214a73c2bd73b5da5f9882ec7bfe9b1c83c63a51R114-R121), [[2]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-f6ad41627bb040e8e4b7e13d214a73c2bd73b5da5f9882ec7bfe9b1c83c63a51R144-R150), [[3]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-f6ad41627bb040e8e4b7e13d214a73c2bd73b5da5f9882ec7bfe9b1c83c63a51R174-R255), [[4]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-f6ad41627bb040e8e4b7e13d214a73c2bd73b5da5f9882ec7bfe9b1c83c63a51L180-R305), [[5]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-f6ad41627bb040e8e4b7e13d214a73c2bd73b5da5f9882ec7bfe9b1c83c63a51R315), [[6]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-f6ad41627bb040e8e4b7e13d214a73c2bd73b5da5f9882ec7bfe9b1c83c63a51L215-R336), [[7]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-f6ad41627bb040e8e4b7e13d214a73c2bd73b5da5f9882ec7bfe9b1c83c63a51R353), [[8]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-f6ad41627bb040e8e4b7e13d214a73c2bd73b5da5f9882ec7bfe9b1c83c63a51L241-R379), [[9]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-f6ad41627bb040e8e4b7e13d214a73c2bd73b5da5f9882ec7bfe9b1c83c63a51R408-R409), [[10]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-f6ad41627bb040e8e4b7e13d214a73c2bd73b5da5f9882ec7bfe9b1c83c63a51L302-R432), [[11]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-d55e46e202e1443376814c891586fe105d25509f22b905c49f66421d6818fc31R39-R51), [[12]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-759b8898f054470a76ad681c97fa2375588f74cd9cfc60bd7a633854ecf2af78R76-R80))

Workflow/linting:

  • The GitHub Actions workflows for device discovery and worker linting are tightened by removing continue-on-error: true, making lint failures blocking. ([[1]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-5825ca970a22eff6f4d1f512a9fcf8b6dd6cc38e2a132379c39062ad2482ababL61), [[2]](https://github.com/netboxlabs/orb-discovery/pull/192/files#diff-cc57007a672d909cc9b067a40e7402b56b05e647ae6aaab926f4bf8b9115a129L61))

These changes make the policy runner more robust, extensible, and testable, especially for backends that need persistent state or custom job parameters.

@leoparente leoparente self-assigned this Sep 29, 2025
Copy link

github-actions bot commented Sep 29, 2025

Coverage

Coverage Report
FileStmtsMissCoverMissing
worker
   main.py54493%185–186, 195, 201
   metrics.py53198%104
   server.py81495%34–36, 167, 170
   version.py7186%14
worker/policy
   manager.py48394%36–37, 52
   runner.py128596%117, 211, 263–265
TOTAL4281896% 

Tests Skipped Failures Errors Time
63 0 💤 0 ❌ 0 🔥 2.581s ⏱️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant