WIP: Fix 4352

[Jeremy-Boyle]

This issue fixes #4352

[Jeremy-Boyle]

One issue to note here this does remove

general.rndcalgo and general.rndsecret,

A migration will have to be made to migrate this key to the new keys. Primary domains will also have to migrate if domain.allowrndctransfer or domain.allowrndcupdate was enabled,

Flow for migration.

• general.rndcalgo and general.rndsecret, -> creates a key named rndc-key in keys to not break old configurations.
• if domain.allowrndctransfer or domain.allowrndcupdate was enabled -> domain.allowedkeys updated with migrated rndc-key.
• general.rndccontrolkeys updated with migrated rndc-key.
• general.defaultrndccontrolkey updated with migrated rndc-key.

I'm new to how migrations work and will need help or at least some pointers on how to test migrations, I have created a migration .php file, which has not been pushed yet because I am unsure how that works can someone please assist with the migrations ?

[Jeremy-Boyle]

@NHellFire @sestary Any idea how we can sync from the database file back into opnsense for transfers ? I have solved the issue with the journal restarts, running rndc sync, freeze and thaw poplulates the db file now. But is still removed on config save.

Looks like we would have to parse the db file and update the model, I'm not sure what the best option to do this would be ?

[Jeremy-Boyle]

https://github.com/opnsense/plugins/blob/master/www/OPNProxy/src/opnsense/scripts/OPNProxy/redis_sync_users.py

This is one option that i see for example, i would like to prevent opening the config file if possible. But this is a option, that potentially could have a cron or something that sits and watches for when it needs to update.

[Jeremy-Boyle]

I did see this

#3258

@sestary what was the reasoning behind moving this to the general field ?

With the functionality provided inside of #4177 does this now make sense, also, this would also allow in a future request to migrate secondary zones to also use the keys as well.

[NHellFire]

@NHellFire @sestary Any idea how we can sync from the database file back into opnsense for transfers ? I have solved the issue with the journal restarts, running rndc sync, freeze and thaw poplulates the db file now. But is still removed on config save.

Looks like we would have to parse the db file and update the model, I'm not sure what the best option to do this would be ?

An ideal solution would be to parse the journal with named-journalprint and sync those updates with the opnsense config, then delete the journal.
That way the updated records would be visible in the UI and not be lost when manually editing the zone.

root@OPNsense:~ # named-journalprint /usr/local/etc/namedb/primary/dev.k0s.home.db.jnl
del dev.k0s.home.		86400	IN	SOA	ns.dev.k0s.home. mail.opnsense.localdomain. 2411151601 21600 3600 3542400 3600
add dev.k0s.home.		86400	IN	SOA	ns.dev.k0s.home. mail.opnsense.localdomain. 2411151602 21600 3600 3542400 3600
add dashboard.dev.k0s.home.	0	IN	A	198.18.28.1
add external-dnsdashboard.dev.k0s.home. 0 IN TXT	"heritage=external-dns,external-dns/owner=dev,external-dns/resource=ingress/kubernetes-dashboard/kubernetes-dashboard"
add external-dnsa-dashboard.dev.k0s.home. 0	IN TXT	"heritage=external-dns,external-dns/owner=dev,external-dns/resource=ingress/kubernetes-dashboard/kubernetes-dashboard"

Alternatively, rather than processing all the individual updates manually, we could just process the updated zone and delete any records from the opnsense config that are no longer in the zone:

root@OPNsense:~ # named-checkzone -Djs full dev.k0s.home /usr/local/etc/namedb/primary/dev.k0s.home.db
dev.k0s.home.				      86400 IN SOA	ns.dev.k0s.home. mail.opnsense.localdomain. 2411151639 21600 3600 3542400 3600
dev.k0s.home.				      86400 IN NS	ns.dev.k0s.home.
dashboard.dev.k0s.home.			      0	IN A		198.18.28.1
external-dnsa-dashboard.dev.k0s.home.	      0	IN TXT		"heritage=external-dns,external-dns/owner=dev,external-dns/resource=ingress/kubernetes-dashboard/kubernetes-dashboard"
external-dnsdashboard.dev.k0s.home.	      0	IN TXT		"heritage=external-dns,external-dns/owner=dev,external-dns/resource=ingress/kubernetes-dashboard/kubernetes-dashboard"

We'd need to ensure that the journal is has been applied to the opnsense config any time we're listing or modifying the zone to avoid conflicts.

Both formats are easy enough to parse, I'm just not sure how to go about integrating that

[Jeremy-Boyle]

Hi @NHellFire , yes i thought of that same example and started writing some code to do that.

One option i was thinking was to hook it with the PHP, to open the file and create the record and handle the relations, However, this would have a issue with that you would have to have a button to sync it that the user would have to push or you could hook it to do the check every time someone goes to look at it. Both didn't really seem like good options. I have not looked around for any examples, but would it be possible to add a php based file cron that would run periodically that would allow this ? That way we can avoid manually editing the config ?

What do you think about this ?

When user goes to page -> function will run manually when looking at any records for that zone -> adds them before grabbing the records -> records are populated in the ui. This is a potential example

plugins/dns/bind/src/opnsense/mvc/app/controllers/OPNsense/Bind/Api/RecordController.php

Line 60 in 598cf8c

public function searchRecordAction()

of how the search works. Would have to add something in general I think to kick it off ?

Cron -> syncs at a configurable time interval that can be set by the user?

Alternatively, rather than processing all the individual updates manually, we could just process the updated zone and delete any records from the opnsense config that are no longer in the zone

This could be a good way forward, however once we sync it with the config, it will move that record into the primary db. With the fix that i have here it will add it to the primary with syncing / freezing, that removes it from the journal . So always reading the journal as the might not be 100% i would have to check how that works.